20. 12. 2024
Atlassian, Development
Distributing software content and making it available to users is not an easy task.
First, you can have many different types of content like RPMs, ISO files, Python packages, and so on and so forth.
And then, repositories often change. In the R&D team, we’re always implementing new features and fixing bugs, which in other words means that we are continuously releasing new content that must be well organized, pushed to dedicated repositories, and accessible by the end user. Furthermore, the number of daily tasks that are required to have a solid content management pipeline is quite high: this requires implementing automated tasks to carry out boring and repetitive operations when needed.
To fulfill these requirements, and with the goal of modernizing our repository management platform, we decided to migrate to Pulp.
According to its official website, Pulp:
- is a platform for managing repositories of software packages and making them available to a large number of consumers
- can locally mirror all or part of a repository
- can manage many types of content from multiple sources in one place
- has a REST API for management
- is free and open-source
So, at first glance, it seemed the right and obvious choice for us.
Was it for real? Two months after its preliminary introduction, I can answer: “Yes, it is”. Here’s our story.
Installing Pulp
I will not spend too much time talking about this: the installation of the platform went quite smoothly, we followed the procedure documented in the official User Guide of the tool with very little customization. No problem at all.
Mirroring Our Existing Repositories
We started mirroring our existing repositories within Pulp. This is one of the most common scenarios which Pulp is used for, and it’s also included among the recipes on the official website. For simplicity, I’ll use the pulp-admin client to show the various commands to run to achieve the mirroring, but you can obtain the same results by querying the REST API.
Imagine we want to mirror the existing NetEye 4.13 repository, hosted at http://ourpreviousrepos.com/repos/neteye-4.13.
First, we can create the repo:
$ pulp-admin rpm repo create --repo-id=neteye-4.13 --feed=http://ourpreviousrepos.com/repos/neteye-4.13
Successfully created repository [neteye-4.13]
As you’ve noticed, we pass two parameters to the command:
--repo-idrepresents the unique ID of the repository within Pulp, and is mandatory;--feedis required if we want to sync the content of our repo from an existing one, which corresponds to our scenario.
Pulp is nice enough to inform us of the successful creation of the repository, so we can proceed to sync the content by typing:
$ pulp-admin rpm repo sync run --repo-id=neteye-4.13
The command triggers a Pulp async task that downloads the content of the existing repository, adds the content to the Pulp repo, creates the metadata, and publishes the repo via HTTPS. To keep the repository up-to-date with the feed counterpart, synchronization tasks can be scheduled when needed. Pulp will take care of executing them.
If the above task goes well, we now have a fully functional mirror of our existing NetEye 4.13 repository. Cool right?
This is even cooler if you think about the following scenario:
ACME, a NetEye customer, wants to keep its NetEye installation always up-to-date. Since they are super paranoid about security, they want to mirror the official NetEye repositories within their infrastructure. So, they install Pulp on a host that is only allowed to reach the NetEye official repositories to download their content, thus reducing the potential attack surface as much as possible. At this point, mirroring the desired repos is just a matter of a few clicks.
I’ll stop here for this episode; stay tuned for the next in my series about Pulp.
Further Reading and Resources
- A very interesting presentation on Pulp from Michael Hrivnak of Red Hat
- The video of the presentation
Andrea Avancini
DevOps Engineer at Würth Phoenix
Loving understanding of how things work, how things can be automated, and how to apply new technologies when needed. Passionate about technology, open-source software, and security. I found Würth Phoenix the right place for this.
In the past, I co-founded a cybersecurity startup that produces security solutions for mobile apps and blockchain.
Previously, I worked as researcher at Fondazione Bruno Kessler of Trento. My research was mainly focused on web and mobile app security and testing.
I got my PhD in Computer Science at the University of Trento.
Author
Latest posts by Andrea Avancini
18. 10. 2021
NetEye
New Feature for NetEye 4.20
01. 10. 2021
Business Service Monitoring, NetEye
Hosts and NetEye Upgrade