08. 07. 2019
Andrea Avancini
Business Service Monitoring, NetEye
ISO Generation with Ansible – Part 2
As mentioned in the previous post of this series, at Würth Phoenix we build our NetEye ISOs from scratch every night to be sure that every morning everything will be fresh.
To perform this activity with no manual intervention in a robust, repeatable, and reliable way, the R&D team uses Jenkins, a well known open source tool and also one of the cornerstones of our Continuous Integration process.
To automate the build process of our ISO, we implemented a Jenkins pipeline that is responsible for creating, testing and deploying the ISO as depicted in the following screenshot. I will describe the main stages of the pipeline in the following sections.
Creating the ISO
Problem: How can we automatically build a NetEye ISO?
Our workhorse for the ISO creation is Ansible. In fact, our ISO builder is an Ansible playbook that calls several Ansible roles.
One might wonder: Why Ansible? Yes, Ansible is open source, extendable, well documented, both in terms of official documentation and online resources, and with a large community of users and contributors.
But, is Ansible able to give the user the flexibility to implement a task like creating an ISO?
The answer is yes. And the proof is that we are able to build our NetEye ISO daily with zero hassle.
The Ansible playbook is invoked by Jenkins any time a build of an ISO is required. If we modify the playbook source code, Jenkins is also responsible for running our Molecule test suite in order to verify the correctness of our Ansible code.
So let’s assume we didn’t break anything with our last Ansible commit: then Jenkins can start the build of the ISO by actually calling our Ansible code. The creation stage produces the .iso file, but we still have to verify if that ISO can pass our quality check.
Testing the ISO
Problem: How can we automatically create a virtual machine, mount the ISO on it, verify that the installation of NetEye worked, and that NetEye itself works, all from Jenkins?
Testing the ISO means verifying that:
- The NetEye installation process completed without errors
- The NetEye installed from the ISO works as expected
While digging around to find some tool that could do what needed, we came across Packer, an open source tool for creating machine images for various platforms, starting from a single configuration file (see the example below). We easily integrated it into our Jenkins pipeline.
With Packer, we automatically create a virtual machine that is based on the NetEye ISO we want to test. If VM creation is successful, meaning that the installation process of NetEye completed without errors, Packer runs our end2end test suite to verify that the version of NetEye just installed on this VM is working as expected. Below you can see a part of our configuration file for Packer.
{
"provisioners": [
{
"type": "ansible",
"playbook_file": "provisioners/ansible/setup_neteye.yml"
},
{
"type": "shell",
"script": "provisioners/scripts/run_tests.sh"
}
],
"builders": [{
...
"iso_url": "/tmp/neteye4.6-centos7.stable.iso",
...,
"disk_size": "{{user `disk_size`}}",
"headless": "{{user `headless`}}",
"http_directory": "http",
"boot_wait": "5s",
"boot_command": [
"<enter>"
],
...
"ssh_timeout": "{{user `ssh_timeout`}}",
...
}],
"variables": {
"cpus": "<num-cpus>",
"disk_size": "<disk-size>",
"headless": "true",
"memory": "<memory>"
}
}
If Packer says everything is fine, the Jenkins pipeline proceeds to the deployment stage.
Deployment of the ISO
Having reached this point in our Jenkins pipeline, we know for sure that our new ISO has passed our quality checks and thus can be deployed to be used internally and also externally by our customers. But there is no magic here. With Jenkins, we just upload the ISO on our servers, ready to be downloaded by our customers.
Andrea Avancini
DevOps Engineer at Würth Phoenix
Loving understanding of how things work, how things can be automated, and how to apply new technologies when needed. Passionate about technology, open-source software, and security. I found Würth Phoenix the right place for this.
In the past, I co-founded a cybersecurity startup that produces security solutions for mobile apps and blockchain.
Previously, I worked as researcher at Fondazione Bruno Kessler of Trento. My research was mainly focused on web and mobile app security and testing.
I got my PhD in Computer Science at the University of Trento.
Author
Latest posts by Andrea Avancini
18. 10. 2021
NetEye
New Feature for NetEye 4.20
01. 10. 2021
Business Service Monitoring, NetEye
Hosts and NetEye Upgrade