25. 09. 2023
Icinga Web 2, NetEye, Unified Monitoring
02. 07. 2021
Enrico Alberti
Icinga Web 2, NetEye
Monitor Services Automatically with Icinga Director Automation
In recent months many enterprise customers have asked me for a solution to create services on their monitored Icinga hosts automatically. They want a single point of insertion (like a list) for all services on the host as well as for the NetEye system to do all the work managing their creation and deletion.
How we can approach this use case?
IDEA: We can use the custom variables and properties defined on a Host Object and use their values together with the Director Automation feature to manage services’ lifetimes.
The Director Automation module works with the Import Source and Sync Rule feature, which allows fetching data from various data sources (internal/external), manipulating it, and then performing actions (create/modify/delete) on any Icinga Objects.
A simple example of applying this use case is the automatic monitoring of specific running services or processes on a Windows or Linux host. To reach our goal we divide the problem into 3 steps:
1. Create and populate Custom Variables
First of all, we need to create the variables that the user can populate as properties of the Host Object, taking into consideration the elements necessary for the creation of the service.
The main elements are:
- host_os it’s really important to distinguish which kind of OS host we are working on (e.g. Linux or Windows), because the command used is different
- automatic_services_list in this array variable we put all the names of services that we want to monitor
- automatic_processes_list in this array variable we put all the names of processes that we want to monitor
2. Create the Import Sources
We can use the SQL Import source directly on Director DB to extract all the instances from the variables “automatic_*_list” and the related Host OS that we populated in the step before.
HINTS:
- To get the correct key for the table, we concatenate the service name and the host object
- We need two different import sources to manage both the services and processes list
3. Create the Sync Rules
The last main step is to create the Sync Rules, which are related to the Import Sources above, in order to automatically manage all the services on each host object. The most important thing is to correctly choose the Import Template that will be used for monitoring the service.
The properties needed for the service are:
- object_name the name of the service (e.g. Automatic Service – NAME_OF_THE_SERVICE)
- host the destination host of this specific service
- import the Service Template used for monitoring — this property is set based on the value of the Host OS (e.g. if
host_os=linuxthen apply the template that checks Linux services, or ifhost_os=windowsthen apply the template that checks Windows services)
HINTS:
- We configure the policy with
override updateandpurgeset to YES, so that when the variable values on Host Object change the service, it’s modified accordingly.
Now when running the Import Sources and the Sync Rules, services are automatically created in Director. The Director’s activity log shows precisely what changes are waiting to be deployed to Icinga. Clicking on each element displays an exact diff of the old and new configurations. Thus the user always has control over the infrastructure impact of Rules.
If all seems good, we just need to deploy the configuration to Icinga and all our services will be automatically monitored.
TIPS
This approach can be extended to multiple use cases, for example to monitor different disk partitions on hosts, or to monitor the multiple interfaces on a network switch (very useful).
To make the automation even more automatic, you can schedule the Job to run every X minutes via Import and Sync.
Useful link:
https://icinga.com/blog/2019/04/23/monitoring-automation-with-icinga-the-director/
https://icinga.com/docs/icinga-director/latest/doc/70-Import-and-Sync/
Enrico Alberti
I’ve always been fascinated by the IT world, especially by the security environment and its architectures.
The common thread in my working experience is the creation of helpful open-source solutions to easily manage the huge amount of security information.
In the past years, my work was especially focused on Cyber Kill Chain, parsing and ELK Stack but in order to start from the beginning...
In 2010 I left my birthplace, the lovely Veneto, looking for a new ´cyber´ adventure in Milan. After graduating in Computer Systems and Networks Security, I worked for 6 years as a Cyber Security Consultant.
During the first 5 years, I explored the deep and manifold world of cybersecurity, becoming passionate about open source solutions. After that, I decided to challenge myself joining a Start-up company focusing on SOC services (I’m a proud member of the Blue Team!).
In Wuerth Phoenix, I would like to personalize the NetEye System for each one of our costumers, in order to develop the perfect product for their needs, by combining all my past experiences and skills.
Author
Latest posts by Enrico Alberti
28. 12. 2023
Log Management, Log-SIEM, NetEye
Monitor Fleet Elastic Agents with NetEye Extension Packs (NEP)
28. 10. 2022
Log Management, Log-SIEM, NetEye
Syslog Collection with Elastic under Distributed NetEye Monitoring
19. 07. 2022
Contribution, NetEye, Unified Monitoring
Integration of Centreon Plugins into NetEye Extension Packs