23. 12. 2024
APM, Development, Log-SIEM, NetEye
In one of my previous posts I mentioned the importance of downtime scheduling and shared an interesting example of a PowerShell script for managing downtime in Windows environments.
Recently a customer asked me how to manage downtime with a similar solution, but for several hundred Linux servers monitored under NetEye. The reason is very simple: security updates and special maintenance are increasingly frequent, cannot be scheduled in a fixed manner, and a reboot is necessary at the conclusion of this type of operation. Therefore it’s necessary to use downtime in an automatic way via script directly from the rebooted host.
In this scenario it’s not possible to use the Scheduled Downtimes feature available in the Director module. But it is possible to define a simple script to be executed in an automatic restart procedure before the reboot is actually performed ‒ to guarantee that the host involved is in downtime on the NetEye system including all of its service checks.
The bash script that I share below is organized as follows: in the first part I initialize some variables to automatically identify the hostname, create the start date STARTDATE and the end date ENDDATE (with a margin of 20m, as the expected time for the conclusion of the reboot), define a comment and the author of the downtime. Subsequently a PARAMETERS variable is defined which allows you to collect all the parameters together in JSON format in order to perform the downtime on the host on the NetEye system.
Finally, through the curl command, the script authenticates itself with the NetEye Master host using a dedicated API user, contacts the action schedule-downtime, and applies the downtime both to the host and to all its service checks using the “all_services”:true option.
Here’s an example script:
export HOST=`hostname`
export STARTDATE=`date +%s`
export ENDDATE=`date +%s -d "+20 minutes"`
export COMMENT="autodowntime_scheduled_after_update"
export AUTHOR="autodowntime"
PARAMETERS='{"type":"Host","filter":"match(\"'$HOST'\",host.name)","all_services":true,"fixed":true,"start_time":"'$STARTDATE'","end_time":"'$ENDDATE'","author":"'$AUTHOR'","comment":"'${COMMENT}'","pretty":true}'
curl -k -s -u autodowntime:XXXXXXXXXXXX -X POST "https:/neteye-master.my.domain:5665/v1/actions/schedule-downtime" -H 'Accept: application/json' -d ''$PARAMETERS''
The script needs a dedicated API user enabled to run downtime. This user must be defined in the file /neteye/shared/icinga2/conf/icinga2/conf.d/api-users.conf as shown in the following example:
# cat /neteye/shared/icinga2/conf/icinga2/conf.d/api-users.conf /**
object ApiUser "autodowntime" {
password = "XXXXXXXXXXXX"
permissions = [
"objects/query/Host",
"objects/query/Service",
"objects/query/Downtime",
"actions/schedule-downtime",
"actions/reschedule-downtime",
"actions/remove-downtime",
"events/*",
"status/query",
]
}
These Solutions are Engineered by Humans
Did you find this article interesting? Does it match your skill set? Our customers often present us with problems that need customized solutions. In fact, we’re currently hiring for roles just like this and others here at Würth Phoenix.
Giuseppe Di Garbo
Consultant at Würth Phoenix
Hi everybody. I’m Giuseppe and I was born in Milan in 1979. Since the early years of university, I was attracted by the Open Source world and operating system GNU\Linux. After graduation I had the opportunity to participate in a project of a startup for the realization of an Internet Service Provider. Before joining Würth Phoenix as SI consultant, I gained great experience as an IT consultant on projects related to business continuity and implementation of open source software compliant to ITIL processes of incident, change and service catalog management. My free time is completely dedicated to my wife and, as soon as possible, run away from Milan and his caotic time and trekking discover our beautiful mountain near Lecco for relax and lookup the (clean) sky.
Author
Latest posts by Giuseppe Di Garbo
20. 12. 2024
Atlassian, NetEye, Service Management
Managing Alerts with JSM: Focus on Incident Management (Part 2)
22. 11. 2024
Atlassian, Service Management
Managing Alerts with JSM: Focus on Incident Management (Part 1)
27. 06. 2024
ITOA, NetEye, Unified Monitoring
Streamlining Icinga Director with Grafana
09. 05. 2024
NetEye
How to Customize NetEye Reports for Your Brand