18. 03. 2025 Rocco Pezzani Icinga Web 2, ITOA, NetEye, UI, Unified Monitoring

A First Step towards Multitenancy in Icinga 2

Our older customers surely know that, in its earliest releases, NetEye 4 had no support for multitenancy. From a data perspective, there’s just one big bin we throw everything into to be used later. Then, a debate about multitenancy began.

Here at Würth Phoenix we’ve discussed at length about the best way to segregate data without impacting too much on usability, performance and overall costs of the NetEye 4 Platform, and we’re still discussing it to this day. All of us walk down the path of multitenancy knowing this is a long journey, and that sometimes the road may become a mountain that we’ll have to overcome sooner or later.

One of the biggest mountains is at the very core of NetEye 4: Icinga 2 does not have true support for multitenancy. Grafana restricts access and privileges at the Organization level, and something similar can be done with GLPI at the Entity level; Elasticsearch implements document-level security, while InfluxDB uses a more complex layout to separate external metrics in different databases.

But for Icinga 2 it’s a completely different story. Monitored Objects, their status and their whole history are all stored in a single database on MariaDB, and all generated performance data is stored in a single database on InfluxDB. Usually, this is not a problem, because:

It’s possible to define filtering on Icinga Web 2 Roles to limit access to Monitored Objects
Icinga Web 2 shows performance data about the Monitored Object you’re viewing

But, what happens if you need a more complex visualization? If you’re not satisfied with the Dashboards Icinga Web 2 provides, you have to switch to Grafana: Grafana Dashboards like NetEye Problems (see it on NetEye Demo) or more complex visualizations are cool and useful, but they require direct access to the Icinga 2 database on MariaDB through a dedicated Data Source.

As a result, a person who can access a Grafana Organization containing such a Data Source can easily browse all the contents of that database, gaining access to the data of other customers. Therefore, using Grafana Dashboards with the Icinga 2 database has been marked as “not safe” (or, “use at your own risk”, if you prefer).

Recently, some NetEye Cloud Customers asked to have their own version of the NetEye Problems dashboard, so I was tasked with finding a solution. And surprisingly, it came easily, quickly and elegantly. Nothing short of a miracle.

The Egg of Columbus

For the sake of conversation, let’s use the name IDO for the Icinga 2 database on MariaDB.

IDO stores information about all Monitored Objects: everything you can see in Icinga Web 2 is stored in the IDO, from the name of a Host Object to notifications sent to a user. Icinga 2 connects to one or more IDOs using one or more IdoMySqlConnections, one for each IDO to connect to. As you can see, there is no filtering option for an IdoMySqlConnection, so all objects managed by Icinga 2 will inevitably be dumped into the connected IDO.

Most of us (me included) tried to find a way to divide data into multiple IDOs, but this is indeed impossible. Then, I suddenly remembered a feature I used when playing with some DBMS’s. From JOIN Syntax – MariaDB Knowledge Base:

Each table can also be specified as db_name.tabl_name. This allows to write queries which involve multiple databases.

Then, the idea: what happens if, in a database, I create a VIEW that gets data from a different Database? Something like this:

CREATE DATABASE <tenant_id>_icinga;
CREATE USER '<tenant_id>_ro'@'%' WITH IDENTIFIED BY '<password>';
GRANT SELECT ON <tenant_id>_icinga.* TO '<tenant_id>_ro'@'%';
FLUSH PRIVILEGES;
USE <tenant_id>_icinga;

CREATE VIEW 'icinga_hosts' AS
SELECT *
FROM icinga.icinga_hosts
WHERE <something that can filter a Tenant>;

I now have a database that contains only the VIEWs I want: accounts which have been granted solely the SELECT privilege on that database cannot go anywhere else in the DBMS. And, if I can find that <something that can filter a tenant> for the WHERE clause, I’ll hit the jackpot: all VIEWs I define in this database can return data pertaining to a certain tenant and nothing more, just as expected.

How to Filter a Tenant

In 2024, we added multitenancy support on Alyvix together with a Web Interface for Alyvix on NetEye. To better integrate Alyvix and NetEye 4, a strong bond between a Host Object and its Alyvix Tenant needs to be established.

This bond has been created by our developers by defining a dedicated Custom Variable on all Host Objects named neteye_tenant. This variable can contain the name of a Tenant that has been created using the neteye tenant config create command and nothing else. This means each Host Object can have the exact name of the NetEye Tenant it is assigned to.

Originally, this Custom Variable was made available only for NetEye’s Alyvix module, but we foresaw its potential and released it for NetEye Core . As a result, I can now build a strong rule that can be used to filter objects by Tenant. Now, I can complete the WHERE clause of my CREATE VIEW statement:

CREATE VIEW 'icinga_hosts' AS
SELECT *
FROM icinga.icinga_hosts
WHERE host_object_id in (
    SELECT object_id
    FROM icinga_customvariables
    WHERE varname = 'neteye_tenant' and varvalue = <tenant_id>
);

And, as you can imagine, it works as designed.

Completing the Job

Now that the basic idea has been defined, the whole thing can be completed by adding a VIEW for each table required by my Grafana Dashboards. And, since I maintained the same name as the Source Tables, NetEye Problems worked from the beginning without a single update. Lucky me, I remembered cross-db queries and neteye_tenant.

Some Considerations

Now I have some final considerations: querying a VIEW will inevitably result in a READ LOCK on the Source Table. For tables that are relatively small, this should not pose a problem, even if they are updated multiple times per second. Everything changes when you need access to larger tables, like icinga_history: on big systems, this table can easily have up to several million rows, requiring some time for a READ LOCK to release. This might not be a good thing, so I suggest to avoid creating a VIEW for very large tables.

Also, many of you might say: IDO is old, has been replaced by IcingaDB, and NetEye itself has to be updated. Yes, you’re right: NetEye 4 will move to IcingaDB without doubt. But looking at the existing Icinga Documentation it doesn’t seem to me that it can support data segregation by Tenant. Therefore, this approach will most likely remain unchanged.

One last thing: we’re in the process of conducting some performance tests with large numbers of tenants. Once completed, you’ll be able to have this feature directly in your NetEye 4 on-premises setup. Stay tuned for news.

These Solutions are Engineered by Humans

Did you find this article interesting? Does it match your skill set? Our customers often present us with problems that need customized solutions. In fact, we’re currently hiring for roles just like this and others here at Würth Phoenix.