In a previous blog post, I described how Elastic Stack fits within the High-Available cluster architecture of NetEye 4 and, in particular, how the correct configuration of the Quorum is mandatory to prevent losing your data or even developing inconsistencies.
With the upgrade to NetEye 4.8, we updated Elastic Stack to the new major version 7. This new version introduces a significant change in the management of the Elastic Stack Quorum.
In versions of Elastic Stack prior to 7, forming an Elastic Stack cluster required some manual configuration not only to discover nodes (discovery.zen.ping.unicast.hosts), but also to avoid critical situations such as split-brain conditions or data inconsistencies (discovery.zen.minimum_master_nodes). These configurations were mandatory and had to be continuously updated to reflect the current status of the architecture (e.g., if you decide to add a new master node, you must adjust minimum_master_nodes to avoid split-brain situations).
Starting with Elastic Stack 7, this configuration process was greatly simplified, so that you could form a cluster out-of-the-box without any special configuration. However, the basic configuration is not recommended for production systems.
For this reason, since NetEye 4.8 we used new configuration APIs to properly deploy safe and reliable Elastic Stack clusters:
discovery.seed_hosts: a list of nodes which a new node needs to contact the very first time when forming or joining a cluster
initial_master_nodes: this is used to ensure that before forming a cluster, all nodes listed in this configuration must be connected
As soon as the cluster is formed, Elastic Stack will take care of determining the number of master nodes, which will then take care of maintaining and securing the cluster. Thus all those configurations are automatically maintained and must not be adapted even if you want to scale up your master nodes.
Please note that manual configuration is still necessary if you want to remove pre-existing master nodes, by informing your Elastic Stack cluster that a node is going to be removed.
Hi, my name is Michele Santuari and I am a Telecommunication engineer felt in love with OpenFlow, the first attempt of centralized network management, provisioning, and monitoring. I embraced the Software Defined Networking approach to discover a passion for programming languages. Now, I am into Agile methodologies and crazy development process management.
Author
Michele Santuari
Hi, my name is Michele Santuari and I am a Telecommunication engineer felt in love with OpenFlow, the first attempt of centralized network management, provisioning, and monitoring. I embraced the Software Defined Networking approach to discover a passion for programming languages. Now, I am into Agile methodologies and crazy development process management.
In order to be able to carry out detailed network monitoring, an IT administrator naturally wants to know what is happening in his or her network. To obtain this information, the network flows must of course be analyzed. Many network Read More
In this blog, I'll propose and describe a solution for detecting potential infostealers targeting Chromium-based browsers, taking a cue from the research exposed by Google's Chrome Security Team (Detecting browser data theft using Windows Event Logs). Obviously a solution using Read More
Recently (in September 2023) NetEye integrated version 8.8 of the Elastic Stack, which is just one of many Elastic updates brought into NetEye 4. Since this Elastic update there was a major upgrade (from version 7.17) coming with many breaking Read More
The Fleet Management feature was automatically enabled with NetEye release 4.30, and with the current 4.31 version all the Elastic Stack packages will be upgraded to major version 8. These two milestones will permit us to centrally manage log ingestion Read More
Anyone who has joined the beautiful world of logging has collided, sooner or later, with the collection via syslog protocol. More than 40 years have passed since syslog was invented, and in that time there have been several attempts by Read More