06. 11. 2024
AI, Log-SIEM, Machine Learning, NetEye
15. 03. 2024
Matteo Cipolletta
APM, Log-SIEM, NetEye
Unleashing Elastic APM: Containerized Scalability Explored
Introduction: Unveiling Elastic APM in Containerized Environments
In today’s dynamic digital landscape, where every interaction matters, understanding the intricacies of application performance has become paramount. Elastic APM is a powerful toolset within the Elastic Stack included in the NetEye SIEM Module, and designed to provide unparalleled insights into the performance of your applications. As organizations increasingly adopt containerized environments for their scalability, flexibility, and efficiency, integrating Elastic APM into these ecosystems emerges as a strategic move to ensure seamless observability.
Elastic Observability: The Key to Insightful Analysis
Elastic Observability represents a comprehensive approach to understanding the performance and behavior of your applications and infrastructure. It encompasses various tools and features within the Elastic Stack, including metrics, logs, and traces, seamlessly integrated to offer a holistic view of your entire environment. At its core lies Elastic APM (Application Performance Monitoring), a specialized solution tailored to dissect the intricacies of application performance with precision.
Unlocking the Power of APM
Elastic APM empowers developers and operations teams to gain deep insights into the inner workings of their applications. By capturing crucial metrics such as response times, error rates, and throughput, APM enables stakeholders to identify performance bottlenecks, optimize resource utilization, and enhance the user experience. Moreover, with distributed tracing capabilities, APM provides a detailed map of interactions between various components of modern, microservice-based architectures, facilitating effective troubleshooting and optimization.
Containerization: Revolutionizing Deployment
In the realm of modern infrastructure management, containerization has emerged as a game-changer. By encapsulating applications and their dependencies into portable, lightweight containers, organizations can achieve unprecedented scalability, efficiency, and agility. Containers allow for rapid deployment, seamless scaling, and consistent environments across diverse infrastructure platforms, making them an ideal choice for modern, cloud-native architectures.
Containerizing Elastic APM: Scaling Horizontally with Ease
In this blog post, we delve into the convergence of Elastic APM and containerization, exploring how leveraging containers can enhance the scalability and flexibility of APM deployments. Specifically, we’ll focus on deploying multiple instances of Elastic APM agents running APM servers within containerized environments. By harnessing the power of containers, organizations can effortlessly scale their APM infrastructure horizontally to meet the evolving demands of their applications, all while maintaining optimal performance and resource efficiency.
The Problem
Suppose you’re already using a Fleet Policy in Kibana to manage an APM Server integration running on your NetEye:
As you can see from the screenshot above, this integration sends data to the namespace namespace-1. As you can imagine this solution isn’t very scalable since you may want to configure other services that need to send data to different namespaces (say, namespace-2) to segregate data and provide access to different groups of Developers.
However at the time of this writing, Elastic in NetEye is at v8.11.3, which does not allow multiple APM Server integrations in the same Fleet Policy.
The Solution
To resolve this problem in practice, we need to deploy an Elastic Agent Container, managed by the Fleet Server, on which we can deploy another Fleet Policy with the APM Server Integration that will send data to namespace-2.
The architecture
The following schema presents a possible architecture to deploy an Elastic Agent Container:
The APM Server on the container will, obviously, listen on a different port.
Before deploying the container we need to prepare the second Fleet Policy with the APM Server Integration:
Once we’ve prepared the integration, we can pull the Elastic Agent image provided by Elastic using podman:
podman pull docker.elastic.co/beats/elastic-agent-complete:${ELASTIC_VERSION}
Then we need to deploy the container and enroll the Elastic Agent into our Fleet Server by following the Elastic Documentation. Finally you can start to deploy Elastic APM Agents in your applications.
Please remember to open the respective firewall ports and enable traffic from your applications to NetEye on the ports needed for allowing data collection.
These Solutions are Engineered by Humans
Are you passionate about performance metrics or other modern IT challenges? Do you have the experience to drive solutions like the one above? Our customers often present us with problems that need customized solutions. In fact, we’re currently hiring for roles just like this as well as other roles here at Würth Phoenix.
Matteo Cipolletta
I'm an IT professional with a strong knowledge of Security Information and Event Management solutions.
I have proven experience in multiple Enterprise contexts with managing, designing, and administering Security Information and Event Management (SIEM) solutions (including log source management, parsing, alerting and data visualizations), its related processes and on-premises and cloud architectures, as well as implementing Use Cases and Correlation Rules to enable SOC teams to detect and respond to cyber threats.
Author
Latest posts by Matteo Cipolletta
18. 10. 2024
Log Management, Log-SIEM, NetEye
Offloading Data Enrichment to Satellite Machines with Logstash
14. 06. 2024
APM, NetEye, Real User Experience, Visual Synthetic Monitoring
The Right Monitoring Tool: Elastic Synthetic Browser Monitor vs. Alyvix