02. 10. 2014 Thomas Forrer Log Management, Log-SIEM, NetEye

NetEye: Integration Logstash/Elasticsearch/Kibana

From the logs to Kibana

From the logs to Kibana

You probably already heard about Elasticsearch and its potential. Elasticsearch is a full-text search engine based on Lucene. It provides a RESTful web interface and schema-free JSON documents. To be able to better display logs collected by NetEye, we integrated three open source projects: Logstash, Elasticsearch and Kibana.

Logstash parses logs and submits them to Elasticsearch, which saves them in a structured way. Finally, Kibana takes the role of displaying all the collected data within NetEye Syslog View.

With Logstash and Elasticsearch logs can be parsed in real time, providing us the possibility to see live logs parsed and filtered as they come:

NetEye Logs parsed in real time

Logs parsed in real time

Additionally, with the new Kibana 3 Frontend NetEye users can easily create a multitude of useful dashboards, aggregating the data coming from logs in order to display very interesting statistics.

You can generate your individual dashboards defining settings according to your business’ needs.

NetEye Kibana Create your personal dashboard

Create your personal dashboard

The example below shows a dashboard displaying the total count of users that have used a particular program of the MS Office suite through citrix (this can be useful to determine the number of effectively needed licenses).

NetEye Kibana Example: General Citrix usage by Application

Example: General Citrix usage by Application

Another example could be a dashboard showing the statistics about accesses to websites on your local webserver:

NetEye Kibana Example: Website Access

Example: Website Access

Thomas Forrer

Thomas Forrer

Team Leader Research & Development at Würth Phoenix
Hi folks! I began loving computer since 1994, it was still the time of windows 3.1. Immediately I learned starting DOS games from the command promt, and while typing some white text on black background I felt like some hackish dude in a hollywoodian movie. Later during the studies at the university, I discovered the magic world of opensource, and it was love at first sight. Finally I got rid of BSOD's =) I love everything that is connected to some network, especially in a security perspective. My motto is: "With motivation, nothing is impossibile. It only requires more time."

Author

Thomas Forrer

Hi folks! I began loving computer since 1994, it was still the time of windows 3.1. Immediately I learned starting DOS games from the command promt, and while typing some white text on black background I felt like some hackish dude in a hollywoodian movie. Later during the studies at the university, I discovered the magic world of opensource, and it was love at first sight. Finally I got rid of BSOD's =) I love everything that is connected to some network, especially in a security perspective. My motto is: "With motivation, nothing is impossibile. It only requires more time."

4 Replies to “NetEye: Integration Logstash/Elasticsearch/Kibana”

  1. Sebastian says:

    Dear Thomas,

    I struggled across this blog looking for experiences about reporting a Citrix site with the tools from the Elastic ELK stack.

    Specially the screenshot about the application usage looks very interesting.
    Would it be possible to share some more details about this setup?
    Which logs did you query to build the dashboard?
    Which was the Citrix version behind?

    Regards
    S.

    1. Hi Sebastian,

      the events are taken from the Windows Event Log, and are the “PROCESS STARTED” events (see https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/event.aspx?eventID=4688). The version of Citrix doesn’t matter in this case, because by identifying the right process name we can detect which users opens the applications exposed by citrix and create this nice dashboards. To forward the windows events I used our Safed Agent (You can find info about Safed in the Download section of this blog).

      I hope I answered your questions.

  2. David says:

    I am a kibana novice. thanks for sharing.
    See your last example “trends respect to last day”.
    I really want to know how you calculate the rate of change, and with the rate of change of the positive and negative to the corresponding figures with the picture.

    1. Hi David,

      this post was talking about Kibana 3 and Elasticsearch version 1.x.
      Now the Elastic Stack has grown to Version 6, and the widget that you mentioned was deprecated in new versions.

      You can achieve a similar result with the timelion plugin for Kibana for example with moving averages.
      Give a look here:

      https://www.elastic.co/guide/en/kibana/current/timelion-conditional.html

      Kind regards

Leave a Reply

Your email address will not be published. Required fields are marked *

Archive