17. 08. 2016
Events, NetEye, Real User Experience, Unified Monitoring
05. 10. 2016
Susanne Greiner
Events, NetEye, Real User Experience, Unified Monitoring
Congratulations to the winners of the NetCla Challenge
More than 100 Teams were competing, more than 25 sent in a solution, the best reaching a Macro-F1 scorse higher 0.88.
Last Friday, after six long weeks, the time had finally come. During ECML-PKDD conference at Riva del Garda the best of the competing approaches have been described and discussed. The participants had the possibility to get answers directly from the organizers and last but not least Iryna Haponchyk – leader of the winning team – was awarded 1000 Euro for the solution with the highest macro-F1 score, or better for having created a model capable of producing such a score. Here you can see the beaming winner during the discovery callenge prize ceremony.
Iryna explained her team trained a standard multi-class linear SVM classifier, having preliminarily enriched the presented attribute set with features generated using a random forest and features encoding the notion of interdependency between the examples that go close to each other in time.
Lei Xu, leader of the team placed second, handed in a solution based on an ensemble approach that consists of one main model, which performs a sequence of binary and multi-class classification to deal with the imbalance of target classes, and a couple of add-on ones, which further increase the accuracy of the work.
Martin Wistuba, leader of the team placed third, presented still a different solution. Their approach is based on state of the art methods for automatic machine learning hence no human intervention took place for preprocessing, feature engineering nor model parameter optimization.
So far so good, but what is the concrete achievement of the winners and are there any future prospects of the challenge outcomes?
Let me briefly summarize the task of the challenge:
Würth Phoenix S.rl. is continuously collecting network traffic data on request level. These data can be used for several tasks:
- Trend analysis to detect resource bottle-necks,
- essential improvement of end-user productivity,
- management awareness through SLA reports,
- proactive prevention of outages
just to mention a few.
The task of the NetCla challenge was to train a model on part of the data of a single average working day to be able to predict the remaining data of the very same working day.
Each request is characterized by various metrics, e.g. latencies and throughput measured by the probe. The objective of the challenge was, given a transmission in the network, to predict the application that is transmitting the data (a single label, multi-class classification task).
So now we know that it is possible to predict the application responsible for the request with a high accuracy. What can this knowledge be used for? For example checks on potential changes within the network (as soon as new traffic gets notably harder to predict, there might be a reason for that). Methods based on machine learning can be expected to be more sensitive to such changes than common practice methods that are merely based on the expected signal mean.
The challenge data were only taken from a single working day. Analogous data from different days should be use for training and testing respectively to gain a first estimate for the day specificity of the methods.
Herewith Würth-Phoenix S.r.l. congratulates the NetCla challenge winners, thanking them for their enthusiastic participation, and wishing them all the best for their future.
Susanne Greiner
Hi there! My name is Susanne and I joined Würth-Phoenix early in 2015. Ever since I can remember computers and the perfection that can be reached by them have been very fascinating for me. I built my first personal PC using components from about 20 broken ones at the age of 11 and fell in love with open source, visualization and data analysis shortly afterwards. I hold a master in experimental physics (University of Erlangen, Germany) and a PhD in computer science (Universtiy of Trento, Italy) my main interests are machine learning, visualization techniques, statistics and optimization. As long as an algorithm of mine runs at night and I get new interesting results the morning after I am able to sleep well. Beside computers I also like music, inline skating, and skiing.
Author
Latest posts by Susanne Greiner
21. 09. 2018
NetEye, Service Management
HackTheAlps Challenge with Würth Phoenix
04. 04. 2018
Anomaly Detection, Events, ITOA, NetEye
Würth Phoenix @ GrafanaConEu 2018
27. 03. 2018
Anomaly Detection, ITOA, NetEye, Visual Synthetic Monitoring
Multi-Level Dashboarding with Grafana – Use Case: NetEye ITOA | Alyvix
13. 11. 2017
NetEye
Deep Learning – a Recent Trend and Its Potential