Publication
Monitoring and detection of anomaly in microservices environments
| Summary: | Microservices architectures have become increasingly popular in recent years because of their scalability and agility. However, the distributed nature of this architecture also introduces some challenges, especially in terms of monitoring and detecting anomalies. Anomaly detection is the process of identifying anomalous events or patterns in data that do not conform to expected behavior. In microservices environments, this eventually becomes very important, since the number of services tends to grow increasingly, making the interaction between them complex. Because it is recent, there are still few studies on the best approaches to detecting anomalies in microservices. This thesis investigates how well PyOD library algorithms can detect anomalous behavior in a microservices dataset. PyOD is an open-source Python toolbox for performing scalable outlier detection on multivariate data. Some benefits of PyOD are that it is scalable, includes several algorithms, and can detect anomalies in multivariate data. We also review among the PyOD, KNN and HBOS algorithms, which one performs better at detecting anomalies. To evaluate the approach, we used TraceRCA dataset to detect anomalies such as application bugs, CPU exhausted, and network jam. This dataset contains logs from a real microservices system. The preliminary results show that the HBOS algorithm performs better than kNN, with Recall and F1-Score of 83% and 91%, respectively, while for kNN these metrics were 80% and 89%, respectively. |
|---|---|
| Main Authors: | Landim, Lauriana Patricia Tavares |
| Subject: | Microservices Monitoring Anomaly detection Py0D Outliers algorithms Microserviços Monitorização Deteção de anomalias Py0D Algoritmos outliers |
| Year: | 2023 |
| Country: | Portugal |
| Document type: | master thesis |
| Access type: | open access |
| Associated institution: | Instituto Politécnico de Castelo Branco |
| Language: | English |
| Origin: | Repositório Científico do Instituto Politécnico de Castelo Branco |
| Summary: | Microservices architectures have become increasingly popular in recent years because of their scalability and agility. However, the distributed nature of this architecture also introduces some challenges, especially in terms of monitoring and detecting anomalies. Anomaly detection is the process of identifying anomalous events or patterns in data that do not conform to expected behavior. In microservices environments, this eventually becomes very important, since the number of services tends to grow increasingly, making the interaction between them complex. Because it is recent, there are still few studies on the best approaches to detecting anomalies in microservices. This thesis investigates how well PyOD library algorithms can detect anomalous behavior in a microservices dataset. PyOD is an open-source Python toolbox for performing scalable outlier detection on multivariate data. Some benefits of PyOD are that it is scalable, includes several algorithms, and can detect anomalies in multivariate data. We also review among the PyOD, KNN and HBOS algorithms, which one performs better at detecting anomalies. To evaluate the approach, we used TraceRCA dataset to detect anomalies such as application bugs, CPU exhausted, and network jam. This dataset contains logs from a real microservices system. The preliminary results show that the HBOS algorithm performs better than kNN, with Recall and F1-Score of 83% and 91%, respectively, while for kNN these metrics were 80% and 89%, respectively. |
|---|