Publication
Improving Intrusion Detection Systems: Challenges with Public Datasets and the Role of Explainable AI: A Practical Guide Using NFS-2023-TE and HIKARI-2021
| Summary: | Intrusion detection systems (IDS) based on public datasets often show promising results in academic papers but fail to perform effectively in real-world scenarios due to flaws in dataset creation. This discrepancy raises the question of whether explainable algorithms should be essential when building machine learning-based intrusion detection systems. While recent research has highlighted some deficiencies in these studies, new datasets have emerged, and practical guides on addressing these issues are lacking. This thesis extends previous work by evaluating improvements in recent datasets and providing new insights based on our findings. Our study reveals persistent issues in existing datasets and demonstrates, through explainable AI techniques, why building intrusion detection systems with these datasets should be approached with caution. By contributing guidelines on what to avoid when developing an intrusion detection system, we also illustrate how certain aspects of the process require deeper analysis before proposing new models. This research underscores the critical need for more robust and representative datasets in IDS development, paving the way for more reliable and practical cybersecurity solutions. |
|---|---|
| Main Authors: | Ludovico, Toscano |
| Subject: | intrusion detection explainable AI NIDS public dataset HIKARI-2021 NFS-2023-TE Cybersecurity SDG 8 - Decent work and economic growth |
| Year: | 2024 |
| Country: | Portugal |
| Document type: | master thesis |
| Access type: | open access |
| Associated institution: | Universidade Nova de Lisboa |
| Language: | English |
| Origin: | Repositório Institucional da UNL |
| Summary: | Intrusion detection systems (IDS) based on public datasets often show promising results in academic papers but fail to perform effectively in real-world scenarios due to flaws in dataset creation. This discrepancy raises the question of whether explainable algorithms should be essential when building machine learning-based intrusion detection systems. While recent research has highlighted some deficiencies in these studies, new datasets have emerged, and practical guides on addressing these issues are lacking. This thesis extends previous work by evaluating improvements in recent datasets and providing new insights based on our findings. Our study reveals persistent issues in existing datasets and demonstrates, through explainable AI techniques, why building intrusion detection systems with these datasets should be approached with caution. By contributing guidelines on what to avoid when developing an intrusion detection system, we also illustrate how certain aspects of the process require deeper analysis before proposing new models. This research underscores the critical need for more robust and representative datasets in IDS development, paving the way for more reliable and practical cybersecurity solutions. |
|---|