Anomaly Detection Using Artificial Intelligence Methods: Support Vector Machine and Decision Tree Algorithms

Abstract

For anyone using a network, safeguarding personal or sensitive information is a top priority. Anomaly detection plays a crucial role in spotting irregular patterns that could signal malicious behaviour and help to prevent data breaches, defend against cyberattacks, and ensure the integrity of network resources. Hence, this study explores how machine learning can be used to detect unusual or suspicious activity in network traffic,  a key step in protecting systems from threats like Denial of Service (DoS) attacks, unauthorised access, and probing attempts. Using the improved NSL-KDD dataset, which builds on the original KDD Cup 1999 dataset, two popular algorithms: Decision Trees and Support Vector Machines (SVM) with a polynomial kernel were tested. The models were applied to classify network traffic as either normal or potentially harmful, helping assess their effectiveness in identifying anomalies and supporting stronger network security. The Decision Tree model demonstrated a remarkable accuracy of 99.7% with an F-score of 0.997, showcasing its robustness in detecting prevalent attack patterns with high precision. Conversely, the Polynomial Kernel SVM exhibited a slightly lower accuracy of 99.5% but demonstrated stronger generalisation across various classes, reflected by a macro average F-score of 0.72. This indicates its superior capability to handle diverse and complex anomalies. The findings highlighted that both models are highly effective for intrusion detection, with the Decision Tree excelling in overall accuracy and the SVM offering a more balanced performance across a range of attack types. This research contributes to the ongoing development of intrusion detection systems, providing valuable insights into the trade-offs between model complexity, computational efficiency, and detection accuracy in real-time network security contexts.