Lightweight Quantized XGBoost for Botnet Detection in Resource-Constrained IoT Networks

The rapid expansion of IoT devices has introduced significant security challenges, with malware authors constantly evolving their techniques to exploit vulnerabilities in IoT networks. Despite this growing threat, progress in developing effective detection solutions remains limited. In this study, we present an ML-based framework for detecting and classifying network threats targeting IoT environments. Using the CTU-IoT-Malware-Capture 2023 dataset and the UNSW Bot-IoT dataset, we transformed the task into a structured multi-class classification problem to better reflect real-world detection challenges. Our primary contribution lies in demonstrating the effectiveness of post-training quantization on gradient-boosted models, specifically a Quantized XGB variant enhanced with histogram-based quantization. This approach significantly reduces model size and inference time without sacrificing accuracy. The proposed model achieved high classification accuracies of 99.93% and 99.99% on the two datasets, while the quantization step led to 1.42× and 3× improvements in inference speed, and reductions in model size by 3.61× and 2.71×, respectively, making it well-suited for deployment in resource-constrained IoT settings. This work demonstrates not only the effectiveness of gradient boosting in handling complex traffic data but also introduces an efficient optimization strategy for real-time IoT threat detection.