Machine Learning-based Approach for Malaysian Food Image Detection, Weight Estimation and Calories Estimation for Diabetes Management

Diabetes is a major public health concern in Malaysia, where many struggle with blood sugar control due to unbalanced diets and improper nutritional intake. Tracking dietary intake is essential for informed food choices, but existing tools rely on manual self-reporting, which is time-consuming and prone to errors. Thus, this study proposes a machine learning-based approach to automate food detection, weight estimation and calories estimation from images, focusing on Malaysian cuisine. It consists of three phases. The first phase is food detection and segmentation, where a YOLOv11 custom model is developed to detect, classify, localize, and segment food items. The second phase is food weight estimation, where image-extracted features are used to estimate food weight. Various ensemble regression models, including Random Forest, XGBoost, LightGBM, and CatBoost, are evaluated for this task. These models are trained on a dataset of five Malaysian food classes with 2,369 food weight entries and seven features. The third phase is food calories estimation, which automates the retrieval of calories data from the Nutritionix API to compute total calories intake based on the estimated food weight. These phases are then integrated into a web-based app prototype. The proposed approach achieved a 99.22% classification accuracy and a 99.25% F1-score in detection and segmentation. The CatBoost model is chosen as the food weight estimation model, as it achieved the lowest MAPE of 4.22%. The calories estimation module successfully retrieved calories data and calculated total calories. Overall, this approach provides a reliable and automated dietary monitoring solution for diabetes management.