Towards Secure and Efficient Human Fall Detection: Sensor-Visual Fusion via Gramian Angular Field with Federated CNN

This article presents a human fall detection system that addresses two critical challenges: privacy preservation and detection accuracy. We propose a comprehensive framework that integrates state-of-the-art machine learning models, multimodal data fusion, federated learning (FL), and Karush-Kuhn-Tucker (KKT)-based resource optimization. The system fuses data from wearable sensors and cameras using Gramian Angular Field (GAF) encoding to capture rich spatial-temporal features. To protect sensitive data, we adopt a privacy-preserving FL setup, where model training occurs locally on client devices without transferring raw data. A custom convolutional neural network (CNN) is designed to extract robust features from the fused multimodal inputs under FL constraints. To further improve efficiency, a KKT-based optimization strategy is employed to allocate computational tasks based on device capacity. Evaluated on the UP-Fall dataset, the proposed system achieves 91% accuracy, demonstrating its effectiveness in detecting human falls while ensuring data privacy and resource efficiency. This work contributes to safer, scalable, and real-world-applicable fall detection for elderly care.