Joint Angle-Based Activity Recognition System for PARO Therapy Observation

PARO social robot has been introduced as an alternative therapy for dementia people. However, actual therapy implementation should be evaluated to improve the effectiveness of therapy. We, therefore, propose a system that able to observe and analyze the activity interaction between caregiver and patient in PARO therapy. In this paper, we focus on developing efficient human activity recognition. The system uses features of the multi-camerabased human skeleton as the input information. We classify the activity recognition process into lower-body and upper-body recognition, which use 11 and 12 joint angles input, respectively. To extract the input data, the system uses a spatiotemporal method of multiclass Support Vector Machine (SVM) algorithm. We use the ADASYN approach for resampling imbalanced datasets. The proposed model has been tested in a sensor room, which utilizes 6 Kinect Azure cameras, to demonstrate its effectiveness. The result shows the proposed method succeeded in recognizing seven basic therapy activities. Then, the proposed method improved the accuracy of the conventional method based on therapeutic scenarios. Furthermore, the therapy's observation method of activity interaction will be conducted.