Concept of Seamless Physical Observation of Human Hand Through Block Design Test

A seamless physical monitoring system for human observation is required to address the privacy and comfort concerns of the observed individuals. In this paper, we propose a human physical monitoring system integrated with a block design game-based approach to capture both human hand physical and cognitive capacities. By adopting this concept, we are able to naturally acquire human physical data. We have developed a smart block design, equipped with several sensors to track human activities from a physical standpoint. Supported by human skeleton and hand pose recognition, the data from Kohs blocks are processed in a human musculoskeletal simulation using MuJoCo, which comprises 59 hill-type muscles for each limb. To analyze performance for future studies, we conducted preliminary experiments involving humans playing the Kohs block design test. The results demonstrate that the proposed model effectively recognizes human hand muscle activity and muscle tension. Furthermore, the proposed concept is not only effective for assessing physical capabilities but can also be extended to cognitive capacities.