Skip to main navigation Skip to search Skip to main content

A Platform-Agnostic Multimodal Digital Human Modelling Framework: Neurophysiological Sensing in Game-Based Interaction

  • Daniel J. Buxton*
  • , Mufti Mahmud
  • , Jordan J. Bird
  • , Thomas Hughes-Roberts
  • , David J. Brown
  • *Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Digital Human Modelling (DHM) is increasingly shaped by advances in artificial intelligence (AI), wearable biosensing, and interactive digital environments, particularly in research addressing accessibility and inclusion. However, many AI-enabled DHM approaches remain tightly coupled to specific platforms, tasks, or interpretative pipelines, limiting reproducibility, scalability, and ethical reuse. This paper presents a platform-agnostic DHM framework designed to support AI-ready multimodal interaction research by explicitly separating sensing, interaction modelling, and inference readiness. The framework integrates the OpenBCI Galea headset as a unified multimodal sensing layer, providing concurrent Electroencephalogram (EEG), Electromyogram (EMG), Electro-oculogram (EOG), Photoplethysmogram (PPG), and inertial data streams, alongside a reproducible, game-based interaction environment implemented using SuperTux. Rather than embedding AI models or behavioural inference, physiological signals are represented as structured, temporally aligned observables, enabling downstream AI methods to be applied under appropriate ethical approval. Interaction is modelled using computational task primitives and timestamped event markers, supporting consistent alignment across heterogeneous sensors and platforms. Technical verification via author self-instrumentation confirms data integrity, stream continuity, and synchronisation; no human-subjects evaluation or AI inference is reported. Scalability considerations are discussed with respect to data throughput, latency, and extension to additional sensors or interaction modalities. Illustrative use cases demonstrate how the framework can support AI-enabled DHM and HCI studies, including accessibility-oriented interaction design and adaptive systems research, without requiring architectural modifications. The proposed framework provides an emerging-technology-focused infrastructure for future ethics-approved, inclusive DHM research.

Original languageEnglish
Title of host publicationDigital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management - 17th International Conference, DHM 2026, Held as Part of the 28th HCI International Conference, HCII 2026, Proceedings
EditorsVincent G. Duffy
PublisherSpringer Science and Business Media Deutschland GmbH
Pages125-134
Number of pages10
ISBN (Print)9783032298386
DOIs
StatePublished - 2026
Event17th International Conference on Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management, DHM 2026, held as part of the 28th International Conference on Human-Computer Interaction, HCII 2026 - Montreal, Canada
Duration: 26 Jul 2026 → 31 Jul 2026

Publication series

NameLecture Notes in Computer Science
Volume16719 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference17th International Conference on Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management, DHM 2026, held as part of the 28th International Conference on Human-Computer Interaction, HCII 2026
Country/TerritoryCanada
CityMontreal
Period26/07/26 → 31/07/26

Bibliographical note

Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2026.

Keywords

  • Accessibility and Inclusion
  • Digital Human Modelling
  • Game-Based Interaction
  • Multimodal Neurophysiological Sensing
  • Platform-Agnostic Frameworks

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science

Fingerprint

Dive into the research topics of 'A Platform-Agnostic Multimodal Digital Human Modelling Framework: Neurophysiological Sensing in Game-Based Interaction'. Together they form a unique fingerprint.

Cite this