A Machine Learning Pipeline for Biofeedback-Driven, Self-guided Virtual Reality Therapy Using Speech-Based Arousal Detection

Virtual Reality (VR) is an emerging field, where its uses can go beyond entertainment purposes. In recent years, anxiety levels across the population have increased, making it particularly daunting for many to speak in front of others. VR can be used as a tool to prepare those who are anxious or suffer from social or public speaking anxiety (S/PSA) and integrate biofeedback to modulate behaviour. In order for a biofeedback VR system to positively impact S/PSA, reliable methods have to be developed to create accurate readings of users’ physiological stress levels. One of the data modalities employed by this could be speech stress patterns to determine stress levels in users’ voice. This paper proposes extracting several features from four different audio datasets to be trained by a convolutional neural network (CNN). The proposed pipeline has yielded results of high accuracy. If the model detects anxiety and stress levels, other diagnostic tools such as heart rate and brain activity data extracted via electroencephalogram data (EEG) can be used. The combination of biofeedback measures will develop an accurate account of the presence of psychological distress. Upon confirmation of users’ stress levels spiking, biofeedback and evidence based approaches can be implemented to help calm the user down. It is fundamental that the model developed precisely detects stress - failure to do so would lead to a negative experience of the tool and might potentially enhance their anxiety. This research offers a promising approach that can be used to address this widespread psychological challenge.