Emotion and Engagement Across the Idol Spectrum: Comparing Virtual and Human Idols

The increasing adoption of virtual idols in entertainment platforms raises critical questions about how viewers respond to their emotional performances compared to human idols. Despite their growing presence, little is known about whether and how emotional expressivity differs across performer modalities and content formats, or how these differences influence viewer engagement. This research investigates how emotional expressions by virtual and human idols shape engagement across competence- and warmth-oriented content formats. Drawing on a dataset of 16,583 YouTube videos, we apply multimodal deep-learning models to extract facial and vocal emotional cues and link them with behavioral engagement metrics. We find that virtual idols express a broader latitude of negative emotions, and their engagement follows a U-shaped relationship, where high emotional extremity increases engagement. Conversely, human idols generate greater engagement through contextual alignment: competence-oriented formats benefit from positive valence, while excessive positivity in warmth-related formats lowers engagement. This study contributes to emerging research on AI-mediated performance and emotional analytics by identifying how the ontological form of the performer interacts with content format to shape viewer responses. These findings offer actionable insights for entertainment practitioners navigating emotional design in AI-integrated entertainment.