Advancing User-Voice Interaction: Exploring Emotion-Aware Voice Assistants Through a Role-Swapping Approach
Paper i proceeding, 2025

As voice assistants (VAs) become increasingly integrated into daily life, the need for emotion-aware systems that can recognize and respond appropriately to user emotions has grown. While significant progress has been made in speech emotion recognition (SER) and sentiment analysis, effectively addressing user emotions-particularly negative ones-remains a challenge. This study explores human emotional response strategies in VA interactions using a role-swapping approach, where participants regulate AI emotions rather than receiving pre-programmed responses. Through speech feature analysis and natural language processing (NLP), we examined acoustic and linguistic patterns across various emotional scenarios. Results show that participants favor neutral or positive emotional responses when engaging with negative emotional cues, highlighting a natural tendency toward emotional regulation and de-escalation. Key acoustic indicators such as root mean square (RMS), zero-crossing rate (ZCR), and jitter were identified as sensitive to emotional states, while sentiment polarity and lexical diversity (TTR) distinguished between positive and negative responses. These findings provide valuable insights for developing adaptive, context-aware VAs capable of delivering empathetic, culturally sensitive, and user-aligned responses. By understanding how humans naturally regulate emotions in AI interactions, this research contributes to the design of more intuitive and emotionally intelligent voice assistants, enhancing user trust and engagement in human-AI interactions.

Emotion-Aware Voice Assistants

Role-Swapping Approach

Speech and Linguistic Analysis

Speech Emotion Recognition (SER)

Författare

Yong Ma

Universitetet i Bergen

Yuchong Zhang

Kungliga Tekniska Högskolan (KTH)

Di Fu

University of Surrey

Stephanie Zubicueta Portales

Norges teknisk-naturvitenskapelige universitet

Danica Kragic

Kungliga Tekniska Högskolan (KTH)

Morten Fjeld

Chalmers, Data- och informationsteknik, Interaktionsdesign och Software Engineering

Universitetet i Bergen

Lecture Notes in Computer Science

0302-9743 (ISSN) 1611-3349 (eISSN)

Vol. 15802 LNCS 303-320
9783031929762 (ISBN)

13th International Conference on Distributed, Ambient and Pervasive Interactions, DAPI 2025, held as part of the 27th HCI International Conference, HCII 2025
Gothenburg, Sweden,

Ämneskategorier (SSIF 2025)

Språkbehandling och datorlingvistik

Människa-datorinteraktion (interaktionsdesign)

Jämförande språkvetenskap och allmän lingvistik

DOI

10.1007/978-3-031-92977-9_19

Mer information

Senast uppdaterat

2025-06-19