See or Hear? Exploring the Effect of Visual/Audio Hints and Gaze-assisted Instant Post-task Feedback for Visual Search Tasks in AR
Paper in proceeding, 2023

Augmented reality (AR) is emerging in visual search tasks for increasingly immersive interactions with virtual objects. We propose an AR approach providing visual and audio hints along with gaze-assisted instant post-task feedback for search tasks based on mobile head-mounted display (HMD). The target case was a book-searching task, in which we aimed to explore the effect of the hints together with the task feedback with two hypotheses. H1: Since visual and audio hints can positively affect AR search tasks, the combination outperforms the individuals. H2: The gaze-assisted instant post-task feedback can positively affect AR search tasks. The proof-of-concept was demonstrated by an AR app in HMD and a comprehensive user study (n=96) consisting of two sub-studies, Study I (n=48) without task feedback and Study II (n=48) with task feedback. Following quantitative and qualitative analysis, our results partially verified H1 and completely verified H2, enabling us to conclude that the synthesis of visual and audio hints conditionally improves the AR visual search task efficiency when coupled with task feedback.

Augmented reality

Visual hint

Gaze assistance

Audio hint

User study

Search task

Instant post-task feedback

Author

Yuchong Zhang

Chalmers, Computer Science and Engineering (Chalmers), Interaction Design and Software Engineering

Adam Nowak

Lodz University of Technology

Yueming Xuan

Student at Chalmers

A. Romanowski

Lodz University of Technology

Morten Fjeld

Chalmers, Computer Science and Engineering (Chalmers), Interaction Design and Software Engineering

University of Bergen

Proceedings - 2023 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2023

1113-1122
9798350328387 (ISBN)

22nd IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2023
Sydney, Australia,

Subject Categories

Language Technology (Computational Linguistics)

Human Computer Interaction

DOI

10.1109/ISMAR59233.2023.00128

More information

Latest update

1/9/2024 1