Perceptual Optimization of Audio-visual Media: Moved by sound.

Book chapter, 2007

Virtual Reality (VR) research is gradually shifting focus from pictorial to perceptual realism where the optimization of media synthesis and reproduction technologies is based on end-users’ subjective or objective responses. In this paper our work on multisensory perceptual optimization in motion simulators is presented. Spatial presence and illusory self-motion ratings were used to determine and evaluate the most instrumental acoustic cues in audio-visual or purely auditory virtual environments. Results show how sound can enhance users experience or, alternatively, compensate for a reduced visual representation. In addition, we present a pilot study in the cinema investigating the effects of minimized visual content on spatial presence and emotional responses. In conclusion, we discuss how similar experimental methodologies can advance the understanding of traditional audio-visual media perception mechanisms and test new multisensory media forms with a reduced cognitive load.