Filling-in visual motion with sounds
Journal article, 2008

Information about the motion of objects can be extracted by multiple sensory modalities, and, as a consequence, object motion perception typically involves the integration of multi-sensory information. Often, in naturalistic settings, the flow of such information can be rather discontinuous (e.g. a cat racing through the furniture in a cluttered room is partly seen and partly heard). This study addressed audiovisual interactions in the perception of time-sampled object motion by measuring adaptation aftereffects. We found significant auditory after-effects following adaptation to unisensory auditory and visual motion in depth, sampled at 12.5 Hz. The visually induced (cross-modal) auditory motion after-effect was eliminated if visual adaptors flashed at half of the rate (6.25 Hz). Remarkably, the addition of the highrate acoustic flutter (12.5 Hz) to this ineffective, sparsely time-sampled, visual adaptor restored the auditory after-effect to a level comparable to what was seen with high-rate bimodal adaptors (flashes and beeps). Our results suggest that this auditory-induced reinstatement of the motion after-effect from the poor visual signals resulted from the occurrence of sound-induced illusory flashes. This effect was found to be dependent both on the directional congruency between modalities and on the rate of auditory flutter. The auditory filling-in of time-sampled visual motion supports the feasibility of using reduced frame rate visual content in multisensory broadcasting and virtual reality applications. (C) 2008 Elsevier B.V. All rights reserved.

Multisensory integration

INTEGRATION

VISION

Vision

SIGNALS

DEPTH

INFORMATION

SENSORY MODALITIES

Motion after-effect

Audition

CAPTURE

ADAPTATION

PSYCHOPHYSICS

PERCEPTION

Author

Alexander Väljamäe

Chalmers, Civil and Environmental Engineering, Applied Acoustics

S. Soto-Faraco

Hospital Sant Joan de Deu

University of Barcelona

Catalan Institution for Research and Advanced Studies

Acta Psychologica

0001-6918 (ISSN)

Vol. 129 2 249-254

Subject Categories

Civil Engineering

DOI

10.1016/j.actpsy.2008.08.004

More information

Latest update

12/5/2019