Filling-in visual motion with sounds
Artikel i vetenskaplig tidskrift, 2008

Information about the motion of objects can be extracted by multiple sensory modalities, and, as a consequence, object motion perception typically involves the integration of multi-sensory information. Often, in naturalistic settings, the flow of such information can be rather discontinuous (e.g. a cat racing through the furniture in a cluttered room is partly seen and partly heard). This study addressed audiovisual interactions in the perception of time-sampled object motion by measuring adaptation aftereffects. We found significant auditory after-effects following adaptation to unisensory auditory and visual motion in depth, sampled at 12.5 Hz. The visually induced (cross-modal) auditory motion after-effect was eliminated if visual adaptors flashed at half of the rate (6.25 Hz). Remarkably, the addition of the highrate acoustic flutter (12.5 Hz) to this ineffective, sparsely time-sampled, visual adaptor restored the auditory after-effect to a level comparable to what was seen with high-rate bimodal adaptors (flashes and beeps). Our results suggest that this auditory-induced reinstatement of the motion after-effect from the poor visual signals resulted from the occurrence of sound-induced illusory flashes. This effect was found to be dependent both on the directional congruency between modalities and on the rate of auditory flutter. The auditory filling-in of time-sampled visual motion supports the feasibility of using reduced frame rate visual content in multisensory broadcasting and virtual reality applications. (C) 2008 Elsevier B.V. All rights reserved.

VISION

PSYCHOPHYSICS

DEPTH

Multisensory integration

Audition

INTEGRATION

Vision

PERCEPTION

CAPTURE

ADAPTATION

INFORMATION

SIGNALS

SENSORY MODALITIES

Motion after-effect

Författare

Alexander Väljamäe

Chalmers, Bygg- och miljöteknik, Teknisk akustik, Rumsakustik

S. Soto-Faraco

Hospital Sant Joan de Deu

Institucio Catalana de Recerca I Estudis Avancats

Universitat de Barcelona

Acta Psychologica

0001-6918 (ISSN)

Vol. 129 249-254

Ämneskategorier

Samhällsbyggnadsteknik

DOI

10.1016/j.actpsy.2008.08.004