Tagungsband - DAGA 2024
Beitrag
Audiovisual prosody perception in cochlear implant recipients - assessment based on virtual reality (no manuscript)
Zusammenfassung:
Linguistic prosody plays a vital role in verbal communication. Amongst others, it carries information on sentence stress or the distinction between questions and statements. Cochlear Implant (CI) recipients are restricted in the use of acoustic prosody cues, especially in terms of the voice fundamental frequency. However, prosody is also perceived visually, as head and facial movements accompany the vocal expression. To date, few studies have addressed multimodal prosody perception. Controlled manipulations of acoustic cues are a valuable method to examine and quantify prosody perception. It could be shown that CI recipients had about four times higher F0-detection thresholds and shallower psychometric functions than normal-hearing listeners. For visual prosody, however, such a technique is more demanding. Here, a novel approach based on animations via virtual humans is used. Such a method has the advantage that – in parallel to acoustic manipulations – head and facial movements can be parametrized.Based on this method a study is shown that examines audiovisual prosody perception in CI recipients. Besides behavioral measures, eye-tracking and pupillometry data are collected to assess selective attention and cognitive load of the participants. Results from postlingually deafened CI recipients are compared with a control group of normal-hearing listeners.