Communication in restricted situations: The use of orofacial expressions in different speech modes and when wearing a face mask
Loading...
Supplementary material
Other Title
Authors
Mahdinazhad Sardhaei, N.
Żygis, M.
Sharifzadeh, Hamid
Żygis, M.
Sharifzadeh, Hamid
Author ORCID Profiles (clickable)
Degree
Grantor
Date
2023-09-14
Supervisors
Type
Conference Contribution - Poster Presentation
Ngā Upoko Tukutuku (Māori subject headings)
Keyword
facial masks and communication
speech analysis
psychoacoustics
computational linguistics
speech analysis
psychoacoustics
computational linguistics
ANZSRC Field of Research Code (2020)
Citation
Mahdinazhad Sardhaei, N, Żygis, M., & Sharifzadeh, H. (2023, September 13-15). Communication in restricted situations: The use of orofacial expressions in different speech modes and when wearing a face mask [Poster presentation]. 8th Conference of Gesture and Speech in Interaction (GeSpIn2023), Nijmegen, Netherlands. https://hdl.handle.net/10652/6549
Abstract
Speakers in their communications integrate auditory and visual information produced by face and body to communicate intended meanings. One of the core questions on the association between visual and spoken components of language is how various visual movements accompanying the speech contribute to the way the speech is uttered i.e., speech prosody. A substantial number of studies have demonstrated the prominent contribution of facial expressions and other forms of visual correlates to the auditory properties of prosody in voiced speech. However, the literature has merely questioned what happens when F0, the most important prosodic cue, is absent from the acoustic signal, as is the case in whispered speech. A possible answer to this question can be interpreted in terms of trading relations. While on one hand, according to trade-off hypothesis, different modalities can compensate for the absence or reduced occurrence of another based on the requirements of situational constrains, on the other hand, hand-in-hand hypothesis views the relation between gestures and speech in parallel or redundant rather than compensatory in the sense that gestures basically express information that can be derived from the spoken content alone.. Building on these two hypotheses, we address the relation between orofacial expressions and acoustic cues of prosody under two specific communicative constraints. First, we examine if the relation between orofacial gestures and acoustic cues of intonation changes in two speech modes of normal and whispered speech differing in the presence or absence of fundamental frequency. Second, based on previous empirical studies reporting the effect of face masks on acoustic deterioration of an utterance, we investigate if various orofacial gestures such as the movements of eyebrows, lip aperture, and eye-squint are affected by wearing a protective face mask in comparison to a condition where the speakers do not use a facial mask. We will also study the extent the orofacial expressions interact with prosody in voiced vs. whispered to express intonational differences between questions with a rising intonation and statements with a falling F0
Publisher
Permanent link
Link to ePress publication
DOI
Copyright holder
Authors
Copyright notice
All rights restricted