Cross-modal Correspondence Between Acoustic Feature and Shape

다국어 초록 (Multilingual Abstract)

Our brain tends to associate stimulus features across the senses in a non-random manner. For example, people show consistency in labelling a rounded shape ‘maluma’/‘bouba’ and a spiky shape ‘takete’/‘kiki’. Previous studies have attributed this phenomenon to the correspondence between sound and shape, but without controlling for other potential factors (i.e., linguistic/orthographical factors). The present study examines the role of acoustic aspect per se by manipulating articulatory gestures to generate synthetic speech sounds not confined to a specific language. Participants were asked to choose either a rounded or spiky shape to indicate the shape that better matched each synthetic speech sound. The results demonstrate that shape choice was systematically mapped on to the dimensions manipulated to generate the sounds. These results indicate that acoustic features indeed drive the association between sound and visual shape.

참고문헌 (Reference)

1 Styles, S. J., "When does Maluma/Takete fail? Two key failures and a meta-analysis suggest that phonology and phonotactics matter" 8 : 1-17, 2017

2 Lee Masson, H., "Visual and haptic shape processing in the human brain : Unisensory processing, multisensory convergence, and top-down influences" 26 : 3402-3412, 2016

3 Gaißert, N., "Visual and haptic perceptual spaces show high similarity in humans" 10 : 1-20, 2010

4 Nielsen, A., "The sound of round : Evaluating the sound-symbolic role of consonants in the classic Takete-Maluma phenomenon" 65 : 115-124, 2011

5 Maurer, D., "The shape of boubas : Sound-shape correspondences in toddlers and adults" 9 : 316-322, 2006

6 Brainard, D. H., "The psychophysics toolbox" 10 : 433-436, 1997

7 Davis, R., "The fitness of names to drawings. A cross-cultural study in Tanganyika" 52 : 259-268, 1961

8 Taitz, A., "The audiovisual structure of onomatopoeias: An intrusion of real-world physics in lexical creation" 13 : e0193466-, 2018

9 Pelli, D. G., "The VideoToolbox software for visual psychophysics : Transforming numbers into movies" 10 : 437-442, 1997

10 Nam, H., "TADA : An enhanced, portable Task Dynamics model in MATLAB" 115 : 2430-2430, 2004

11 Ramachandran, V. S., "Synaesthesia-a window into perception, thought and language" 8 : 3-34, 2001

12 Ozturk, O., "Sound symbolism in infancy : Evidence for sound-shape cross-modal correspondences in 4-month-olds" 114 : 173-186, 2013

13 Hinton, L., "Sound Symbolism" Cambridge University Press 1994

14 Cuskley, C., "Phonological and orthographic influences in the bouba-kiki effect" 81 : 119-130, 2017

15 Tarte, R., "Phonetic symbolism in adult native speakers of Czech" 17 : 87-94, 1974

16 D’Onofrio, A., "Phonetic detail and dimensionality in sound-shape correspondences : Refining the Bouba-Kiki paradigm" 57 : 367-393, 2014

17 Nielsen, A., "Parsing the role of consonants versus vowels in the classic Takete-Maluma phenomenon" 67 : 153-163, 2013

18 Ashby, F. G., "On the dangers of averaging across subjects when using multidimensional scaling or the similarity-choice model" 5 : 144-151, 1994

19 Cox, T. F., "Multidimensional scaling" Chapman & Hall 2001

20 Fort, M., "Looking for the bouba-kiki effect in pre-lexical infants" 71-76, 2013

21 Köhler, W., "Gestalt psychology" Liveright 1947

22 Köhler, W., "Gestalt psychology" Liveright 1929

23 Donaldson, G. S., "Effects of vowel context on the recognition of initial and medial consonants by cochlear implant users" 27 : 658-677, 2006

24 Spence, C., "Crossmodal correspondences: A tutorial review" 73 : 971-995, 2011

25 Ahlner, F., "Cross-modal iconicity: A cognitive semiotic approach to sound symbolism" 38 (38): 298-348, 2010

26 Fort, M., "Consonants are more important than vowels in the Bouba-kiki Effect" 58 : 247-266, 2015

27 Nam, H., "Computational simulation of CV combination preferences in babbling" 41 : 63-77, 2013

28 Bremner, A. J., ""Bouba"and"Kiki"in Namibia? A remote culture make similar shape-sound matches, but different shape-taste matches to Westerners" 126 : 165-172, 2013