‘제스쳐링(gesturing)’을 통한 일본어 동작동사 구현 및 습득을 위한 모바일 어플리케이션 개발

다국어 초록 (Multilingual Abstract)

The purpose of this research is to develop a mobile application to search and upload images which use tagging information to create a direct link to the meaning of a word. In this paper, we use the term ‘gesturing’ (as opposed to ‘gesture’) which means ‘movements of the arms and hands and are closely synchronized with the flow of speech’ (McNeil, 1992：11). ‘Gesturing’ differs from ‘gesture’ in the sense that ‘gesturing’ is a process that enables the meaning of a word to be visualized by means of images that are vividly expressed by non-verbal elements such as concrete acts, objects, and tools. The ability to visualize the meaning of a word by way of gesturing can be foreseen to have many educational and learning benefits (especially motional verbs which will be the focus of this research). The images that are being photographed and uploaded by the mobile application developed as a result of this research will be combined with the image database which has already been developed by the meta image search engine (-LEE & HAN (2011)-).

국문 초록 (Abstract)

본 연구은 태그 정보를 통하여 단어의미와 직결된 이미지를 검색하고 업로드할 수 있는 모바일 어플리케이션을 개발하는 것을 목적으로 한다. 본 연구에서 팔, 손의 움직임, 그리고 발화의 흐름과 연동되어 나타나는 기존의 ‘제스쳐(gesture-McNeil, 1992：11-)와 대비된 ‘제스쳐링(gesturing)이라는 개념을 도입하였다. 제스쳐링은 구체적인 행위, 대상, 도구 등과 같은 비언어적인(non-verbal) 요소를 사용하여 단어의 의미를 실제 언어활동과 최대한 흡사한 방식으로 시각화·형상화 시킨다는 점에서 기존의 제스쳐와는 차별화된다고 할 수 있다. 또한, 언어습득, 특히 제2외국어의 습득(Second Language Acquisition)에 있어서 무의미한 제스쳐가 아닌 의미와 직결된 제스쳐의 병행이 단어의 의미복원에 유효한 교육적·학습적 효과를 발휘할 것으로 기대할 수 있다. 마지막으로 본 연구의 모바일 어플리케이션을 통해 구현되는 제스쳐링 이미지는 기존의 이미지 데이터베이스(2011(이ㆍ한))에 통합 관리·운영된다.

참고문헌 (Reference)

1 이준서, "틀의미론(Frame Semantics)에 입각한 일본어 ‘Ingestion’ 동사의 ‘문화 이미지 프레임(Culture-Image-Frame)’ 연구" 글로벌일본연구원 (20) : 45-62, 2013

2 이준서, "일본어교육용 이미지 검색엔진 구축" 한국일어교육학회 (20) : 159-169, 2011

3 이준서, "일본어교육용 웹 이미지사전의 효과성 검증" 일본연구소 35 : 379-397, 2012

4 工藤真由美, "アスペクト․テンス体系とテクスト-現代日本語の時間の表現-" ひつじ書房 2002

5 D'Ausilio, A., "The role of the mirror system in mapping complex sounds into actions." 27 (27): 5847-5848, 2007

6 Cook, S. W., "The role of gesture in learning：Do children use their hands to change their minds?." 7 (7): 211-232, 2006

7 Susan Goldin-Meadow, S., "The role of gesture in communication and thinking" 11 (11): 419-429, 1999

8 Galantucci, B., "The motor theory of speech perception reviewed." 13 : 361-377, 2006

9 Church, R. B., "The effects of gestural instruction on bilingual children" 7 (7): 303-319, 2004

10 Cohen, R. L., "The effect of encoding variables on the free recall of words and action events" 11 : 575-582, 1983

11 Valenzeno, L., "Teachers' gestures facilitate students' learning : A lesson in symmetry" 28 (28): 187-204, 2003

12 Mehrabian, A., "Silent messages：Implicit communication of emotions and attitudes" Wadsworth 1981

13 Peeck, J., "Retention of pictorial and verbal content of a text with illustration" 66 (66): 880-888, 1974

14 Liberman, A. M., "Perception of the speech code." 74 : 431-461, 1967

15 Cohen, R. L., "On the generality of some memory laws" 22 : 267-281, 1981

16 Levin, J. R., "On pictures in prose" 26 : 233-243, 1978

17 Ronald M. Le Bel, "Motor-auditory-visual integration : The role of the human mirror neuron system in communication and communication disorders" 42 : 299-304, 2009

18 Browman, C. P., "Mind as motion" MIT Press 1995

19 Mulligan, N. W., "Memory for actions：Self-performed tasks and the reenactment effect" 31 : 412-421, 2003

20 Cohen, R. L., "Interevent differences in event memory : Why are some events more recallable than others?" 15 : 109-118, 1987

21 McNeil, D., "Hand and mind：What gestures reveal about thought" University of Chicago Press 1992

22 Susan Wagner Cook., "Gesturing makes memories that last" 63 (63): 465-475, 2010

23 Alibali, M. W., "Gesture-speech mismatch and mechanisms of learning : What the hands reveal about a child's state of mind. Cognitive Psychology, 25(4), p" 25 (25): 468-523, 1993

24 Singer, M. A., "Children learn when their teacher's gestures and speech differ." 16 (16): 85-89, 2005