음성 인식 기술을 활용한 채점자 엄격도 반영 및 유창성 측정 타당성 = Plausibility of reflecting rater severity and measuring fluency based on ASR-based oral proficiency testing

다국어 초록 (Multilingual Abstract)

Serious inherent problems with practicality, intra-rater and inter-rater reliability overshadow the known positive washback effects of performance assessment in language education. In particular, it has been welldocumented that inter-rater reliability poses a serious threat to overall test validity, since individual raters necessarily measure performance according to their own subjective severity criteria in language proficiency. However, language testing has witnessed a remarkable series of breakthroughs in performance assessment during the recent advent of the information era. One such breakthrough utilizes state-of-the-art automatic speech recognition (ASR) technology for oral proficiency interviews(OPI). Granting that current forms of ASR technologies may not produce results with the reliability needed to accommodate highstakes standardized test administration, they do offer aid in approaching the thorny issues of practicality and inherent human inter-rater subjectivity. Accordingly, this paper is intended to investigate the degree to which ASR-based OPI ratings match similar human-conducted OPI ratings by employing correlational analyses on the basis of degrees of rater severity. Furthermore, this paper attempts to explore a method of enhancing the robustness of ASR-based OPI ratings which capitalizes on suprasegmental information by measuring fluency based principally on the test-takers’ response time length.

목차 (Table of Contents)

• Abstract
• Ⅰ. 서론
• Ⅱ. 의사소통 능력의 평가 도구
• Ⅲ. 자동 음성 인식(ASR: Automatic Speech Recognition)
• Ⅳ. 연구 내용 및 방법
• Ⅴ. 결론 및 제언
• 참고문헌
• 부록

참고문헌 (Reference)

1 최인철, "영어 의사소통능력의 모의 구술 면접(Simulated Oral Proficiency Interview) 시험 방식 양상 타당성 검증" 16 (16): 215-246, 2000

2 Chambers, F, "What do we mean by fluency?" 4 : 535-544, 1997

3 Witt, S. M., "Use of speech recognition in computer-assisted language learning" University of Cambridge, UK. 1999

4 Hamburger, H, "Tutorials tools for language learning by two-medium dialogue (n Intelligent language tutors: Theory shaping technology)" Mahwah, NJ: Lawrence Erlbaum 183-199, 1995

5 Shohamy, E, "The validity of direct versus semi-direct oral test" 11 (11): 99-124, 1994

6 Clark, J. L. D, "The test of spoken English as a measure of communicative ability in English-medium instructional settings" Princeton, NJ: Educational Testing Service (7) : 1980

7 Shohamy, E, "The stability of oral proficiency assessment on the oral interview testing procedure" 33 : 527-540, 1983

8 Weir, C. J., "The specification, realization, and validation of an English language proficiency test (In Testing English for university study)" London: Modern English Publications/British Council 45-110, 1988

9 Stansfield, C. W, "The development and validation of a simulated oral proficiency interview" 76 : 129-141, 1992

10 Underhill, N, "Testing spoken language: A handbook of oral testing techniques" Cambridge: Cambridge University Press 1987

11 최인철, "Test of Oral Proficiency(TOP)의 개발 연구" 34 (34): 245-286, 1998

12 "TAST"

13 Cole, R, "Survey of the state of the art in human language technology" Cambridge: Cambridge University Press 1997

14 Alderson, J. C, "Reviews of English language proficiency tests" Washington DC: Teachers of English to Speakers of Other Languages (TESOL) 1987

15 Bernstein, J., "Reliability and validity of PhonePass spoken English tests" 2000

16 Pennington, M. C, "Recent research in L2 phonology: Implications for practice (In Pronunciation pedagogy and theory: New views, new directions)" Aexandria, VA: TESOL Publications 112-130, 1994

17 Cucchiarini, C, "Quantitative assessment of second language learners’ fluency by means of automatic speech recognition technology" 107 (107): 989-999, 2000

18 Williams, M, "Psychology for language teachers: A social constructivist approach" Cambridge: Cambridge University Press 1997

19 Schmidt, R, "Psychological mechanisms underlying second language fluency" 14 : 357-385, 1992

20 Gilbert, J, "Pronunciation practices as an aid to listening comprehension (n A guide for the teaching of second language listening)" San Diego: Dominic Press 97-112, 1995

21 "PhonePass"

22 Yule, G, "Paying attention to pronunciation: The role of self-monitoring in perception" 21 (21): 765-768, 1987

23 Morley, J, "Oral communication in TESOL: Integrating speaking, listening and pronunciation" 25 (25): 51-75, 1991

24 Allen, J, "Natural language understanding (2nd ed)" Redwood City, CA: Benjamin/Cummings Publishings 1995

25 Larson, J, "Montevidisco: An interactive-video simulated visit to Mexico" Provo, Utah: Brigham Young University 1996

26 McNamara, T., "Modelling performance: Opening Pandora’s box" 16 : 159-179, 1995

27 McNamara, T, "Measuring second language performance" London: Longman 1996

28 Bachman, L. F, "Language testing in practice" Oxford: Oxford University Press 1996

29 Lennon, P, "Investigating fluency in EFL: A quantitative approach" 3 : 387-417, 1990

30 Ladd, D. R, "Intonational phonology" Cambridge: Cambridge University Press 1996

31 Entropic, "GrapHvite with HAPI" Cambridge: Entropic Speech Technology 1998

32 Bachman, L. F, "Fundamental considerations in language testing" Oxford: Oxford University Press 1990

33 Clark, J. L. D, "Exploration of speaking proficiency measures in the TOEFL context" Princeton, NJ: Educational Testing Service (4) : 1979

34 Larson, J., "Considerations for testing reading proficiency via computer- adaptive testing (In Studies in language testing, 10. Issues in computer-adaptive testing of reading proficiency)" Cambridge: Cambridge University Press 71-90, 1999

35 Henry, G, "Computer-controlled random access audio in the comprehension approach to second language learning" 20 (20): 255-264, 1987

36 Witt, S. M, "Computer-assisted pronunciation teaching based on automatic speech recognition" Cambridge: Cambridge University Engineering Department 1997

37 Brumfit, C, "Communicative methodology in language teaching: The roles of fluency and accuracy" Cambridge: Cambridge University Press 1984

38 ACTFL, "ACTFL proficiency guidelines.(revised)" Hastings-on-Hudson, NY: ACTFL Materials Center 1999

39 ACTFL, "ACTFL proficiency guidelines" Hastings-on-Hudson, NY: ACTFL Materials Center 1985

40 Rabiner, L. R., "A tutorial on Hidden Markov Models and selected applications in speech recognition" 267-296, 1989

41 Burnstein, J, "A review of computer- based speech technology for TOEFL 2000" Princeton, NJ: Educational Testing Service 1999