한국어 학습자의 발화에 대한 한국어 교사와 비교사 원어민 간의 평가 차이 연구 = An investigation of the differences between novice teachers, experienced teachers and non-teacher native speakers in evaluation of Korean language learners’ speech.

다국어 초록 (Multilingual Abstract)

This study aim is to investigate the scoring patterns in speaking test evaluation amongst novice teachers, experienced teachers and non-teacher natives and to know to what extent they are different from each other. To this end, ten novice teachers, ten experienced teachers and ten non-teacher natives scored twenty examinee’s responses to a semi-direct speaking test consisting of three tasks using a six-point Likert scale for accuracy (segmental; suprasegmental), fluency(speech rate; pause), vocabulary, grammar, content, consistency/conjunction, comprehensibility, and intelligibility. The analysis of raters’ scores through a multi-facet Rasch measurement model showed distinctive variability between the rating patterns of three groups. Novice teachers scored the most strictly, while the non-teachers scored the most leniently. Furthermore the analysis displayed different rating patterns amongst the three groups on the scoring criteria, the task type and the leaners’ Korean language ability. Such findings are expected to contribute not only to a better understanding of the raters’ scoring processes and to the development of a more valid speaking assessment, but also getting more information which part of speech we have to focus on in teaching speaking. (George Mason University, Korea)

참고문헌 (Reference)

1 이향, "한국어 말하기 평가의 발음 영역 채점에서의 채점자 특성에 따른 채점 경향 연구 -한국어 교육 경험과 전공을 중심으로-" 연세대학교 언어연구교육원 한국어학당 39 : 213-245, 2013

2 강석한, "외국인 한국어 말하기 시험의 평가자 요소가 채점에 미치는 영향" 이중언어학회 (55) : 1-29, 2014

3 김중섭, "국제 통용 한국어교육 표준 모형 개발" 국립국어원 2010

4 Lynch, B. K, "Using G-theory and many-facet Rasch Measurement in the development of performance assessments of the ESL speaking skills of immigrants" 15 : 158-180, 1998

5 Bambose, A., "Torn between norms: innovations in world English" 17 (17): 1-14, 1998

6 Clark, L. D., "The FSI/ILR/ACTFL proficiency scales and testing techniques: Development, current status, and needed research" Indiana University 8-15, 1987

7 Frith, J. R., "Testing the FSI testing kit" 11 : 12-14, 1979

8 Walker, R., "Teaching the pronunciation of English as a lingua franca" Oxford University Press 2010

9 Hadden, B. L., "Teacher and Nonteacher Perception of second language Communication" 41 (41): 1-24, 1991

10 Shohamy, E., "Rater reliability of the oral interview speaking test" 16 : 219-222, 1983

11 Schaefer, E., "Rater bias patterns in an EFL Writing assessment" 25 : 465-493, 2008

12 Byrnes, H., "Proficiency: concepts and developments" 18 : 9-10, 1986

13 Bachman, L., "Problems in examining the validity of the ACTFL oral proficiency interview" 10 : 149-163, 1988

14 Galloway, V., "Perceptions of the Communicative Efforts of American Students of Spanish" 64 : 429-433, 1980

15 Henning, G., "Oral proficiency testing: comparative validities of interview, imitation, and completion models" 33 : 315-332, 1983

16 Ludwig, J., "Native-Speaker Judgments of Second-Language Learners’ Efforts at Communication: A Review" 66 (66): 274-283, 1982

17 Chastain, K., "Native Speaker Reaction to Instructor-Identified Student Second-Language Errors" 64 : 210-215, 1980

18 Chastain, K., "Native Speaker Evaluation of Student Composition Errors" 65 : 288-294, 1981

19 Barnwell, D., "Naive’ native speakers and judgements of oral proficiency in Spanish" 6 : 152-163, 1989

20 McNamara, T. F., "Measuring second Language Performance" Longman 1996

21 Nichols, J., "Language study, international study, and education, Profession 88 (MLA)"

22 Smith, L. E., "International intelligibility of English: directions and resources" 4 (4): 333-342, 1985

23 Olsson, M., "Intelligibility: A Study of Errors and Their Importance"

24 Piazza, L. G., "French Tolerance for Grammatical Errors Made by Americans" 64 : 422-427, 1980

25 Ensz, K., "French Attitudes toward Typical Speech Errors of American Speakers of French" 66 : 133-139, 1982

26 Cumming, A., "Expertise in evaluating second language compositions" 7 : 31-51, 1990

27 Eckes, T., "Examing rater effects in TestDaF writing and speaking performance assessments: A many-facet Rasch analysis" 2 : 197-221, 2005

28 Politzer, R. L., "Errors of English Speakers of German as Perceived and Evaluated by German natives" 62 : 253-261, 1978

29 Politzer, R. L., "Errors of English Speakers of German as Perceived and Evaluated by German Natives" 62 : 257-, 1982

30 Carey, M. D., "Does a rater familiarity with a candidate’s Pronunciation affect the rating in oral proficiency interview?" 28 (28): 201-219, 2011

31 Council of Europe, "Common Europe framework of reference for language: Learning, teaching, assessment" Cambridge university Press 2001

32 Wolfe, E. W., "Cognitive differences in Proficient and Nonproficient Essay Scores" 15 (15): 465-492, 1998

33 김현정, "An investigation of novice, developing, and experienced raters’ rating patterns on a second language speaking assessment." 한국응용언어학회 26 (26): 151-182, 2010

34 Guntermann, G., "A Study of the Frequency and Communicative Effects of Errors in Spanish" 62 : 251-, 1978