EBB를 이용한 중등학교 말하기 평가척도 개발 = Constructing rating scales for oral performance tests at secondary schools based on the EBB model.

다국어 초록 (Multilingual Abstract)

This article examines the validity of the empirically derived, binary-choice, boundary-definition (EBB) rating scale for oral performance tests at secondary schools. First, this article takes three problems into account of applying a priori rating scales such as ACTFL guidelines in the secondary school context. It then turns to a posteriori data-based EBB rating scale, and shows advantages of using the inductive scale which can cover the three problems mentioned in the deductive rating scales. Subjects are 20 high school students in an English conversation course and three Korean English teachers. Speech data are collected from the pupils who are asked to tell stories about their past volunteering experiences. Two of the three Korean teachers of this research collaborate to construct the EBB rating scale. Findings indicate that the EBB rating scale mirrors classroom learning in the assesment, divides the subjects in half, and its binary algorithms are clear. ICC inter-rater reliability coefficient is .78 for average measures which is a little lower, but still consistent with the previous result by Upshur and Turner (1995). This paper concludes that the EBB rating scale seems to have high degree of validity and is applicable to the secondary schools.

참고문헌 (Reference)

1 전병만, "초.중등 영어교육 현황분석.(교육부 정책과제 2006-이슈-4)" 교육과학기술부 2006

2 김해선, "중학교 영어 말하기 평가 실태 분석과 제안.석사학위논문.부경대학교" 부경대학교 1999

3 최인철, "중등영어교육 수행평가를 위한 음성인식기술기반 모의구술면접 모형" 한국외국어교육학회 12 (12): 235-266, 2005

4 김덕기, "중등영어 수행능력 절대평가기준 제안: 표현기능을 중심으로" 54 (54): 175-200, 1999

5 신상근, "예비교사의 평가 전문성 신장을 위한 동료평가의 유용성에 관한 연구" 팬코리아영어교육학회 19 (19): 183-8, 2007

6 이 호, "영작문 평가 상황에서 학생들의 평가 과정에 대한 연구" 17 (17): 215-234, 2005

7 임창근, "영어교육 평가에 있어서 질적인 평가와 양적인 평가의 조화" 4 (4): 167-187, 1998

8 이인제, "영어과 교사의 학생 평가 전문성 신장 모형과 기준" 한국교육과정평가원 2004

9 이완기, "영어 평가 방법론" 문진미디어 2003

10 이영식, "영어 작문평가에 대한 채점자 훈련의 원리" 55 (55): 201-217, 2000

11 오준일, "영어 말하기 능력 절대 등급 설정에 관한 연구" 팬코리아영어교육학회 18 (18): 141-162, 2006

12 이 호, "시험문항 지침서를 통한 예비교원 평가훈련" 2007

13 교육과학기술부, "교육통계연보"

14 Messick, S., "Validity in Educational measurement" Macmillan/American Council on Education 13-103, 1989

15 Hudson, T., "Trends in assessment scales and criterion-referenced language assessment" 25 : 205-227, 2005

16 Canale, M., "The theoretical bases of communicative approaches to second language teaching and testing" 1 : 1-47, 1980

17 Shohamy, E., "The power of tests: A critical perspective on the uses of language tests" Pearson Longman 2001

18 Wilds, C., "The oral interview test in Testing Language Proficiency" for Applied Linguistics 29-44, 1975

19 Matthews, M., "The measurement of productive skills: Doubts concerning the assessment criteria of certain public examinations" 44 (44): 117-121, 1990

20 Bachman, L., "The evaluation of communicative language proficiency: A critique of the ACTFL oral interview" 70 : 380-390, 1986

21 Lowe, P, "The ILR oral interview: Origins, applications, pitfalls, and implications" 16 : 230-244, 1985

22 Fulcher, G., "Tests of oral performance: The need for data-based criteria" 41 (41): 287-291, 1987

23 Fulcher, G., "Testing second language speaking" Pearson Longman 2003

24 Adams, M. L., "Testing kit: French and Spanish" Department of State and the Foreign Services Institute 1979

25 Davidson, F., "Testcraft: A teacher's guide to writing and using language test specifications" Yale University Press 2002

26 E.T.S, "TSE and SPEAK score user guide. Retrieved December 20, 2007, from the World Wide Web"

27 Upshur, J., "Systemic effects in the rating of second language speaking ability: Test method and learner discourse" 16 (16): 82-111, 1999

28 Lazaraton, A., "Studies in language testing 14: A qualitative approach to the validation of oral language tests" Cambridge University Press 2002

29 Atkinson, J. M., "Structures in social action: Studies in conversation analysis" Cambridge University Press 1984

30 North, B., "Scaling descriptors for language proficiency scales" 15 (15): 217-263, 1998

31 North, B., "Scales of language proficiency: A survey of some existing systems" Council of Europe 1994

32 ALC, "SST 등급해설"

33 신동일, "SLA 선행 연구에 근거한 말하기 평가척도 제안: 문법 영역 중심으로" 한국영어교육학회 57 (57): 423-450, 2002

34 Brewer, M. B., "Research design and issues of validity in Handbook of research methods in social and personality psychology" Cambridge University Press 2000

35 Magnan, S. S., "Rater reliability of the ACTFL oral proficiency interview" 43 : 267-276, 1987

36 신동일, "Rasch 모형을 이용한 고등학교 영어과 말하기 및 쓰기 능력 등급 기술표 개발" 한국영어교육학회 57 (57): 469-499, 2002

37 Bachman, L., "Problems in examining the validity of the ACTFL oral proficiency interview" 10 : 149-164, 1988

38 Khattri, N., "Principles and practices of performance assessment" Erlbaum 1998

39 Shohamy, E., "Performance assessment in language testing" 15 : 188-211, 1995

40 Lantolf, J. P., "Oral proficiency testing: A critical analysis" 69 : 337-345, 1985

41 Fulcher, G, "Lexis and reality in oral testing" ERIC Clearinghouse on Languages and Linguistic 1988

42 Carroll, B. J., "Language testing: Is there another way in Language Testing" Modern English Publications 1-10, 1982

43 Shohamy, E, "Language testing priorities: A different perspective" 23 (23): 365-394, 1990

44 Bachman, L., "Language testing in practice" Oxford University Press 1996

45 Skehan, P., "Issues in the testing of English for specific purposes" 1 : 202-220, 1984

46 Hwang, Sung-Sam., "Intrarater reliability of speaking tests in high school: A focus on rater's overfit rating patterns" 1 : 93-105, 2007

47 Shrout, P. E., "Intraclass correlations: Uses in assessing rater reliability" 86 (86): 420-428, 1979

48 Finch, A., "Integrating teaching and assessment in the EFL classroom" Sahoe Pyeogron Academy 2005

49 Canale, M., "From communicative competence to communicative language pedagogy in Language and Communication" Longman 1-12, 1983

50 Fulcher, G., "Evaluating quality in second language performance tests" 1 : 3-19, 2007

51 이병민, "EFL 영어학습 환경에서 학습시간의 의미" 10 (10): 107-129, 2003

52 Fulcher, G., "Does the thick description lead to smart tests?: A data-based approach to rating scale construction" 13 (13): 208-238, 1996

53 Chalhoub-Deville, M., "Deriving oral assessment scales across different tests and rater groups" 12 : 16-33, 1995

54 Upshur, J., "Constructing rating scales for second language tests" 49 : 3-12, 1995

55 Pienemann, M., "Constructing an acquisition-based procedure for second language assessment" 10 : 217-234, 1988

56 Council of Europe, "Common European framework of reference for languages: Learning, teaching, assessment" Cambridge University Press 2001

57 Liskin-Gasparro, J. E., "Circumlocution, communication strategies, and the ACTFL proficiency guidelines: An analysis of student discourse" 29 : 317-330, 1996

58 Stein, N. L., "An analysis of story comprehension in elementary school children in New directions in discourse processing" Ablex 53-120, 1979

59 신동일, "ACTFL-SOPI체제의 영어 말하기 평가 현장연구:한국에서의 문제점과 현실적 대안" 56 (56): 309-331, 2001

60 ACTFL, "ACTFL proficiency guidelines. Hastings-on-Hudson" American Council on the Teaching of Foreign Languages 1986

61 박기화, "7차 영어과 교육과정 성취 기준의 적정성 연구: 말하기, 쓰기를 중심으로" 팬코리아영어교육학회 17 (17): 229-250, 2005