A Computerized Testing system that Reduces Backward Reasoning in Multiple-choice Items = 선다형 문항에서 역행추리를 줄이는 컴퓨터화 검사 방식

다국어 초록 (Multilingual Abstract)

A new computerized testing system, called the Computerized Multiple-choice Testing (CMMT) system, was introduced. In this system, questions of multiple choice (MC) items are presented first without options, so that students must generate answers for themselves. They can click for the options when they are ready, and can respond within a brief, specified time period. The present study was performed to examine whether this system is effective in reducing backward reasoning, I. e., using the options of MC items as cues to find the correct answer. One hundred and seventy-seven 6th grade students (12 year olds) were divided into two groups so that mean scores from a prior test were equal: The experimental group took an intervening computerized test in the new format, and the control group in the MC format. Five days after the computerized intervening test, a short answer paper-and-pencil final test was given. Testing effect was greater in the new system than in the MC system. Analysis of the final test response in relation to the intervening test response showed that i) the students retained the correct answer in the new system more than in the MC testing system, and that ii) students corrected their previous failures in the intervening CMMT format more than those in the MC format. These results suggest that the new system is effective in reducing backward reasoning.

국문 초록 (Abstract)

변형 선다형 방식으로 명명된 새로운 컴퓨터화 검사가 소개되었다. 이 방식에서는 선다형 문항의 답지가 없이 먼저 질문만 제시되어 수험자가 스스로 답을 생각해내도록 한다. 수험자가 답을 할 준비가 되면 마우스를 클릭하여 답지 제시를 요청할 수 있는데, 답지는 미리 정해진 짧은 시간 동안만 제시되고 수험자는 그 시간 내에 답지 중 하나를 선택하여 반응한다. 본 연구에서는 이 시스템이 선다형의 약점으로 알려진 역행추리 즉, 답지를 이용하여 정답을 추측해내는 전략을 줄일 수 있는지를 알아보고자 하였다. 6학년 학생 177명을 이전 시험 결과에서의 평균에서 차이가 없게 두 집단으로 나눈 다음, 실험 집단은 새로운 방식으로 통제 집단은 선다형 방식으로 시험을 보도록 한 다음, 5일 후에 일부 문항을 단답형으로 다시 시험을 보게 하였다. 중간 본 시험에서의 반응을 고려하여 최종 검사 점수를 분석한 결과, ⅰ) 선다형보다 새로운 방식으로 보았을 때 정답을 더 잘 유지하였으며, ⅱ) 선다형보다 새로운 방식으로 시험을 보았을 때 전에 틀렸던 문항에서 더 적게 틀림을 발견하였다. 이 결과는 새로운 컴퓨터화 검사 방식이 역행추리를 줄이는데 효과적임을 시사한다.

목차 (Table of Contents)

• Introduction
• Method
• Results
• References
• 요약

참고문헌 (Reference)

1 Glover,J.A, "The “testing” phenomonon: Not gone but nearly forgotten" 81 (81): 392-399, 1989

2 Roediger, H. L, "The power of testing memory: Basic research and implications for educational practice" 2 : 31-74, 2006

3 Veloski, J. J, "Patients don’t present with five choices: An alternative to multiple-choice tests in assessing physician’s competence" 74 : 539-546, 1999

4 Park,J, "Learning in a new computerized testing system" 97 (97): 436-443, 2005

5 Braswell, J, "Item formats for assessment in mathematics. In Construction versus Choice in cognitive Measurement." Lawrence Erlbaum Associate 1993

6 Blaxton,T, "Investigating dissociations among memory measures: Support for a transfer appropriate processing framework" 15 : 657-668, 1989

7 Landauer, T. K, "Introduction to Latent Semantic Analysis" 25 : 259-284, 1998

8 Sireci, S. G, "Innoviative item formats in computer-based testing: In pursuit of construct representation ㅑㅜ Handbook of test development" Lawrence Earlbaum Associate 2006

9 Parshall, C. G, "Innovative item types for computerized testing. In Computerized adaptive testing: Theory and Practice" Kluwer Academic Publishers 129-148, 2000

10 Carpenter, S. K, "Impoverished cue support enhances subsequent retention: Support for the elaborative retrieval explanation of the testing effect" 34 : 268-276, 2006

11 Park, J, "Higher retentention after a new take-home computerized test" 39 (39): 538-547, 2008

12 Kulhavy, R. W, "Delay-retention effect with multiple-choice tests" 63 (63): 505-512, 1972

13 Surber, J. R, "Delay-retention effect in natural classroom settings" 67 (67): 170-172, 1976

14 Leacock & Chodrow, "C-rater: Automatied scoring of the short-answer questions" 37 : 389-405, 2003

15 Rudner, L. M, "Automated essay scoring using Bayes' theorem" 1 (1): 2002