Paper-based versus Image-based Scoring of Student Essays

다국어 초록 (Multilingual Abstract)

Essays have been incorporated into many of the standardized language testing programs since assessing students’ writing ability directly using essays has many values. However, scoring a large quantity of student essays is costly and time consuming. Paper-based scoring, which refers to scoring of essays on paper as submitted by students, had been the standard practice of essay scoring until image-based scoring, which refers to scoring of scanned images of essays, became available due to advances in computerized scanning technicology. Because various aspects of image-based scoring could make it potentially different from paper-based scoring, causing concerns for the reliability of the assessment made, the current study compared paper-based and image-based essay scoring modes using essays from the Georgia 8th Grade Supplemental Writing Assessment in order to examine the impact of image-based scoring on the reliability of essay test scores. The results showed that paper-based and image-based scoring were similar in terms of reliability, providing evidence for the comparability of paper-based and image-based scoring. Although limited in scope, this study adds to the literature on writing assessment by presenting empirical evidence in support of the transition from paper-based scoring practice to image-based scoring practice.

참고문헌 (Reference)

1 Powers, D., "Will they think less of my handwritten essay if others word-processed essays" 31 : 220-233, 1994

2 Sturman, L, "To e or not to e? A comparison of electronic marking and paper-based marking" 2003

3 ETS, "The results of the NAEP 1993 field test for the 1994 National Assessment of Educational Progress" Author 1993

4 Twing, J., "The Comparability of paper-based and image-based marking of a high-stakes, large-scale writing assessment" 2003

5 이요안, "Reevaluating Advanced Speaking Proficiency for Korean Teachers of English: A Case Study" 한국응용언어학회 25 (25): 303-336, 2009

6 Powers, D, "Qualifying essay readers for an on-line scoring network" ETS Report 1998

7 Bennett,R.E, "On-line assessment and the comparability of score meaning (ETS RM-03-05)" Educational Testing Service 2003

8 Chung, G., "Methodological approaches to online scoring of essays" National Center for Research on Evaluation,Standards,and Student Testing 1997

9 Lunz, M. E, "Measuring the impact of judge severity on examination scores" 3 : 331-345, 1990

10 Linacre,J.M, "Many-facet Rasch measurement" MESA Press 1989

11 안경자, "Learning to Write Academic English: A Sociocultural Perspective on L2 Learners’ Writing Portfolios" 한국응용언어학회 25 (25): 1-34, 2009

12 Bachman, L. F, "Language testing in practice" Oxford University Press 1996

13 Kreiman,C, "Investigating the effects of training and rater variables on reliability measures: A comparison of standup local scoring, online distributed scoring, and online local scoring" 2007

14 Linacre,J.M, "Facets Rasch measurement computer program. Version 3.58.0"

15 UCLES, "Electronic return of marks project" UCLES 2004

16 Royal-Dawson,L, "Electronic marking with ETS software" 2003

17 Powers, D, "Effects of mode of presentation on essay scores (ETS RM-97-08)" ETS 1997

18 Yuah Vicky Chon, "Collocations in L2 Writing and Rater's Perceived Writing Proficiency" 한국응용언어학회 25 (25): 101-129, 2009

19 Zhang, Y, "Applying the online scoring network (OSN) to advanced placement program (AP) tests" Educational Testing Service 2003

20 Whetton, C., "An evaluation of on-line marking" 2002

21 Powers, D., "A pilot test of online essay scoring" ETS 1997

22 Stemler, S. E, "A comparison of consensus, consistency, and measurement approaches to estimating inter-rater reliability"

23 Cohen,J, "A coefficient of agreement for nominal scales" 20 : 37-46, 1960

24 Raikes, N, "A Report on the March 2004 OCR On-Screen marking Trial: The examiners’ experience" Internal UCLES report 2004

25 유호정, "A Description of Connecting Reading and Writing in L2 Academic Writing through an Analysis of Classroom Talk" 한국응용언어학회 24 (24): 147-169, 2008