Developing a Score Scale in Relation to Measurement Error for Mixed Format Tests

국문 초록 (Abstract)

이 연구의 목적은 혼합형 검사에서 아크사인 척도화 방법을 이용하여 조건적 측정오차의 크기를 유사하게 만드는 절차에 대해 기술하고 그 결과를 제시하는데 있었다. 3모수 로지스틱 문항반응 모형과 일반화 부분점수 문항반응 모형을 이용하여 혼합형 검사에서의 관찰점수를 추정한 후, 이것을 아크사인 방법으로 점수를 비선형으로 변환했다. 다음으로 이 점수를 다시 일정한 크기의 조건적 측정오차를 갖도록 선형으로 변환했다. 연구결과 기본 검사형 및 동등화된 검사형에서 조건적 측정오차의 크기는 의도한 대로 유사하였다. 이 연구에서는 또한 T-점수, 스테나인, NCE 점수 등을 이용했을 때 조건적 측정 오차의 크기를 함께 제시하였다. 연구 결과 혼합형 검사에서 얻어진 총점을 아크사인 방법으로 척도를 개발했을 때 다른 방법으로 척도를 개발했을 때 보다 조건적 측정오차의 크기가 균일했다.

다국어 초록 (Multilingual Abstract)

The purpose of this study was to describe the arcsine scaling procedure for equalizing measurement error for total scores on mixed format tests and to evaluate performance of the procedure in terms of a constant conditional standard errors of measurement (CSEM). The estimated observed scores using a mixture of the three‐parameter logistic and generalized partial credit IRT models were transformed to establish a reported scale score that was intended to possess the property of a constant CSEM. Four real mixed format tests were used to demonstrate the applicability of the arcsine scaling procedure to achieve a constant CSEM. The results showed that the arcsine scaling procedure performed well for mixed format tests resulting in approximately equalized CSEMs for base forms as well as equated forms. When compared to some other types of scale scores(T score, Stanine, and NCE), the arcsine transformed scale scores produced the most stable CSEMs.

목차 (Table of Contents)

• Ⅰ. Introduction
• Ⅱ. Method
• Ⅲ. Results
• Ⅳ. Discussion

참고문헌 (Reference)

1 반재천, "국가수준 학업성취도 평가의 척도 개발" 한국교육평가학회 19 (19): 113-135, 2006

2 Kolen, M. J., "Test equating, linking, and scaling. in: Methods and practices" Springer Verlag 2004

3 Lord, F. M., "Standard errors of measurement at different score levels" 21 : 239-243, 1984

4 Petersen, N. S., "Scaling, norming, and equating. in: Educational measurement" American Council on Education and Macmillan 221-262, 1989

5 Kolen, M. J., "Scaling the ACT Assessment. in: Methodology used in scaling the ACT Assessment and P-ACT+" ACT, Inc 35-55, 1989

6 Kolen, M. J., "Scaling and norming. in: Educational measurement" American Council on Education and Macmillan 154-186, 2006

7 Kim, S., "STUIRT. in: A computer program for scale transformation under unidimensional item response theory models" Iowa Testing Programs, The University of Iowa 2004

8 Brennan, R. L., "Raw-score conditional standard errors of measurement in generalizability theory" 22 : 307-331, 1998

9 Wang, T., "Psychometric properties of scale scores and performance levels for performance assessments using polytomous IRT" 37 : 141-162, 2000

10 Kolen, M. J., "POLYEQUATE: A computer program" University of Iowa

11 Muraki, E., "PARSCALE (version 4.1). in: IRT item analysis and test scoring for rating scale data" Scientific Software, Inc 2003

12 Lee, W., "Multinomial and compound multinomial error models for tests with complex item scoring" 31 (31): 255-274, 2007

13 Thissen, D., "Item response theory for scores on tests including polytomous items with ordered responses" 19 : 39-49, 1995

14 Lee, W., "Estimators of conditional scale-score standard errors of measurement: A simulation study" 37 : 1-20, 2000

15 Kolen, M. J., "Defining score scales in relation to measurement error" 25 : 97-110, 1988

16 Kolen, M. J., "Conditional standard errors of measurement for scale scores" 29 : 285-307, 1992

17 Brennan, R. L., "Conditional scale score standard errors of measurement under binomial and compound binomial assumptions" 59 : 5-24, 1999

18 Lord, F. M., "Comparison of IRT true score and equipercentile observed score “equatings.”" 8 : 452-461, 1984

19 Lord, F. M., "Applications of item response theory to practical testing problems" Lawrence Erlbaum 1980

20 Kim, S., "An extension of four IRT linking methods for mixed format tests" 43 : 53-76, 2006

21 Lee, W., "A strong true-score model for polytomous items" University of Iowa 2006

22 Muraki, E., "A generalized partial credit model: Application of an EM algorithm" 16 : 159-176, 1992

23 반재천, "2003~2008년 국가수준 학업성취도 평가의 조건적 측정오차와 분류일관성" 한국교육평가학회 23 (23): 323-345, 2010