IRT 공통 능력척도 개발을 위한 공통-피험자 척도연계 방법과 공통-문항 척도연계 방법의 상대적 기능 연구 = An Investigation of the Relative Performance of the IRT Scale Linking Methods Using Common Examinees versus Common Items to Develop a Common Scale

국문 초록 (Abstract)

본 연구의 목적은 문항반응이론(IRT) 능력척도 연계를 위해 최대우도(ML) 능력추정치와 Bayes 사후기대(EAP) 능력추정치를 각각 이용하는 공통-피험자 Mean-Sigma 방법(CE ML & CE EAP)의 수행 기능을 공통-문항 Mean-Sigma 방법(CI MS)과 Stocking-Lord 검사특성곡선 방법(CI SL)의 수행 기능과 비교하여 분석하는 것이다. 컴퓨터 모의실험을 통해 피험자 집단 간의 비동등성의 수준이 공통-문항 설계와 공통-피험자 설계 모두에 일정하도록 통제한 다양한 척도 연계 조건 하에서 네 가지 척도연계 방법의 수행을 비교하였다. 모의실험 결과, CE ML 방법은 표준오차가 크지만 편의(bias)가 낮은 방식으로 척도연계를 수행하는 반면, CE EAP 방법은 표준오차는 작지만 편의는 큰 방식으로 수행하였다. 이와 관련하여, CE EAP 방법은 표본의 크기가 작을 때(예, 500) 상대적으로 안정적인 수행을 보인 반면, CE ML 방법은 표본의 크기가 클 때(예, 3,000) 상대적으로 우수한 수행을 보였다. 표본크기와 상관없이, 척도연계의 안정성과 정확성에 있어 10% (혹은 20%) 공통 피험자 비율을 이용하는 CE ML 방법은 40개의 문항 중 10개의 (혹은 20개의) 공통 문항을 이용하는 CI MS 방법과 유사한 수행 경향을 보였다.

다국어 초록 (Multilingual Abstract)

The purpose of this study is to investigate the performance of the mean-sigma (MS) IRT scale linking methods using maximum likelihood (ML) ability estimates versus Bayes expected a posteriori (EAP) ability estimates for common examinees (CE), in comparison to the MS and Stocking-Lord (SL) methods using common items (CI). The relative performance of the two CE (CE ML & CE EAP) methods and the two CI (CI MS & CI SL) methods was examined by using computer simulations to control the nonequivalence level between equating groups equally for both the CE and CI designs. As a main result, the CE ML method tended to yield larger standard error but lower bias in the process of scale linking than did the CE EAP method. In addition, the CE EAP method peformed more stably than the CE ML method when the sample size was as small as 500, whereas the latter outperformed in linking accuracy the former when the sample size was as large as 3,000. Regardless of sample size, in stability and accuracy of scale linking, the CE ML method using 10% (or 20%) common examinees of the total sample size tended to perform similarly to the CI MS method using 10 (or 20) common items out of the total 40 items.

참고문헌 (Reference)

1 Loyd, B. H, "Vertical equating using the Rasch model" Journal of Educational Measurement 17 : 179-193, 1980

2 Kolen, M. J, "Test equating, scaling, and linking: Methods and practices (2nd ed.)" Springer 2004

3 Kim, S, "STUIRT: A computer program for scale transformation under unidimensional item response theory models" 2004

4 Hanson, B. A, "Obtaining a common scale for item response theory item parameters using separate versus concurrent estimation in the common item equating design" Applied Psychological Measurement 26 : 3-24, 2002

5 Ogasawara,H, "Marginal maximum likelihood estimation of item response theory (IRT) equating coefficients for the common-examinee design" Japanese Psychological Research 43 (43): 72-82, 2001

6 Bock, R. D, "Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm" Psychometrika 46 : 443-459, 1981

7 Ogasawara,H, "Least squares estimation of item response theory linking coefficients" Applied Psychological Measurement 25 : 373-383, 2001

8 Hambleton, R. K, "Item response theory: Principles and applications" Kluwer 1985

9 Marco,G.L, "Item characteristic curve solutions to three intractable testing problems" Journal of Educational Measurement 14 : 139-160, 1977

10 Haebara,T, "Equating logistic ability scales by a weighted least squares method" Japanese Psychological Research 22 : 144-149, 1980

11 Kim, S, "Effects on scale linking of different definitions of criterion functions for the IRT characteristic curve methods" Journal of Educational and Behavioral Statistics 32 : 371-397, 2007

12 Stocking, M. L, "Developing a common metric in item response theory" Applied Psychological Measurement 7 : 201-210, 1983

13 Kim,S, "CE_ST: A computer program for IRT scale transformation using common examinees' ability parameters" 2009

14 Zimowski, M. F, "BILOG-MG: Multiple-group IRT analysis and test maintenance for binary items" Scientific Software International 2003

15 Mislevy, R. J, "BILOG 3: Item analysis and test scoring with binary logistic models" Scientific Software 1990

16 Lord,F.M, "Applications of item response theory to practical testing problems" Erlbaum 1980

17 Mislevy, R. J, "Adaptive EAP estimation of ability in a microcomputer environment" Applied Psychological Measurement 6 : 431-444, 1982

18 Divgi,D.R, "A minimum chi-square methods for developing a common metric in item response theory" Applied Psychological Measurement 9 : 413-415, 1985