A Comparison of Parametric and Nonparametric DIF Detection Methods

국문 초록 (Abstract)

차별적 문항기능은 문항의 편파성을 찾아내려는 통계적 기법으로 연구되고 있다. 본 연구는 자료가 비모수적으로 생성되었을 때 우도비 검정과 같은 모수적 방법과 SIBTEST, Mantel-Haenszel 검사와 같은 비모수적 방법이 어떻게 작동하는 지 비교하였다. 모의실험에서는 수험자의 수와 능력의 분포가 달라질 때의 변화를 보았다. 제 1종 오류와 함께 차별적 기능 문항이 전체 문항에서 차지하는 비율이 달라질 때의 검증력 또한 조사했다. 그 결과 Mantel-Haenszel 방법이 제 1종 오류를 가장 잘 통제하였으나 SIBTEST의 결과 또한 유사했다. 우도비 검증법은 일부 조건에서 상당한 제 1종 오류의 증가를 보였다. 검증력에 있어서 역시 대체로 우도비 검증법의 결과가 비모수적 방법에 비해 좋지 않았다. 집단 내 피험자의 수가 증가함에 따라 모든 방법의 검증력은 좋아졌지만 집단의 크기가 커져도 모수적 방법이 비모수적 방법보다 나은 결과를 보이지는 못했다. 연구의 한계와 연구방향 또한 논의되었다.

다국어 초록 (Multilingual Abstract)

Differential item functioning (DIF) has been studied as a statistical method of finding possible item bias. The current study compared statistical performances of parametric—the Likelihood Ratio test (LR-test)—and nonparametric—SIBTEST and the Mantel-Haenszel test (MH-test)—DIF detection methods when datasets were nonparametrically simulated. Simulation factors included sample size and latent ability distribution. Type I error rates and the power of detecting DIF items with varying percentages of DIF items were investigated. The results showed that the MH-test controlled the type I error rate best across all conditions, followed closely by SIBTEST. The LR-test showed substantial inflation of type I error rates under some conditions. The LR-test's power of detecting DIF was inferior to that of the nonparametric methods. Although increasing sample size improved the performance of all methods, a large sample size did not guarantee the superiority of parametric methods. The limitations of this study and future directions were also discussed.

참고문헌 (Reference)

1 Thissen, D, "Use of item response theory in the study of group differences in trace lines" Erlbaum 147-169, 1988

2 Raju,N.S, "The area between two item characteristic curves" Psychometrika 53 : 495-502, 1988

3 Bolt, D. M, "Testing features of graphical DIF: Application of a regression correction to three nonparametric statistical tests" Journal of Educational Measurement 43 : 313-333, 2006

4 Ramsay,J.O, "TESTGRAF manual" McGill University 2000

5 Stout, W. F, "SIBTEST user manual"

6 DeMars,C.E, "Modification of the Mantel-Haenszel and logistic regression DIF procedures to incorporate the SIBTEST regression correction" Journal of Educational and Behavioral Statistics 34 : 149-170, 2009

7 Camilli, G, "Methods for identifying biased test items" Sage 1994

8 Woods,C.M, "Likelihood-ratio DIF testing: Effects of nonnormality" Applied Psychological Measurement 32 : 511-526, 2008

9 Ramsay,J.O, "Kernel smoothing approaches to nonparametric item characteristic curve estimation" Psychometrika 56 : 611-630, 1991

10 Thissen,D, "IRTLRDIF (Version 2)"

11 Raju, N. S, "IRT-based internal measures of differential functioning of items and tests" Applied Psychological Measurement 19 : 353-368, 1995

12 De Boeck, P, "Explanatory item response models: A generalized linear and nonlinear approach" Springer-Verlag 2004

13 Holland, P. W, "Differential item performance and the Mantel-Haenszel procedure" Erlbaum 129-145, 1988

14 Penfield, R. D, "Differential item functioning and item bias: Handbook of statistics" Psychometrics 2007

15 Williams, N. J, "DIF identification using HGLM for polytomous items" Applied Psychological Measurement 30 : 22-42, 2006

16 Penfield, R. D, "Assessing differential item functioning in performance assessment: Review and recommendations" Educational Measurement: Issues and Practice 19 : 5-15, 2000

17 Lord,F.M, "Applications of item response theory to practical testing problems" Erlbaum 1980

18 Swanson, D. B, "Analysis of differential item functioning (DIF) using hierarchical logistic regression models" Journal of Educational and Behavioral Statistics 27 : 53-75, 2002

19 Shealy, R, "An item response theory model for test bias and differential test functioning" Erlbaum 197-240, 1993

20 Cohen, A. S, "An investigation of the likelihood ratio test for detection of differential item functioning" Applied Psychological Measurement 20 : 15-26, 1996

21 Li, H, "A new procedure for detection of crossing DIF" Psychometrika 61 : 647-677, 1996

22 Shealy, R, "A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF" Psychometrika 58 : 159-194, 1993

23 Bolt,D.M, "A Monte Carlo comparison of parametric and nonparametric polytomous DIF detection methods" Applied Measurement in Education 15 : 113-141, 2002