구조방정식모형을 이용한 차별문항기능의 탐지: MACS와 MIMIC의 비교 = Detection of differential item functioning using structural equation modeling: A comparison of MACS and MIMIC

다국어 초록 (Multilingual Abstract)

Two models, MACS and MIMIC, can be used to detect Differential Item Functioning(DIF) in a Structural Equation Modeling framework. Although these two models can be considered as special cases of general Structural Equation Models, they may perform differently in various research contexts due to differences in statistical assumptions and the way in which each model uses data. In particular, since MIMIC model requires some additional assumptions, its performance may decline when those assumptions are not satisfied. Furthermore, the performance of MIMIC model will be superior to that of MACS model when sample sizes vary among groups because the former uses a single dataset including group variable(s), unlike the latter. Although many articles have commented on these predictions, no systematic research comparing the performance of the two models under these circumstances had yet to be conducted. Thus, we investigated the differences in performance of these two models under various conditions, specifically the size of impact, differences in measurement variable reliability, sample size ratio, total sample size, the size of differential item functioning, and the strategy for detecting DIF through a Monte Carlo simulation study. We found that the performance of MIMIC model in detecting uniform DIF did not decline significantly, although one of its additional assumptions was violated. Moreover, MIMIC model was superior to MACS model when sample sizes differed between two groups. Finally, we emphasize the importance of employing appropriate strategies for effective use of the two models to detect uniform DIF.

국문 초록 (Abstract)

구조방정식모형을 이용하여 차별문항기능을 탐지하는 방법으로 평균 및 공분산구조(MACS) 모형과 다지표-다원인(MIMIC) 모형의 두 가지를 들 수 있다. 두 모형은 모두 구조방정식모형의 특수한 경우에 해당하지만, 모형 간에 통계적 가정 및 자료의 입력 방식이 다르기 때문에 연구 상황에 따라 그 수행이 상이할 수 있다. 특히 MIMIC 모형은 MACS 모형에 비해 추가적인 가정을 필요로 하기 때문에, 가정이 위배될 경우 균일적 차별문항기능 탐지에 더 불리할 수 있다. 또한 MACS 모형과 달리 MIMIC 모형은 집단변수를 포함한 단일 입력자료를 사용하기 때문에, 집단 간에 표본크기가 상이할 경우 MIMIC 모형이 MACS 모형보다 균일적 차별문항기능 탐지에 더 유리할 것으로 예상할 수 있다. 이러한 가능성에 대해 여러 문헌에서 지적되었음에도 불구하고, 다양한 상황에서 두 모형의 차이를 체계적으로 비교한 연구는 발견되지 않았다. 따라서 다양한 연구 상황을 반영한 몬테 카를로 모의실험(Monte Carlo simulation)을 통해 균일적 차별문항기능 탐지에 대한 두 모형의 수행을 비교하였다. 구체적으로는 집단효과, 측정변수 신뢰도의 차이, 전체 표본크기, 표본크기의 비율, 차별문항기능의 크기, 차별문항기능 탐지 전략 등을 체계적으로 조작하였으며, 이에 따른 두 모형의 수행을 비교하였다. 비교 결과, 추가적인 가정이 위배되는 상황에서도 MIMIC 모형의 균일적 차별문항기능 탐지율이 MACS 모형에 비해 크게 저하되지 않았으며, 표본 크기가 집단 간에 다른 경우 MIMIC 모형이 MACS 모형보다 우수한 탐지율을 보였다. 단, 두 모형의 효과적인 사용을 위해서는 적절한 탐지 전략이 필요하므로, 이에 대해 논의한 후 현실적으로 바람직한 방안을 제안하였다.

참고문헌 (Reference)

1 성태제, "차별기능(편파성)문항 추출을 위한 Raju 방법과 MH 방법의 비교 연구" 6 (6): 91-12, 1993

2 김성훈, "문항차별기능의 인지심리학적 해석 가능성 검토" 8 (8): 39-62, 1995

3 이순묵, "다집단 분석의 문제: 평균구조분석에서의 측정원점 동일성 검증 필요 여부" 한국교육평가학회 23 (23): 391-416, 2010

4 김한조, "다집단 분석에서의 부분 동일성 검증 전략들 : 적절한 전략을 찾기 위한 시뮬레이션 연구" 성균관대학교 대학원 2010

5 Barendse, M. T., "Using restricted factor analysis with latent moderated structures to detect uniform and nonuniform measurement bias : a simulation study" 94 : 117-127, 2010

6 Thissen, D., "Use of item response theory in the study of group differences in trace lines, In Test Validity" Lawrence Erlbaum Associates, Inc 147-172, 1988

7 Hancock, G. R., "Type I error and power of latent mean methods and MANOVA in factorially invariant and noninvariant latent variable systems" 7 (7): 534-556, 2000

8 Lee, J. H., "Type I Error and Power of the Mean and Covariance Structure Confirmatory Factor Analysis for Differential Item Functioning Detection: Methodological Issues and Resolutions" The University of Kansas 2009

9 Rock, D. A., "The use of analysis of covariance structure for comparing the psychometric properties of multiple variables across populations" 13 : 408-418, 1978

10 Hogarty, K. Y., "The quality of factor solutions in Exploratory factor analysis : The influence of sample size, communality, and overdetermination" 65 (65): 202-226, 2005

11 Pennell, R., "The influence of communality and N on the sampling distributions of factor loadings" 33 (33): 423-439, 1968

12 Byrne, B. M., "Testing for the Equivalence of Factor Covariance and Mean Structures : The Issue of Partial Measurement Invariance" 105 (105): 456-466, 1989

13 Cohen, J., "Statistical Power Analysis for Behavioral Sciences" Lawrence Erlbaum Associates, Inc 1988

14 Jöreskog, K. G., "Statistical Analysis of sets of congeneric tests" 36 (36): 109-133, 1971

15 Oort, F. J., "Simulation study of item bias detection with restricted factor analysis" 5 (5): 107-124, 1998

16 Bentler, P. M., "Significant tests and goodness of fit in the analysis of covariance structure" 88 : 588-606, 1980

17 Drasgow, F., "Scrutinizing Psychological Tests : Measurement Equivalence and Equivalent Relations with External Variables" 95 : 134-135, 1984

18 MacCallum, R. C., "Sample size in factor analysis" 4 (4): 84-99, 1999

19 SAS Institution Inc., "SAS/STAT® 9.3 User's Guide" SAS Institute Inc 2011

20 Gelin, M. N., "Operating characteristics of the DIF MIMIC approach using Jöreskog’s covariance matrix with ML and WLS estimation for short scales" 6 (6): 573-588, 2007

21 Meredith, W., "On the misuse of manifest variables in the detection of measurement bias" 57 (57): 289-311, 1992

22 Smith, L. L., "On the Usefulness of Item Bias Analysis to Personality Psychology" 28 : 754-763, 2002

23 Muthén, L. K., "Mplus User’s Guide" Muthén & Muthén 2010

24 Paxton, P., "Monte Carlo Experiments : Design and Implementation" 8 (8): 287-312, 2001

25 Millsap, R. E., "Methodological Review: Statistical Approaches for Assessing Measurement Bias" 17 (17): 297-334, 1993

26 Meredith, W., "Measurement Invariance, Factor Analysis and Factorial Invariance" 58 (58): 525-543, 1993

27 Schmitt, N., "Measurement Invariance : Review of practice and implications" 18 : 210-222, 2006

28 Raju, N. S., "Measurement Equivalence : A Comparison of Methods Based on Confirmatory Factor Analysis and Item Response Theory" 87 (87): 517-529, 2002

29 Little, T. D., "Mean and Covariance Structures(MACS)Analyses of Cross-Cultural Data : Practical and Theoretical Issues" 32 (32): 53-76, 1997

30 Miller, T. R., "Logistic Discriminant Function Analysis for DIF Identification of Polytomously Scored Items" 30 (30): 107-122, 1993

31 Muthén, B. O., "Latent variable structural equation modeling with categorical data" 22 : 43-65, 1983

32 Jöreskog, K. G., "LISREL 7: A Guide to the Program and Applications" SPSS Publications 1989

33 Marsh, H. W., "Goodness-of-fit indices in comfirmatory factor analysis : the effect of sample size" 103 : 391-341, 1988

34 Jöreskog, K. G., "Estimation of a Model with Multiple Indicator and Multiple Causes of a Single Latent Variable" 70 (70): 631-639, 1975

35 Holland, P. W., "Differential Item Functioning and the Mantel-Haenszel Procedure, In Test Validity" Lawrence Erlbaum Associates, Inc 129-145, 1988

36 Chan, D., "Detection of Differential Item Functioning on the Kirton Adaptation-Innovation Inventory Using Multiple-Group Mean and Covariance Structure Analyses" 35 (35): 169-199, 2000

37 Yoon, M., "Detecting Violations of Factorial Invariance Using Data-Based Specification Searches : A Monte Carlo Study" 14 (14): 435-463, 2007

38 Stark, S., "Detecting Differential Item Functioning With Confirmatory Factor Analysis and Item Response Theory : Toward a Unified Strategy" 91 (91): 1292-1306, 2006

39 Mellenberg, G. J., "Contingency Table Models for Assessing Item Bias" 7 (7): 105-118, 1982

40 Brown, T. A., "Confirmatory Factor Analysis for Applied Research" The Guilford Press 2006

41 Lubke, G. H., "Can unequal residual variances across groups mask differences in residual means in the common factor model?" 10 (10): 175-192, 2003

42 Steenkamp, J. E. M., "Assessing Measurement Invariance in Cross-National Consumer Research" 25 : 78-79, 1998

43 Lord, F. M., "Application of item response theory to practical testing problems" Lawrence Erlbaum Associates, Inc 1980

44 Kaplan, D., "A study of the power associated with testing factor mean differences under violations of factorial invariance" 2 (2): 101-118, 1995

45 Muthén, B. O., "A structural probit model with latent variables" 74 : 807-811, 1979

46 Sörbom, D., "A general method for studying differences in factor means and factor structure between groups" 27 : 229-239, 1974

47 Maydeu-Olivares, A., "A cautionary note on using G2(dif)to assess relative model fit in categorical data analysis" 41 : 55-64, 2006

48 Vandenberg, R. J., "A Review and Synthesis of the Measurement Invariance Literature : Suggestions, Practices, and Recommendations for Organizational Research" 3 (3): 4-7, 2000

49 Shealy, R., "A Model-based Standardization Approach that separates true bias/DIF from Group ability differences and Detects Test Bias/DTF as well as Item Bias/DIF" 58 (58): 159-194, 1993

50 Ackerman, T. A., "A Didactic Explanation of Item Bias, Item Impact, and Item Validity from a Multidimensional Perspective" 29 (29): 67-91, 1992

51 Camilli, G., "A Critique of the chi-square method assessing item bias" Laboratory of Educational Research, University of Colorado at Boulder 1979

52 Meade, A. W., "A Comparison of Item Response Theory and Confirmatory Factor Analytic Methodologies for Establishing Measurement Equivalence/Invariance" 7 : 361-388, 2004