Empirical Comparisons of Analytic Strategies for MIMIC DIF Analysis: A Potential Solution for Biased Anchor Set

다국어 초록 (Multilingual Abstract)

The purpose of this Monte Carlo study was to evaluate the performance of the multiple indicators and multiple causes (MIMIC) confirmatory factor analysis (CFA) for detecting differential item functioning (DIF). Specifically, this study compared different application strategies including two conventional testing approaches (forward-inclusion, backward-elimination) and five test statistic values (uncorrected or Bonferroni-corrected LR, △CFI of 0.01 or 0.002, △SRMR of 0.005) across conditions of different item type, test length, sample size, impact, and DIF type and DIF size in a target item and an anchor set. In addition, the author proposed an alternative testing approach (effects-coded backward-elimination) as a potential solution for arbitrary choice of a DIF-free anchor set. Simulation results indicated that when an anchor set was truly biased, only the proposed approach performed adequately under several conditions. False positive rates were controlled at the nominal alpha level (with Bonferroni-corrected LR) or slightly inflated (with uncorrected LR) as the DIF contamination rate in a scale decreased.

참고문헌 (Reference)

1 MacIntosh, R., "Variance estimation for converting MIMIC model parameters to IRT parameters in DIF analysis" 27 : 372-379, 2003

2 Fleishman,J.A, "Using MIMIC models to assess the influence of differential item functioning"

3 Lee,J, "Type I error and power of the MACS CFA for DIF detection: Methodological issues and resolutions" University of Kansas 2009

4 Christensen, H, "The “common cause hypothesis” of cognitive aging: Evidence for not only a common factor but also specific associations of age with vision and grip strength in a cross-sectional analysis" 16 : 588-599, 2001

5 Finch,H, "The MIMIC model as a method for detecting DIF: Comparison with Mantel-Haenszel, SIBTEST and the IRT likelihood ratio test" 29 : 278-295, 2005

6 Byrne, B. M, "The MACS approach to testing for multigroup invariance of a second-order structure: A walk through the process" 13 : 287-321, 2006

7 Woods, C. M., "Testing for nonuniform differential item functioning with multiple indictor multiple cause models" 35 : 339-361, 2011

8 Woods,C.M, "Testing for differential item functioning with measures of partial association" 33 : 538-554, 2009

9 Cheung, G. W, "Testing factorial invariance across groups: A reconceptualization and proposed new method" 25 : 1-27, 1999

10 McDonald,R.P, "Test theory: Unified treatment" Lawrence Erlbaum Associates 1999

11 Bollen,K.A, "Structural equations with latent variables" Wiley 1989

12 Barrett,P, "Structural equation modeling: Adjusting model fit" 42 : 815-824, 2007

13 American Educational Research Association, "Standards for educational and psychological testing" American Educational Research Association 1999

14 Muthén, B. O, "Some uses of structural equation modeling in validity studies: Extending IRT to external variables, in Test Validity" Lawrence Erlbaum 213-238, 1988

15 Chen,F.F, "Sensitivity of goodness of fit indexes to lack of measurement invariance" 14 : 464-504, 2007

16 Drasgow,F, "Scrutinizing psychological tests: Measurement equivalence and equivalent relations with external variables are central issues" 95 : 134-135, 1984

17 SAS Institute, "SAS/STAT 9.2 user's guide" SAS Institute Inc 2002

18 Meade, A. W, "Power and sensitivity of alternative fit indices in test of measurement invariance" 93 : 568-592, 2008

19 González-Romá, V, "Power and Type I error of the mean and covariance structure analysis model for detecting differential item functioning in graded response items" 41 : 29-53, 2006

20 Mehta, P. D, "People are variables too: Multilevel structural equations modeling" 10 : 259-284, 2005

21 Teresi,J.A, "Overview of quantitative measurement methods: Equivalence, invariance, and differential item functioning in health applications" 44 : 39-49, 2006

22 Muthén, L.K, "Mplus user’s guide. (6th Ed.)" Muthén & Muthén 1998

23 Lee, J, "Methodological issues in using structural equation models for testing differential item functioning, in Cross-cultural data analysis: Methods and applications" Routledge 57-86, 2010

24 Camilli, G, "Measurement methods for the social sciences series: Methods for identifying biased test items (Vol. 4)" Sage 1994

25 Raju, N. S., "Measurement equivalence: A comparison of methods based on confirmatory factor analysis and item response theory" 87 : 517-529, 2002

26 Little,T.D, "Mean and covariance structures (MACS) analyses of cross-cultural data: Practical and theoretical issues" 32 : 53-76, 1997

27 Muthén, B. O, "Latent variable analysis with categorical outcomes: Multiple-group and growth modeling in Mplus" University of California and Muthén & Muthén 2002

28 Muthén, B. O., "Instructionally sensitive psychometrics: Application of a new IRT-based detection technique to mathematics achievement test items" 28 : 1-22, 1991

29 Fleishman, J. A, "Impact of differential item functioning on age and gender differences in functional disability" 57 : 275-283, 2002

30 Woods, C. M, "Illustration of MIMIC-Model DIF Testing with the Schedule for Nonadaptive and Adaptive Personality" 31 : 320-330, 2009

31 Jones,R.N, "Identification of measurement differences between English and Spanish language versions of the Mini-Mental State Examination: Detecting differential item functioning using MIMIC modeling" 44 : 124-133, 2006

32 Raudenbush, S. W, "Hierarchical linear models: Applications and data analysis methods (2nd ed.)" Sage 2002

33 Millsap, R. E, "Four unresolved problems in studies of factorial invariance, in Contemporary psychometrics" Lawrence Erlbaum Associates, Inc 153-172, 2005

34 Stark, S, "Examining the effects of differential item/test functioning (DIF/DTF) on selection decisions: When are statistically significant effects practically important" 89 : 497-508, 2004

35 Woods,C.M, "Evaluation of MIMIC-model methods for DIF testing with comparison to two-group anlaysis" 44 : 1-27, 2009

36 Cheung, G. W., "Evaluating goodness-of-fit indexes for testing measurement invariance" 9 : 233-255, 2002

37 Samejima,F, "Estimation of latent ability using a response pattern of graded scores"

38 Jöreskog, K. G, "Estimation of a model with multiple indicators and multiple causes of a single latent variable" 10 : 631-639, 1975

39 Finch, H, "Estimation of MIMIC model parameters with multilevel data" 18 : 229-252, 2011

40 Navas-Ara, M. J, "Effects of ability scale purification on identification of DIF" 18 : 9-15, 2002

41 Stark, S., "Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy" 91 : 1292-1306, 2006

42 Dorans, N. J, "DIF detection and description: Mantel Haenszel and standardization, in Differential Item Functioning" Lawrence Erlbaum 35-66, 1993

43 Brannick,M.T, "Critical comments on applying covariance structure modeling" 16 : 201-213, 1995

44 Reise, S. P, "Confirmatory Factor Analysis and item response theory: Two approaches for exploring measurement invariance" 114 : 552-566, 1993

45 Everson, H. T, "Beyond individual differences: Exploring school effects on SAT scores" 39 : 157-172, 2004

46 Gallo, J. J, "Age differences in the symptoms of depression: A latent trait analysis" 49 : 251-264, 1994

47 Mellenbergh, G. J, "A unidimensional latent trait model for continuous item responses" 29 : 223-237, 1994

48 Mackinnon, A., "A short form of the Positive and Negative Affect Schedule: Evaluation of factorial validity and invariance across demographic variables in a community sample" 27 : 405-416, 1999

49 Vandenberg, R. J, "A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research" 3 : 4-69, 2000

50 Kamata, A., "A note on the relation between factor analytic and item response theory models" 15 : 136-153, 2008

51 Little, T. D., "A non-arbitrary method of identifying and scaling latent variables in SEM and MACS models" 13 : 59-72, 2006

52 Sörbom,D, "A general method for studying differences in factor means and factor structure between groups" 27 : 229-239, 1974

53 Maydeu-Olivares, A, "A cautionary note on using G2 (dif) to assess relative model fit in categorical data analysis" 41 : 55-64, 2006

54 Meade, A. W, "A Monte-Carlo study of confirmatory factor analytic tests of measurement equivalence/ invariance" 11 : 60-72, 2004