교육 및 심리검사의 구인 타당도 검증을 위한 완전-정보 문항 요인분석의 적용 = The Application of Full-information Item Factor Analysis to Assess Construct Validity of An Educational And Psychological Test

다국어 초록 (Multilingual Abstract)

  Even though exploratory linear factor analysis using a product moment correlation matrix is commonly applied to assess construct validity of an educational and psychological test, this method is not appropriate for item-level data having categorical response format because traditional factor analysis assumes continuous ratings and normality. The use of tetrachoric correlations may be a theoretically better alternative, but it is known that it may have some practical problems and does not necessarily provide satisfactory improvement in results. This study introduces full-information item factor analysis which is known to be free from such problems and compares its performance in assessing construct validity with those of other factor analysis methods. In addition, a multidimensional item response theory model which is mathematically identical to full-information item factor analysis is also described, and its complementary use is illustrated with a real dataset from a mathematical achievement test. Finally, through a simple simulation study, the three factory analysis methods are compared and a problem of the use of product moment correlation matrix is identified.

국문 초록 (Abstract)

  교육 및 심리검사의 구인 타당도(construct validity)를 경험적으로 검증하기 위해 흔히 사용되는 문항 간의 적률상관행렬에 기반을 둔 선형적 요인분석은 범주형 자료로서의 각 문항 변수를 연속 변수로 본다거나 충족되기 어려운 정규성을 가정하는 등의 문제점을 가지고 있다. 사분상관계수(tetrachoric correlation coefficient)를 기본 자료로 사용하는 요인분석은 이보다 이론적으로 적합한 방법이기는 하지만, 구인타당도 검증에 있어서 실행상의 문제점이 발생할 수도 있는 것으로 알려져 있다. 이 논문에서는 이러한 문제점들로부터 자유롭기 때문에 보다 나은 구인 타당도 검증이 가능하리라고 기대되는, 완전-정보 문항  요인분석 (FIFA: full-information item factor analysis) 방법을 소개하고 앞의 방법들과 비교하여 그 차이점을 분석한다. 또한, FIFA와 수리적으로 동일한 방법론인 다차원 문항반응이론(MIRT: multidimensional item response theory) 모형을 함께 소개하고 그 관계를 설명한 후에, 상호보완적인 이용을 예시한다. 수학 성취도 측정을 위한 실제 검사 자료를 이용하여 1) 적률상관계수를 사용한 요인분석, 2) 사분상관계수를 사용한 요인분석, 그리고 3) FIFA를 적용했을 때 그 결과상의 차이점을 비교한 후, 같은 자료에 대하여 FIFA와 MIRT 모형을 통하여 구인을 확인해 나가는 과정을 제시하였다. 마지막으로, 세 방법론 간의 비교를 위한 간단한 시뮬레이션 연구를 실시하여 적률상관행렬을 이용한 요인분석의 문제점을 확인하였다.

목차 (Table of Contents)

• Ⅰ. 서론
  Ⅱ. 수학 성취도 검사의 구인 타당도: PMFA, TCFA, 그리고 FIFA의 적용 및 비교
  Ⅲ. FIFA (cMIRT 모형)를 이용한 구인 타당도 검증의 실제
  Ⅳ. PMFA, TCFA, 그리고 FIFA의 비교를 위한 시뮬레이션 연구
  Ⅴ. 논의 및 결론
  참고문헌
  저자소개
  〈ABSTRACT〉
  부록 A : TESTFACT 코드와 관련 자료의 형태
  

참고문헌 (Reference)

1 "학교학습과 교육평가. (개정판)" 서울: 교육과학사 1998

2 "타당도와 신뢰도" 서울: 양서원 1995

3 "요인분석의 기초" 서울: 교육과학사 2000

4 "dimensionality for a set of item response data? In D Modern theories of measurement University of Ottawa" 87-92, 1994

5 "Unidimensional IRT calibration of compensatory and noncompensatory multidimensional items" 113-127, 1989

6 "The past and future of multidimensional item response model" 21 : 25-36, 1997

7 "The discriminating power of items that measure more than one dimension" 361-373, 1991

8 "Technical report of the NAEP 1996 State Assessment Program in Mathematics National Center for Education Statistics" Washington, D.C.: National Center for Education Statistics 1997

9 "TESTFACT: Test scoring, item statistics, and item factor analysis [Computer program manual]" Chicago: Scientific Software International 1998

10 "TESTFACT: Test scoring, item statistics, and item factor analysis [Computer program manual]" Chicago: Scientific Software International 1998

11 "TESTFACT 4.0 [Computer Program and manual]" Lincolnwood, IL: Scientific Software International -4, 2003

12 "Some generalizations of the Rasch model An application of the hierarchical generalized linear model" 1998

13 "Scoring subscales using multidimensional item response theory models. Poster presented at the annual meeting of the American Psychology Association" 2005

14 "Robustness of item parameter estimation programs to assumptions of unidimensionality and normality" 25 : 146-162, 2001

15 "PRELIS 2: User's Reference Guide." [Computer Program] Lincolnwood: Scientific Software International 1999

16 "POLYFACT version 2 [computer program]" Princeton, NJ: Educational Testing Service 1999

17 "New technique for the dimensionality assessment of standardized test data University of Illinois at Urbana-Champaign" 55-, 1994

18 "Model selection methods for unidimensional and multidimensional IRT models" 2006

19 "Marginal maximum likelihood estimation of item parameters application of an EM algorithm" 443-459, 1981

20 "Item response theory: Parameter estimation techniques (2nd ed.)" New York: Marcel Dekker 2004

21 "Item response theory for psychologists" Mahwah, NJ, US: Lawrence Erlbaum Associates, Inc., Publishers 2000

22 "Introduction to Modern and Classical Test Theory" 1986

23 "Full-information factor analysis for polytomous item responses Paper presented at the Annual Meeting of the American Educational Research Association" 1993

24 "Factor analysis for items or testlets scored in more than two categories" Mahwah, NJ, US: Lawrence Erlbaum Associates, Inc., Publishers 141-186, 2001

25 "Factor analysis" Hoboken, NJ: John Wiley 2 : 143-164, 2003

26 "Factor Analysis" 1998

27 "Exploring nonlinear models in personality assessment Development and preliminary validation of a negative emotionality scale" 64 : 545-576, 1996

28 "Estimation of compensatory and noncompensatory multidimensional item response models using Markov chain Monte Carlo" 27 : 395-414, 2003

29 "Empirical comparison between factor analysis and multidimensional item response models" 457-477, 1991

30 "Comparing multidimensional and unidimensional proficiency classifications: Multidimensional IRT as a diagnostic aid" 40 : 255-275, 2003

31 "Application of modern psychometric theory in personality" 317-349, 1985

32 "An empirical comparison of multidimensional item response data using TESTFACT and NOHARM" New Orleans, Louisiana 2002

33 "A simulation comparison of parametric and nonparametric dimensionality detection procedures" 19 : 67-91, 2006

34 "A comparison of selected empirical methods for assessing the structure of responses to test items" 27 : 159-203, 2003

35 "A comparative study of test data dimensionality assessment procedures under nonparametric IRT models" 28 : 3-24, 2004