자기보고식 핵심역량 측정에서 응답양식의 탐색 및 효과 : 다차원 명명반응모형 및 문항반응나무모형의 적용 = Response Style Analysis in the Self-report Measures of Core Competencies using Multidimensional Nominal Response Model (MNRM) and Item Response Tree Model (IRTree)

국문 초록 (Abstract)

본 연구는 평정형 척도를 활용한 자기보고식 핵심역량 검사에서 다차원 명명반응모형(Multidimensional Nominal Response Model: MNRM) 및 문항반응나무모형(Item Response Tree: IRTree)을 활용하여 응답양식(response style)을 측정하였다. 구체적으로 국내 핵심역량 검사에서 대표적으로 사용되고 있는 5점 리커르트식 척도에서 양극단에 있는 범주를 선호하는 극단응답양식(Extreme Response Style: ERS)과 중간범주를 선호하는 중간응답양식(Midpoint Response Style: MRS)에 초점을 두어 탐색하였다. 한국교육종단연구2013의 대규모 학생 역량조사 자료를 활용하여 분석한 결과는 다음과 같다. 첫째, MNRM과 IRTree 모두에서 응답양식을 통제하지 않은 모형에 비해 응답양식을 통제한 모형의 적합도가 보다 양호하였다. 둘째, ERS나 MRS 중 1개의 응답양식만 통제하였을 때보다 ERS 및 MRS를 동시에 통제하였을 때 모형의 적합도가 더 양호하였다. 셋째, MNRM과 IRTree 중 본 자료에는 MNRM에 기반한 응답양식 탐색 모형의 수행능력이 더 양호한 것으로 나타났다. 넷째, 응답양식을 통제한 후 핵심역량 간 상관계수의 강도는 축소되는 경향을 보였다. 이러한 연구결과에 기반하여 신뢰롭고 타당한 핵심역량 측정을 위한 시사점을 도출하였다.

다국어 초록 (Multilingual Abstract)

This study aims to investigate response style in students’ competencies assessment with rating scale. Response style was examined by using two item response theory (IRT) frameworks: multidimensional nominal response model (MNRM) and item response tree model (IRTree) using empirical data sets of students’ competencies from Korean Educational Longitudinal Survey (KELS: 2013) 4th (8th graders). The findings are as follows. First, the models that control for response styles generally showed a better fit than the model that do not take response styles into account. Second, among the models that account for response styles, the models with both ERS and MRS fitted better than the models with either ERS or MRS. Third, as a result of using the two IRT frameworks such as MNRM and IRTree to examine students’ response style in competency assessment, MNRM showed a better fit than IRTree and it also had good discriminant validity between the competency trait and response styles. Forth, the correlations between ten sub-scales of key competencies decreased when response style was considered. Finally, theoretical and applied implications for future research were discussed.

목차 (Table of Contents)

• Ⅰ. 서 론 Ⅱ. 응답양식 탐색 모형 Ⅲ. 연구 방법 Ⅳ. 연구결과 Ⅴ. 요약 및 논의

참고문헌 (Reference)

1 김경희, "한국 초․중등 학교교육 성과종단조사 체제 구축을 위한 기초 연구(Ⅰ)" 한국교육과정평가원 2017

2 민무숙, "청소년의 다문화수용성 조사 연구(2012-51)" 한국여성정책연구원 2012

3 성은모, "청소년 역량지수 측정 및 국제비교 연구Ⅰ: IEA ICCS 2016 -총괄보고서(14-R20)" 한국청소년정책연구원 2014

4 손원숙, "자기보고식 설문 응답척도 사용의 개인차 및 일관성" 한국교육학회 55 (55): 23-43, 2017

5 이순묵, "요인분석의 기초" 교육과학사 2000

6 손원숙, "심리측정에서 비극단 응답양식의 효과: 다집단 잠재계층 요인분석의 적용" 한국교육학회 54 (54): 105-125, 2016

7 장윤선, "문항반응나무(IRTree) 모형: 자기보고식 서열척도 자료 분석의 대안적 방법" 한국교육평가학회 32 (32): 303-323, 2019

8 Leventhal, B. C., "treme response style: which model is best?" University of Pittsburgh 2017

9 Chalmers, R., P, "mirt : A multidimensional item response theory package for the R environment" 48 (48): 1-29, 2012

10 Zhang, Y., "Validity of three IRT models for measuring and controlling extreme and midpoint response styles" 11 : 271-, 2020

11 Lucci, M., "Using mixture, multi-process, and other multi-dimensional IRT models to account for extreme and midpoint response style use in personality assessment" University of Pittsburgh 2017

12 Mõttus, R., "The effect of response style on self-reported Conscientiousness across 20countries" 38 (38): 1423-1436, 2012

13 Moors, G., "The effect of response style bias on the measurement of transformational, transactional, and laissez-faire leadership" 21 (21): 271-298, 2012

14 Spratto, E. M., "Seeing the Forest and the Trees : Comparison of Two IRTree Models to Investigate the Impact of Full Versus Endpoint-Only Response Option Labeling" 81 (81): 39-60, 2021

15 van Vaerenbergh, Y., "Response styles in survey research : A literature review of antecedents, consequences, and remedies" 25 (25): 195-217, 2013

16 Baumgartner, H., "Response styles in marketing research : A cross-national investigation" 38 (38): 143-156, 2001

17 Böckenholt, U., "Response style analysis with threshold and multi-process IRT models : A review and tutorial" 70 (70): 159-181, 2017

18 Khorramdel, L., "Recent IRT approaches to test and correct for response styles in PISA background questionnaire data : A feasibility study" 59 (59): 71-92, 2017

19 Bolt, D. M., "Multiscale measurement of extreme response style" 71 (71): 814-833, 2011

20 Wetzel, E., "Multidimensional modeling of traits and response styles" 33 (33): 352-364, 2017

21 Muthén, L. K, "Mplus user’s guide" Muthén & Muthén 2017

22 Ju, U., "Modeling response styles in cross-country self-reports : An application of a multilevel multidimensional nominal response model" 56 (56): 169-191, 2019

23 Böckenholt, U., "Modeling multiple response processes in judgment and choice" 17 (17): 665-678, 2012

24 Huang, H. -Y., "Mixture random-effect IRT models for controlling extreme response style on rating scales" 7 : 1706-, 2016

25 Böckenholt, U., "Measuring response styles in likert items" 22 (22): 69-83, 2017

26 Khorramdel, L., "Measuring response styles across the big five : A multiscale extension of an approach using multinomial processing trees" 49 (49): 161-177, 2014

27 남궁지영, "KEDI 학생역량 조사 연구: 조사 도구 개발 및 타당화" 한국교육개발원 2015

28 현 주, "KEDI 인성검사 실시요강(CRM 2014-111)" 한국교육개발원 2014

29 Aichholzer J., "Intra-individual variation of extreme response style in mixed-mode panel studies" 42 (42): 957-970, 2013

30 Austin, E. J., "Individual differences in response scale use : Mixed Rasch modelling of responses to NEO-FFI items" 40 (40): 1235-1245, 2006

31 Thissen, D., "Handbook of polytomous item response theory models" Routledge/Taylor & Francis Group. 43-75, 2010

32 Jeon, M., "Evaluation on types of invariance in studying extreme response bias with an IRTree approach" 72 (72): 517-537, 2019

33 Samejima, F., "Estimation of latent ability using a response pattern of graded scores" 34 (34): 100-, 1969

34 Schwarz, G., "Estimating the Dimension of a Model" 6 : 461-464, 1978

35 Bock, R. D., "Estimating item parameters and latent ability when responses are scored in two or more nominal categories" 37 (37): 29-51, 1972

36 Wetzel, E., "Consistency of extreme response style and non-extreme response style across traits" 47 (47): 178-189, 2013

37 Bachman, J. G., "Black–White differences in self-esteem : Are they affected by response styles" 90 (90): 624-639, 1984

38 Leventhal, B. C., "Bayesian analysis of Multidimensional Item Response Theory models : A discussion and illustration of three response style models" 16 (16): 114-128, 2018

39 Cheung, G. W., "Assessing extreme and acquiescence response sets in cross-cultural research using structural equations modeling" 31 (31): 187-212, 2000

40 Sclove, S. L., "Application of model-selection criteria to some problems in multivariate analysis" 52 (52): 333-343, 1987

41 Bolt, D. M., "Addressing score bias and differential item functioning due to individual differences in response style" 33 (33): 335-352, 2009

42 Thissen-Roe, A., "A two-decision model for responses to likert-type items" 38 (38): 522-547, 2013

43 Jeon, M., "A generalized item response tree model for psychological assessments" 48 (48): 1070-1085, 2016

44 Falk, C. F., "A flexible full-information approach to the modeling of response styles" 21 (21): 328-347, 2016

45 Wetzel, E., "A Simulation Study on Methods of Correcting for the Effects of Extreme Response Style" 76 (76): 304-324, 2016

46 Akaike, H., "A New Look at the Statistical Model Identification" 19 (19): 716-723, 1974

47 Muraki, E., "A Generalized Partial Credit Model : Application of an EM Algorithm" 16 (16): 159-176, 1992

48 길혜지, "2017 KEDI 학생역량 조사 연구" 한국교육개발원 2017