2008학년도 이후 대학수학능력시험 개선 방안에 대한 비판적 검토

남명호(Myung-ho Nam) 한국교육평가학회 2005 교육평가연구 Vol.18 No.2

The Post 2008 CSAT reform by MOEHRD can be summarized as follows. First, CSAT score report provides only 9 K-stanines. Standardized scores and percentiles won't be given to examinees. Second, CSAT will put much importance on close connection with high school curriculum and assessment of reasoning skills. Third, item pools for CSAT will be constructed as a way of writing items. Administrating a couple of CSATs or two-day CSAT is under consideration. It seems appropriate to keep 9 K-stanines as it is. However, Criterion-referenced method must be taken into account because norm-referenced standard setting method is problematic because actual CSAT score distribution is not normal and there exists a gap between standard and actual grade ratio. Supply type items must be introduced for close connection with high school curriculum and measurement of critical reasoning skills. To do this, it is necessary to develop automated scoring program. Providing multiple chance to take CSAT cannot be the only reason to administer multiple CSATs. The nature and scope of CSAT, standard setting method, and item format must be considered in a comprehensive way. 교육인적자원부에서 발표한 2008학년도 이후의 대학입학제도 개선방안 중 대학수학능력시험 제도 개선 부분의 주요 내용은 첫째, 수능 성적은 표준점수와 백분위를 제외하고 현행과 같은 9등급만을 제공하고 둘째, 고등학교 교육과정과의 연계를 강화하되 사고력 측정을 강화하며 셋째, 2010학년도 시험부터 문제은행식 출제 방식으로 전환하며 연2회 시험 실시 및 이틀간 나누어 시행하는 방안을 검토하는 것으로 요약할 수 있다. 등급 수를 현재와 같이 9등급을 유지한 것은 비교적 적절하나 규준참조식 등급 구분 방법은실제 수능 자료의 비정상적인 분포, 기준 등급과 실제 등급의 격차 등의 문제가 있어 준거참조 방식을 적극 도입해야 한다. 수능 시험과 교육과정의 연계를 더욱 강화하고 고등 사고력을 측정하기 위해서는 서답형 문항의 도입이 필수적이다. 이를 위해 채점 자동화 프로그램을 개발할 필요가 있다. 수능의 복수 시행 여부는 단순히 수험생에게 복수의 시험 기회를 제공한다는 것으로 판단해서는 안되며, 시험의 성격, 시험 범위, 등급 부여 방식, 문항 유형 등을 종합적으로 고려하여야 한다.

초등교육에서 수행평가의 활용 : 과제와 전망

남명호(Myung Ho Nam) 한국초등교육학회 1996 초등교육연구 Vol.10 No.1

Because of its characteristic, the elementary school education, compared with the middle and high school education, needs more criterion-referenced assessment which is concerned with diagnosis of learning ablity, process of learning, and mastery of instruction objectives than norm-referenced assessment which gives knowledge about the children`s rank or relative place in the group. Such characteristics require introducing and applying the performance assessment. Writing, oral examination, science experiment, computer simulation and portfolio are some examples of performance assessment which can be used easily in elementary school. But the performance assessment has some difficulties to practice. First, it needs higher cost and is hard to practice on a large scale. Also, It has time and place limitations, and has much possibility of rater`s error. Finally, it is behind the paper-pencil test in internal consistency. But these weak-points are possible to overcome by the methods, such as use of scientific sampling method, developement of assistant evaluation tools, and rater training.

다문화학생 한국어 능력 진단 검사의 개발 및 타당화

조남민(Nam Min Cho),남명호(Myung Ho Nam) 언어과학회 2013 언어과학연구 Vol.0 No.67

This research has been done as a part of the research on “Korean language skill diagnostic test for multicultural students” in 2012, and is a summary of principles and practice of writing skill test development, and actual result from preliminary test. This research was initiated because there was an emerging needs for diagnostic tool to assess Korean language skills of young learners from multicultural families and those who immigrated recently. Also, this research is a follow up from KSL which aimed to develop Korean language skills of children from multicultural families. Through this research, Test Construct and Analytic Scale of writing test, and scaling methodology were developed. Test Construct and Analytic Scale include (1) Language Use Ability, (2) Content Organization Ability, (3) Metacognitive strategy Ability, and (4) Sociolinguistic Ability, and these were developed through study of advanced research and relevant theories. Pilot test from 202 samples of students (elementary, middle, and high school) showed stable coefficient across different criteria including item discrimination, Cronbach α reliability and inter-scorer reliability in grader confidence based on validity analysis against classical test theory.

대규모시험의 행렬표집검사설계에 관한 고찰

남현우(Nam Hyun-Woo),남명호(Nam Myung-Ho),양길석(Yang Kil-Suk),김진하(Kim Jin-Ha) 충남대학교 사회과학연구소 2009 사회과학연구 Vol.20 No.4

고부담시험에서 문항노출위험과 동등화편차를 최소화 할 수 있는 행렬표집설계 방안

남현우(Hyun-Woo Nam),남명호(Myung-Ho Nam),양길석(Kil-Seok Yang),김진하(Jin-Ha Kim) 한국교육평가학회 2009 교육평가연구 Vol.22 No.3

대학수학능력시험과 같은 고부담 시험에서 가교 역할을 하는 문항의 노출 위험을 줄이면서 동등화 오차를 적게 하는 방안으로서 행렬표집설계를 제안하고 이의 적용 가능성을 밝히고자 본 연구를 수행했다.단순 가교검사설계로 동등화 자료를 수집한 국가수준 학업성취도평가의 2006년과 2007년 고1국어,수학,영어 검사 자료를 다양한 구획 수와 피험자 수의 행렬표집설계로 변형하여 동등화 편차를 계산하였다.단순 가교검사설계 자료의 동등화를 기준으로 하고,행렬표집설계의 구획수를 늘리면서 구획 당 피험자 수와 문항 수를 줄였을 때 동등화 편차에 변화가 있는지를 검토했다.국어와 수학에는 행렬표집설계를 적용할 가능성이 있지만,영어의 경우에는 특성곡선전환 방법이 조심스럽게 사용될 때에만 적용될 수 있다는 결론에 도달했다. This research was performed to suggest a matrix-sampled design, for the purpose of reducing item disclosure risks of anchor items and lowering test takers' burden in large scale high-stakes assessment. Korean, Math, and English test for the students of 10th grade in the year of 2006 and 2007, Korea National Assessment of Educational Achievement were used as an empirical data of this research. We modified item responses of simple anchor test design in to those of matrix-sampled design with varying number of blocks and examinee. RMSD(Root Mean Squared Difference) was calculated to test whether the equating functions of simple anchor test design and matrix-sampled design were different from as the number of blocks was increased but the number of examinee and items of each blocks was decreased. We concluded that the matrix-sampled design could be applied in the Korean and Math test of National Assessment of Educational Achievement, but only the CCT(Characteristic Curve Trans formation) method in the matrix-sampled design could be cautiously used in the English test

온라인 시스템을 활용한 대규모 서답형 평가의 채점 일관성

송미영(Mi-Young Song),김수진(Kim Soojin),김희경(HeeKyoung Kim),남명호(Myung-Ho Nam) 한국교육평가학회 2009 교육평가연구 Vol.22 No.3

대규모 학업성취도 평가의 서답형 문항에 대한 응시자들의 응답은 합숙형 수기 채점 방식으로 채점되어 왔으나,이 방식은 채점자 섭외,채점 장소 확보 등 여러 가지 현실적인 어려움이 있었다.이러한 어려움을 해소하기 위하여 기존 방식의 대안으로 온라인 시스템을 활용하는 방식을 고려할 수 있다.본 연구는 온라인 채점 시스템을 설계․구현하여 실제 적용 가능성을 검증하였다.채점자간 신뢰도를 산출한 결과, 0.810이상의 높은 상관으로 보여 채점자 신뢰도는 상당히 높았다.채점자간 점수 차이가 일정 기준을 초과할 경우 채점자들에 의한 재채점이 진행되었는데,1,2차 시기의 답안을 통합하여 구한 채점의 신뢰도는 1차 시기답안만을 대상으로 한 경우보다 모든 교과에서 증가하였다.이러한 결과로부터 온라인 채점시스템을 활용하면 채점 과정을 실시간으로 인터넷을 통해 모니터할 수 있기 때문에 채점 오류를 최소화하고 채점의 공정성과 신뢰도를 현격히 제고할 수 있으리라 기대된다. National Assessment of Educational Achievement(NAEA) in Korea, a large-scale national assessment has scored students' constructed-response items through in-hands by gathering in one secure dplace. This method had a several difficulties such as recruiting raters, booking a large enough places to keep all the raters and test book lets. This study examined the applicability of on-line scoring system for constructed-response items in a large-scale assessment. The performance samples of about 10,000 nine-graders for five subject areas such as Korean, Social Science, Mathematics, Science, English in NAEA were used. Every construct responses were scored by at least two raters. The responses were re-scored when differences between two raters' scores were greater than a certain criterion. The inter-rater reliabilities were above the 0.81, which is a very high. The result of this study shows that the correlations and agreements between scores by two raters were very high, and re-scoring improved the rater consistency evenmore.This study proved that on-line scoring system would be a stable scoring method of constructed responses for the large-scale assessment.