Investigating the Rationality of Using Composite Scores for a Speaking Placement Assessment

다국어 초록 (Multilingual Abstract)

Kim, Hyun Jung. 2011. Investigating the rationality of using composite scores for a speaking placement assessment. Korean Journal of English Language and Linguistics 11-3, 465-495. In second language performance assessments, analytic scoring is often preferred over holistic scoring in evaluating learners' test performances due to its advantages (e.g., detailed information about examinee ability and raters' separate attention to different components of the test construct). However, a single composite score is often used to infer learners' language ability, even in cases where their test performances have been rated using an analytic scoring rubric. Using a composite score may result in information loss regarding learners' ability and may mislead test users (e.g., teachers) in their decision-making. Therefore, this study investigated the reliability of the analytic ratings as well as the composite scores of a speaking placement test used in a community English program. A total of 215 incoming students took a speaking placement test, which included six semi-direct, computer-delivered tasks (three independent and three integrated). Students' recorded performances were scored by two independent raters on five analytic rating scales. An analysis of the analytic ratings using multivariate generalizability theory found high dependability of the speaking ratings and supported the use of composite scores. The findings suggest that a composite score reflecting multicomponential speaking ability can be reasonably used for placement decisions without being affected by other sources of score variation (e.g., tasks and task types).