This study aims to develop and implement an assessment tool to measure Korean pragmatic competence and to analyze the result with the limitations of the existing research on assessing Korean learners’ L2 pragmatic competence in mind. This study foll...
This study aims to develop and implement an assessment tool to measure Korean pragmatic competence and to analyze the result with the limitations of the existing research on assessing Korean learners’ L2 pragmatic competence in mind. This study followed the evidence-centered assessment design approach suggested by Mislevy et al. (2003) to develop the tool through a reliable and empirical process and conducted reliability test after assessment.
The assessment developed in this study consists of the following three sections: ‘Conversational Implicature Section’ evaluates the subject’s ability to understand implied meaning; ‘Routines Section’ measures the subject’s knowledge of routines; and ‘Judgement of Appropriateness Section’ determines the subject’s judgement ability of the appropriateness. ‘Conversational Implicature Section’ contains a total of 10 items including 5 multiple choice items and 5 multiple choice DCTs. ‘Routines Section’ includes 10 multiple choice DCTs and ‘Judgement of Appropriateness Section’ involves 16 five-scale appropriateness judgment tests. During the item development, survey of the native speakers was conducted and the result was reviewed in addition to preliminary test of Korean learners. Final test was taken by 109 Korean learners with 32 native speakers as a comparison group.
Analysis of the assessment result was conducted in two parts; internal analysis and external analysis. Internal analysis examines the characteristics of items through reliability test to confirm internal consistency of the assessment and the response of the subject by item. On the other hand, external analysis is made to see how Korean learners’ L2 pragmatic competence and the variables of Korean learners are related by statistically analyzing the assessment result of groups by subject variables. Conclusion from the analysis of the assessment result can be summarized as follows: First, every subsection of the assessment showed a high Cronbach’s α, which represents an internal consistency. This implies that the assessment was conducted in a empirical and systematic way. Second, Korean learners had a difficulty understanding particularized conversational implicature in ‘Conversational Implicature Section’ and a difficulty distinguishing routines with similar functions in ‘Routines Section.’ Also, in ‘Judgement of Appropriateness Section,’ they find it difficult to recognize inappropriate utterances in particular. Third, it was found that Korean pragmatic competence was positively influenced by both language proficiency and the period of residence of the subjects, but language proficiency has a stronger correlation with pragmatic competence than the period of stay does. Fourth, Korean learners’ L2 pragmatic competence develops disproportionately in each subsection according to language proficiency. Fifth, overseas Koreans who have been exposed to Korean from the childhood by parents showed similar L2 pragmatic competence level as that of native speakers in all of the subsections even if some of them has lower language proficiency.
To be brief, Korean L2 pragmatic education needs to be provided systematically according to the level of proficiency of each learner. Statistics revealed sociopragmatic competence, above all, should be taught in a more explicit way. In particular, the previous approach to honorific education should be reconsidered as it is closely associated with the appropriateness of utterance.
This study has developed an assessment to measure Korean learners' L2 pragmatic competence through a empirical process and analyzed the results internally and externally. It is significant that this study has been able to present a viable option—which is reliable and practical—to assess Korean learners’ L2 pragmatic competence and to set a direction for Korean L2 pragmatic education.