Comparison of Two Essay Writing Tasks with Automatically Extracted Measures

다국어 초록 (Multilingual Abstract)

The study aims at comparing the effectiveness of lexical and syntactic features of L2 writing samples among three different proficiency levels (low, medium, and high), and between two writing tasks (YELC vs. TOEFL Essay). Specifically, the paper seeks to find out the most effective features in discriminating levels of L2 writing proficiency between lexical complexity measures and syntactic ones. It also tries to examine the use of linguistic features that differ across different L2 writing tasks. A subset of the TOEFL11 corpus, which consists of 11,000 essay samples, was analyzed and results were compared with the results reported on the YELC corpus. The results obtained from automatically extracted lexical and syntactic measures indicate that although members belonging to lexical and syntactic complexity measures can be used in discriminating different proficiency levels reliably, lexical complexity measures, in general, have larger effect sizes than syntactic complexity measures. The paper contributes to the research endeavor that uses computational tools in assessing L2 writing samples with different proficiency levels.

참고문헌 (Reference)

1 이석재, "한국 예비 대학생의 영어 사용 특성 파악을 위한 대규모 공개 영어 학습자 코퍼스 구축 및 분석" 한국콘텐츠학회 14 (14): 1019-1029, 2014

2 최재웅, "대학생 영어 학습자 작문 코퍼스에 대한 주제별 분류: 계층적 군집화" 언어정보연구소 (17) : 93-115, 2013

3 심보경, "대학생 글쓰기·말하기 능력의 상관성" 한국중원언어학회 (31) : 115-133, 2014

4 Rhee, S. C., "Yonsei English Learner Corpus (YELC)" 2012

5 Leech, G., "Word frequencies in written and spoken English: based on the British National Corpus" Longman 2001

6 Harley, B., "Verb lexis in the written compositions of young L2 learners" 11 : 415-440, 1989

7 Reppen, R., "Variation in Elementary Student Language: A Multidimensional Perspective" Northern Arizona University 1994

8 Grant, L., "Using computer-tagged linguistic features to describe L2 writing differences" 9 (9): 123-145, 2000

9 Gail M., "Using Effect Size—or Why the P Value Is Not Enough" 4 (4): 279-282, 2012

10 Levy, R., "Tregex and Tsurgeon: tools for querying and manipulating tree data structure" 2006

11 Silva, T., "Toward an understanding of the distinct nature of L2 writing: The ESL research and its implications" 27 : 657-675, 1993

12 Lu, X, "The relationship of lexical richness to the quality of ESL speakers’ oral narratives" 96 (96): 190-208, 2012

13 Laufer, B., "The lexical profile of second language writing: Does it change over time?" 25 : 21-33, 1994

14 Ferris, D., "The design of an automatic analysis program for L2 text research:Necessity and feasibility" 2 : 119-129, 1993

15 Blanchard, D., "TOEFL11:A corpus of non-native English" Educational Testing Service 2013

16 Ortega, L., "Syntactic complexity measures and their relationship to L2proficiency: A research synthesis of college-level L2 writing" 24 (24): 492-518, 2003

17 Biber, D., "Should we use characteristics of conversation to measure grammatical complexity in L2 writing development?" 45 : 5-35, 2011

18 Wolfe-Quintero, K., "Second language development in writing: Measures of fluency, accuracy, and complexity" University of Hawai’i, Second Language Teaching and Curriculum Center 1998

19 Hyland, K., "Second Language Writing" Cambridge University Press 2003

20 Miller, J. F., "Research in child language disorders: A decade of progress" Pro-Ed 211-220, 1991

21 김지영, "Predicting L2 Writing Proficiency Using Linguistic Complexity Measures: A Corpus-Based Study" 한국영어교육학회 69 (69): 27-51, 2014

22 Cooper, T, "Measuring written syntactic patterns of second language learners of German" 69 (69): 176-183, 1976

23 McNamara, D. S., "Linguistic features of writing quality" 27 (27): 57-86, 2010

24 Linnarud, M., "Lexis in composition: A performance analysis of Swedish learners’written English" Liber Fölag Malmo 1986

25 Hyltenstam, K., "Lexical characteristics of near-native second-language learners of Swedish" 9 : 67-84, 1988

26 박수경, "Lexical Analysis of Korean University Students’ Narrative and Argumentative Essays" 한국영어교육학회 68 (68): 131-157, 2013

27 Toutanova, K., "Feature-rich part-of-speech tagging with a cyclic dependency network" The Association for Computational Linguistics 252-259, 2003

28 Jarvis, S., "Exploring multiple profiles of highly rated learner compositions" 12 : 377-403, 2003

29 Richardson, J., "Eta squared and partial eta squared as measures of effect size in educational research" 6 (6): 135-147, 2011

30 Hunt, K. W., "Do sentences in the second language grow like those in the first?" 4 (4): 195-202, 1970

31 Field, A. P., "Discovering statistics using R" Sage 2012

32 Chaudron, C., "Discourse markedness and structural markedness:The acquisition of English noun phrases" 12 : 43-64, 1990

33 Klee, T, "Developmental and diagnostic characteristics of quantitative measures of children’s language production" 12 : 28-41, 1992

34 Crossley, S., "Computational assessment of lexical differences in L1 and L2 writing" 18 : 119-135, 2009

35 Templin, M., "Certain language skills in children: Their development and interrelationships" The University of Minnesota Press 1957

36 Ai, H., "Automatic treatment and analysis of learner corpus data" John Benjamins 249-264, 2013

37 Read, J., "Assessing vocabulary" Oxford University Press 2000

38 Minnen, G., "Applied morphological processing of English" 7 (7): 207-223, 2001

39 Klein, D., "A generative constituent-context model for improved grammar induction" 128-135, 2002

40 Green, C., "A computational investigation of cohesion and lexical network density in L2 writing" 5 (5): 58-69, 2012

41 권영환, "A Qualitative Study of Writing and Revision for ESL Writers" 한국중원언어학회 (14) : 1-18, 2009

42 Lu, X., "A Corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writer’s language development" 45 (45): 36-62, 2011

43 박선영, "A Corpus-Based Study of Syntactic Complexity Measures as Developmental Indices of College-Level L2 Learners’ Proficiency in Writing" 한국응용언어학회 28 (28): 139-160, 2012