ChatGPT의 문법 및 재구성 피드백이 EFL 대학생들의 글쓰기 성과에 미치는 영향 탐구 = Exploring the Impact of ChatGPT’s Grammar and Reformulation Feedback on the Writing Performance of EFL College Students

다국어 초록 (Multilingual Abstract)

This dissertation investigates the impact of ChatGPT’s grammar and reformulation feedback on the writing performance of EFL (English as a Foreign Language) college students, with particular emphasis on the Complexity, Accuracy, and Fluency (CAF) framework and IELTS General Training writing tasks. While teacher feedback has traditionally been recognized as the most valuable source of input in writing instruction, its effectiveness is often constrained by time, workload, and large class sizes. Automated Writing Evaluation (AWE) systems have attempted to address these challenges by offering immediate and scalable feedback, but they typically focus on surface-level issues and neglect higher-order dimensions of writing such as coherence, cohesion, and lexical diversity. Against this background, the emergence of generative AI tools like ChatGPT introduces new opportunities for providing more authentic, context-sensitive, and reformulative feedback that goes beyond error correction.
Drawing upon Sociocultural Theory (SCT), the Noticing Hypothesis, and the Output Hypothesis, this study situates ChatGPT as a potential “more knowledgeable other” that can scaffold learner development through iterative cycles of output, noticing, and reformulation. Reformulation feedback, in which learners’ original texts are rewritten into more target-like expressions while preserving meaning, has long been considered a powerful pedagogical technique. ChatGPT operationalizes this technique at scale, offering instant reformulations that highlight linguistic gaps and expose learners to authentic usage patterns.
The research employed a quasi-experimental design involving pre-tests, treatments, and post-tests with college-level EFL participants. Instruments included ChatGPT as a feedback tool, IELTS General Training Writing Task 1 (letter writing) and Task 2 (essay writing), and a questionnaire examining learner perceptions. Quantitative analysis was conducted using CAF metrics—such as error-free T-units, clauses per T-unit, dependent clauses per clause, and words per error-free T-unit—along with IELTS band descriptors across four criteria: Task Achievement/Response, Coherence and Cohesion, Lexical Resource, and Grammatical Range and Accuracy. Reliability was ensured through inter-rater scoring and Cronbach’s alpha coefficients, while ethical considerations addressed responsible use of AI in academic contexts.
The findings reveal that ChatGPT feedback led to significant improvements in accuracy and fluency, with learners producing more error-free sentences, sustaining longer stretches of writing, and achieving higher overall IELTS band scores. However, syntactic complexity did not show statistically significant gains, indicating that learners continued to rely on familiar structural patterns rather than experimenting with more advanced clause constructions. Questionnaire data further demonstrated positive student attitudes toward ChatGPT feedback, highlighting perceptions of effectiveness, user-friendliness, and motivational value, though concerns were raised about over-reliance and the need for critical engagement with AI-generated suggestions.
The discussion emphasizes the pedagogical implications of integrating ChatGPT into writing instruction. While not intended to replace teacher feedback, ChatGPT can serve as a supplementary tool that alleviates teacher workload, supports learner autonomy, and democratizes access to high-quality feedback. At the same time, challenges such as reliability, ethical concerns, and the risk of learner passivity necessitate careful pedagogical framing. Teachers are encouraged to foster critical digital literacy, guiding students to reflect on reformulations rather than adopting them uncritically.
In conclusion, this dissertation demonstrates that ChatGPT’s grammar and reformulation feedback can substantially enhance accuracy and fluency in EFL writing, while exerting only a limited influence on complexity. At a time when generative AI is rapidly transforming education, this research underscores both the potential and the limitations of leveraging AI as a pedagogical scaffold and a catalyst for rethinking feedback practices in second language writing instruction.

목차 (Table of Contents)

• Chapter 1. Introduction 1
• 1.1 Research Background 1
• 1.2 Statement of the Problem 5
• 1.3 Purpose of the Study 8
• 1.4 Significance of the Study 9
• 1.5 Research Questions 11
• 1.6 Operational Definition of Key Terms 12
• 1.6.1 Artificial Intelligence in Education (AIEd) 12
• 1.6.2 Automated Writing Evaluation (AWE) 12
• 1.6.3 ChatGPT and Generative AI Feedback 13
• 1.6.4 Reformulation Feedback 14
• 1.6.5 Complexity, Accuracy, and Fluency (CAF) 14
• 1.6.6 GT IELTS Writing Assessment Criteria 16
• 1.7 Organization of the Dissertation 18
• Chapter 2. Literature Review and Theoretical Framework 22
• 2.1 Theories of Language Learning and Writing Development 22
• 2.1.1 Sociocultural Theory (SCT) 23
• 2.1.2 Noticing Hypothesis 25
• 2.1.3 Output Hypothesis 26
• 2.2 Automated Writing Evlauation (AWE) Systems 28
• 2.2.1 From Grammar Checkers to AWE Tools 29
• 2.2.2 Limitations of Traditional AWE 31
• 2.3 Generative AI and ChatGPT in Language Education 33
• 2.3.1 ChatGPT as an Automated Writing Feedback Tool 34
• 2.3.2 Advantages for L2 Writing Development 36
• 2.3.3 Concerns and Challenges (Accuracy, Ethics, Reliability) 38
• 2.4 Reformulation Feedback in Writing Instruction 40
• 2.4.1 Concept and Pedagogical Role of Reformulation 41
• 2.4.2 Empirical Studies on Reformulation in L2 Writing 42
• 2.4.3 Reformulation through AI Tools 44
• 2.5 Complexity, Accuracy, and Fluency (CAF) Framework 46
• 2.5.1 CAF as Indicators of L2 Writing Development 50
• 2.5.2 CAF Metrics in Writing Assessment 48
• 2.6 GT IELTS Writing as a Benchmark 52
• 2.6.1 GT IELTS Writing Task 1 (Letter Writing) 56
• 2.6.2 GT IELTS Writing Task 2 54
• 2.6.3 GT IELTS Band Descriptors 56
• Chapter 3. Research Methodology 60
• 3.1 Research Design 60
• 3.2 Research Context and Participants 63
• 3.3 Instruments 65
• 3.3.1 ChatGPT as Feedback Tool 65
• 3.3.1 GT IELTS Writing Tasks 68
• 3.3.1 Questionnaires 69
• 3.4 Data Collection Procedures 71
• 3.5 Data Analysis 74
• 3.6 Reliability and Validity 77
• 3.7 Ethical Considerations 80
• Chapter 4. Results 83
• 4.1 GT IELTS Writing Task 1 Results 83
• 4.1.1 CAF Analysis for Task 1 83
• 4.1.1.1 Accuracy Indicators (EFT/T, E/T) 84
• 4.1.1.2 Complexity Indicators (C/T, DC/C) 85
• 4.1.1.3 Fluency Indicators (W/T, W/C, W/EFT) 87
• 4.1.2 IELTS Band Score Analysis for Task 1 88
• 4.1.2.1 Task Achievement 89
• 4.1.2.2 Coherence & Cohesion 90
• 4.1.2.3 Lexical Resource 92
• 4.1.2.4 Grammatical Range & Accuracy 93
• 4.1.2.5 Overall Band Score 1 94
• 4.2 GT IELTS Writing Task 2 Results 96
• 4.2.1 CAF Analysis for Task 2 96
• 4.2.1.1 Accuracy Indicators (EFT/T, E/T) 97
• 4.2.1.2 Complexity Indicators (C/T, DC/C) 98
• 4.2.1.3 Fluency Indicators (W/T, W/C, W/EFT) 99
• 4.2.2 IELTS Band Score Analysis for Task 2 101
• 4.2.2.1 Task Response 101
• 4.2.2.2 Coherence & Cohesion 103
• 4.2.2.3 Lexical Resource 104
• 4.2.2.4 Grammatical Range & Accuracy 105
• 4.2.2.5 Overall Band Score 106
• 4.3 Students' Perceptions from Questionnaire 110
• 4.3.1 Effectiveness of ChatGPT Feedback 111
• 4.3.2 User-friendliness 112
• 4.3.3 Confidence 113
• 4.3.4 Motivation 114
• 4.3.5 Comparison with Human Feedback 115
• Chapter 5. Discussion 118
• 5.1 Impact of ChatGPT Feedback on CAF 118
• 5.2 Impact on IELTS Writing Task Performance 122
• 5.3 Students' Attitudes and Perceptions Toward AI Feedback 126
• 5.4 Pedagogical Implications for L2 Writing 130
• 5.5 Challenges and Ethical Concerns 134
• Chapter 6. Conclusion 139
• 6.1 Summary of Key Findings 139
• 6.2 Theoretical Contributions 143
• 6.3 Pedagogical Recommendations 147
• 6.4 Limitations of the Study 151
• 6.5 Suggestions for Future Research 155
• References 160