영어 말하기 평가에서 나타난 한국인 교사 채점자의 채점 경향성 분석

신동광 중앙대학교 외국학연구소 2017 외국학연구 Vol.- No.42

The present study investigated the biases of Korean teachers who were non-native English speakers in rating test takers’ voice information, such as native-like pronunciation. To examine the potential biases, two different scoring methods were adopted. One was a normal scoring method where five qualified Korean teachers assessed 100 test takers’ responses while listening to their voice files. For the other method, the original voice files were transcribed into text files. After a one-year time interval, the same raters reassessed the text files without the test takers’ voice information. The results for the three common rating domains of “task completion,” “discourse organization,” and “language use” significantly different between the two scoring methods. Also, “pronunciation” and “fluency” were more highly correlated with the three domains in the first rating than in the second rating. That is, even when the raters were asked to assess the spoken language of test takers by analytic scoring, ratings for “pronunciation” and “fluency” seemed to be strongly associated with the scoring domains when taking into account the test takers’ voice information. The results were confirmed by interviews with the raters. Also, as expected, four of the five raters scored test takers more severely in the text-based rating. Pedagogical implications are suggested to overcome biases that non-native speaking teacher raters may have in judging oral English performance.

2009 영어과 교육과정 기본 어휘 지침이 교과서 개발에 미친 영향

신동광 중앙대학교 외국학연구소 2015 외국학연구 Vol.- No.34

The present study attempted to investigate how vocabulary regulations stipulated in the 2009 National Curriculum of English have affected the development of English textbooks. That is, by comparing both vocabulary use and readability scores of the textbooks produced by the 2007 National Curriculum of English and those of 2009, this study tried to examine how the more recent vocabulary regulation has contributed to reducing Korean students’ learning burden. For this, a total of 24 textbooks, that is, 12 matching pairs of high school English textbooks from the 2007 and 2009 National Curriculum of English were analyzed. The results show that the vocabulary regulation of the 2009 National Curriculum of English has significantly contributed to reducing the total number of word families. However, the readability indices of the textbooks developed after the 2009 National Curriculum showed that the 2009 textbooks would be more challenging to read. The results also provide implications for the recent 2015 National Curriculum of English, which has the same educational rationale as that of 2009; the learning burden for vocabulary may be reduced, but controlling text difficulty is expected to be difficult. The study ends by providing some suggestions to complement the limitation of the 2015 National Curriculum of English.

영어교육에서 AI 활용의 한계는 어디쯤일까?: 자동 지문 생성 및 자동 문항 생성의 활용 가능성

신동광 한국영어교과교육학회 2022 영어교과교육 Vol.21 No.3

This study explores the feasibility of using Automated Text Generation (ATG) and Automated Item Generation (AIG) in the field of English education. To this end, 60 pre-service teachers directly tried the two programs and participated in improving the quality of the test item through post-editing. The results suggest the following: First, among the participants who corrected the test items of the participants in this experiment, the pre-service teachers modified their test item 391 times in total. Text correction accounted for 75.19% of the modifications, and option correction accounted for 24.8%. In the program user satisfaction survey, 86% of the ATG program respondents showed a positive attitude. In addition, the fact that none of the respondents selected 'very dissatisfied' suggests that there was a high possibility of using ATG at school. On the other hand, the AIG program respondents demonstrated a relatively low level of satisfaction. Overall, 43% responded positively but 33% of the total respondents took a negative stance. Improvements for both programs collected through open-ended questions were also presented. It is, however, important to note that evaluation of whether or not a post-editing attempt was appropriate was beyond the scope of this study.

AI 도구를 활용한 중등교사 영어 출제 연수 사례 연구: ChatGPT를 중심으로

신동광 서울대학교 언어교육원 2023 語學硏究 Vol.59 No.1

This study analyzed the use of various AI tools (e.g., ChatGPT) by ten teachers who participated in a secondary teacher training program hosted by University A in January 2023. The training program was conducted over three days, and after learning how to use AI tools, the teachers were asked to apply the AI tools in the actual practice of test item development, particularly for reading test items in the College Scholastic Ability Test (CSAT). The study divided the test item development process into five stages: 1) passage creation; 2) test item creation; 3) test item selection; 4) vocabulary level control; and 5) refining details such as styling, choice length, and so on. At each stage, the cases showed that Al tools could significantly contribute to improving the efficiency of developing test items. The teachers also responded very positively to the training program, and the conclusion section discussed future challenges and further research.

성인 영어 코퍼스와 아동 영어 코퍼스의 어휘 사용 비교·분석

신동광 중앙대학교 외국학연구소 2016 외국학연구 Vol.- No.38

The present study attempts to compare vocabulary use in adult and children’s English corpora. The ultimate goal of the present study is to investigate what sort of word list is the most appropriate source for children's vocabulary acquisition. To examine this controversial issue, the four word lists BNC-COCA 3000, Kids 2500, CHILDES Parental Corpus Word List of English (CHILDES-P 3000), and the Korean Basic English Word List (KBEWL 3000) . They are lists of about 3,000 words that represent an adult English corpus-based word list, a children’s English corpus-based word list, a child-directed corpus (parental corpus)-based word list, and a children’s English expert consultation-based word list. The results show that each word list has about 65% of the total words in common. In both the children’s literature in English and the adult English corpora , the adult English corpus-based word list of BNC-COCA has the highest text coverage, of 92% and 86%, respectively. Nevertheless, it was also found that there is a big gap in vocabulary use between the adult and children’s corpora. In conclusion, it was suggested that a parental corpus-based word list could serve as a finely-tuned input for children and as an alternative to the existing adult English corpus-based word lists.

코퍼스 분석 프로그램을 활용한 학생들의 영어 쓰기 오류 관리 및 피드백 제공 방안

신동광 강남대학교 인문과학연구소 2013 인문과학논집 Vol.25 No.-

This study proposes a new way to treat learners' grammatical errors in English writing. In particular, this study focused on the use of corpus-based skills for providing error feedback. For this purpose, three first-year college students participated in the study; two from an advanced and one from an intermediate level. They were all non- English majors. The students were required to write a short essay of about a page on a given topic. They were then asked to revise their essay and submit it. After their second draft, the students were introduced to a free online dictionary and were trained to use Concordance Writer v. 3.3, which provides concordance lines of the target word or expressions for the writer. The errors that the subjects made were given unique tags; for example, ‘PRE’ represented a preposition error. By analyzing the error-tagged drafts using the RANGE program, we are able count the number of errors among all of the errors or within a particular error type. We are able to find what kind of error was most frequent in a particular draft and what kind error had increased or decreased in drafts by a particular subject over the experiment period. In addition, when we used the concordance AntConc to analyze the drafts, we were able to search for all those contexts related to a particular error type.The analysis using the RANGE program showed that the students’ errors decreased over their rewriting processes. Interviews with the students revealed interesting insights. It was discovered that the intermediate student could not apply Concordance Writer v.3.3 in her writing because of her limited English proficiency. She was usually not able to identify her own errors; and, when she did, she was unable to understand the sample given in Concordance Writer v.3.3 in order to correct her writing. In most cases, she used the online dictionary to look up words. The advanced students, however, found both the concordance lines and the online dictionary useful. This was because they were proficient enough to recognize, check, and thus correct their errors using the two programs. For further research, a study is recommended using a larger number of subjects, generalizing the results of the present study. 본 연구에서는 실제 원어민 언어 자료인 코퍼스에서 추출한 예시문을 가리키는 ‘콘코던스’와 ‘온라인 사전’의 활용이 쓰기 오류의 수정에 어떠한 영향을 주는지를 중점적으로 살펴보며 코퍼스 분석 프로그램을 활용한 쓰기 오류 피드백 관리 방법을 제안하는 것을 연구의 목적으로 한다. 본 연구는 대표성 있는 표집을 통한 실증적인 통계 자료를 제시하기 보다는 소수의 실험 대상을 통하여 영어 쓰기 오류 지도 및 관리, 그리고 새로운 방식의 오류 피드백 방법에 대한 시사점을 이끌어 내는 데 초점을 두었다. 본 연구에 참여한 실험 대상은 서울 소재 A대학교의 1학년 학생 3명이다. 실험 대상 모두 비영어전공의 학생들로 교양영어 과목을 수강하고 있는 학생들이다. 3명 가운데 2명은 쓰기 능력에서 상위권으로 분류될 수 있는 학생들이며 나머지 1명의 학생은 중위권으로 볼 수 있는 학생이다. 3명 모두 장기간의 영어권 국가의 체류 경험은 없었다.본 연구에서는 실험 대상자로 하여금 동일한 주제에 대하여 A4 한 장 길이의 에세이를 작성하게 하였다. 이후 1주일 간격으로 두 차례 글을 수정하여 제출하게 하였다. 첫 번째 수정은 초안 작성과 마찬가지로 어떠한 참고자료나 피드백 없이 글을 수정하도록 요청하였고 두 번째 수정은 아래 그림과 같이 코퍼스 예시 샘플인 콘코던스와 온라인 사전의 참고가 가능한 Concord Writer v.3.3라는 프로그램을 활용하여 수정하게 하였다. 그 결과 기존 연구들과 마찬가지로 상위권 학생들에게서 콘코던스와 온라인 사전의 참고를 활용하여 에세이를 수정해 보는 것이 오류의 수를 줄이는데 보다 효과적이라는 점을 확인 할 수 있었다 본 연구에서는 코퍼스 프로그램을 통한 오류의 관리나 피드백이 보다 체계적인 교수학습의 방법을 소개함으로써 교육적인 시사점과 추후 연구에 대한 방향성을 제시하였다.

어휘학습을 위한 언어 입력의 제공원으로서 원어민, 비원어민 교사, 교재 비교 연구

신동광 한국외국어교육학회 2015 Foreign languages education Vol.22 No.4

The present study aims to compare teacher talk and the course book used as a source of vocabulary input. For the research purpose, part of Yenny Corpus (Kwon, 2013) was used to analyze the vocabulary content of the two input sources. Yenny Corpus consists of a total of 247,398 words collected from EFL classes of four native and five non-native university lecturers. In this study, however, the data of 1,416 words by the native teachers and 1,651 words by the non-native teachers with their course book scripts were used. To analyze the quantity and quality of the vocabulary use (e.g., level, diversity) of the two input sources, some representative word lists such as BNC-COCA 25,000 word family list topped on Range Program (Heatley & Nation, 2002) were applied. The results show that the teachers (including both native and non-native teachers) orally provide 8,090 word types while their course books introduce 4,713. The non-native teachers provide a little more vocabulary input compared with the native teachers’ talk. In addition, the level of the vocabulary provided by the non-native teachers seems more appropriate for their students’ vocabulary ability. Nevertheless, in order to generalize the findings further research is needed in different settings.

AI 시대의 언어교육, 이대로 괜찮을까?

신동광 국제한국언어문화학회 2021 국제한국언어문화학회 학술대회 Vol.2021 No.10

우리는 어떤 영어 어휘를 학습하고 있는가? 국내외 대표 영어 어휘목록 비교·분석

신동광 중앙대학교 외국학연구소 2014 외국학연구 Vol.- No.30

The present study has three goals. The first is to examine unique features originating from the development process (e.g. vocabulary selection criteria such as frequency, range and dispersion & corpora used for developing vocabulary lists) of ten representative English vocabulary lists which all contain around 3,000 words. Included among these are the Korea Basic English Word List and the NEAT 3000 developed in Korea. The second goal is to report strong and weak points which each vocabulary list has, and the third is to provide some implications of such analysis for developing a new vocabulary list in the future. With these purposes, this study attempts to analyze and compare specifications of the English vocabulary lists and their text coverage in a varied range of corpora of English. This study concludes by providing eight suggestions for the successful development of a newly updated English vocabulary list.

라쉬 모형을 이용한 연어 능숙도 평가 타당성 검증 연구

신동광,이용상 이화여자대학교 교과교육연구소 2019 교과교육학연구 Vol.23 No.5

Although a variety of collocation tests have been developed so far, all of these tests have some limitations in that they do not measure absolute collocation knowledge. The present study was designed to validate the hierarchy of the Collocation Size Test (CST) developed in Kwark, Shin, & Lee (2019) study to measure the size of the absolute collocation knowledge of 160 Korean elementary students. The CST consists of four levels and each level contains 10 items, representing the knowledge of 500 collocations. The validation procedure of the CST was performed, using the Rash model. In order to validate the CST items, the Wright Map was used to check whether their CST could measure the general collocation proficiency of the subject group and the quality of the items was verified through the item fit. In addition, in order to verify whether the items graded for the four levels of the CST could function hierarchically, the mean score and the 95% confidence interval of the 10 items of each level were analyzed. The findings showed that the hierarchical structure of the four levels is appropriately differentiated. 지금까지 다양한 연어 평가 검사지가 개발되었지만 이들 연구들 모두는 연어 평가 문항의 유형이나 특정 연어의 학습 정도 즉 연어 성취도 평가적 성격을 지니고 있을 뿐 절대적인 연어 지식을 측정하지는 못하는 한계점을 가지고 있다. 본 연구에서는 초등학생들의 절대적 연어 지식을 측정 도구 개발의 일환으로 Kwark, Shin과 Lee(2019)의 연구에서 제작된 등급별 연어 능숙도 검사지(Collocation Size Test, CST)의 수준별 위계를 중심으로 타당성을 검증하였다. 본 연구에서 타당화하는 CST는 모두 4개의 수준으로 구성되어 있으며 각 수준은 500개의 연어를 대표하는 10개 문항으로 제작되었다. CST 타당화는 라쉬 모형을 이용하여 실시하였다. 타당화를 위해 우선 Wright Map을 통해 CST가 피험자 집단의 전체 능력을 모두 측정하는지를 살펴보고, 문항적합도를 통해 문항의 양호도를 검증하였다. 다음으로 연어 능숙도의 4단계 수준별로 개발된 CST문항들이 위계적으로 기능하고 있는지를 검증하기 위해 수준별 10개 문항들의 평균 난도와 95% 신뢰구간을 분석하였으며, 마지막으로 의미인식과 형태인식 문항들 간에 4단계의 위계적 구조가 차별화 되는지 확인하였다.