For a More Useful EFL Cognitive Diagnostic Assessment

Yeon-Sook Yi(이연숙) 언어과학회 2016 언어과학연구 Vol.0 No.76

Cognitive diagnostic assessment can provide fine-grained diagnostic feedback and detailed scores reporting to the test-takers. Applying this assessment method to an EFL grammar test, this study aims to see if smaller grain size of grammar attributes can yield meaningful results of assessment. The study also looks into what kind of detailed information on student performance can be provided by utilizing the method. A set of response data to 23 grammar questions, collected from 434 college freshmen, was analyzed using a cognitive diagnostic model. The results present a positive answer to the research question of the feasibility of using smaller and more interpretable of grammar skills. The paper concludes with limitations of the current study and suggestions for future research.

과정중심 EFL 글쓰기에서 동료 피드백 영역에 대한 수준별 학습자 인식 연구

이연숙(Yi, Yeon-Sook) 학습자중심교과교육학회 2019 학습자중심교과교육연구 Vol.19 No.23

최근 EFL 글쓰기 동료 피드백에 대한 활발한 연구에도 불구하고 학습자들이 어떤 피드백을 필요로 하고, 실제로 글수정에 반영하는 동료 피드백의 내용은 무엇인지에 대한 연구는 부족하다. 또한 이를 대학생 대상으로 수준별로 탐색한 연구는 더욱 찾아 보기 힘들다. 본 연구는 그런 의미에서 영어 능숙도가 비교적 높은 집단과 낮은 집단이 글의 구성, 내용, 문법, 어휘, 기술적 사항 등에 관한 피드백을 얼마나 필요로 하고 얼마나 글쓰기 수정에 반영하고 있는지를 탐색해 보고자, 두 개의 대학생 대집단을 대상으로 동료 피드백을 실시하고 이에 대한 이들의 인식을 조사하였다. 연구의 결과로는 첫째, 동료 피드백 중 가장 도움이 되었던 영역에 대해서 두 집단 공히 문법과 단어/표현을 1, 2 순위로 꼽았지만 영어 수준이 비교적 높은 집단은 그 다음으로 글의 기술적 사항을 꼽았고, 낮은 집단은 글의 구성이나 내용에 대한 피드백이 기술적 사항보다 더 도움이 되었다고 응답하였다. 또한 동료 피드백 중 어떤 요소를 글 수정에 많이 반영했는지에 대해서는, 문법과 단어/표현, 기술적 사항에 대한 반영 빈도는 두 집단이 큰 차이를 보이지 않았지만, 글의 구성과 내용 측면에서 낮은 집단은 높은 집단에 비 해 훨씬 더 높은 빈도를 보였다. 또한 자유응답 질문을 통해 동료 피드백의 강점과 약점에 대한 학습자들의 인식을 파악하고 이를 본 연구의 결과와 통합하여 동료 피드백 시행 시 도움이 될 수 있는 사항들을 결론에 제안하였다. Despite a plethora of research that has been recently conducted in the EFL writing peer feedback, there is a paucity of research regarding the areas of peer feedback learners need to receive and actually incorporate in their revision after receiving peer feedback. It is harder to find such research that studied different proficiency groups of adult learners or college students. In that context, the current study examined what areas of peer feedback two different proficiency groups of college students wanted the most out of such domains as organization, content, grammar, vocabulary, and mechanics. The survey also asked what areas of peer feedback they incorporated when rewriting their argumentative essay. The research findings included the following: First, grammar and vocabulary ranked the first and second in both proficiency groups when asked what types of peer feedback were the most helpful, while mechanics followed right after in the higher group whereas organization/content followed in the lower group. As for the question of what areas of feedback they incorporated the most, not much difference was observed in the frequency of grammar, vocabulary and mechanics in both groups but the lower group showed a higher frequency of incorporation in the areas of essay organization and content. Their perceptions were interpreted based on their responses to the open-ended survey questions about the effectiveness of peer feedback. Pedagogical implications and suggestions were presented in the discussion of the paper.

대학 영어 프로그램에 대한선행연구 분석과 앞으로의 연구방향

이연숙(Yeon Sook Yi) 언어과학회 2014 언어과학연구 Vol.0 No.70

This study examines a range of research studies that have been actively conducted for the purpose of improving college English programs in Korea. While reviewing the studies and searching for avenues for future research, it applies a systematic perspective of viewing the college English program as an integrated language curriculum. The present study finds that most college English programs need to align the components of the curriculum, particularly between curriculum objectives and assessment standards. It also suggests that objectives be made more clear and exit exams be developed that reflect the curriculum goals and proficiency standards. Practical issues of optimal class hours and class size are also discussed in light of improving the program.

인지 진단 평가 모델 비교 연구

이연숙(Yeon-Sook Yi) 한국교육평가학회 2013 교육평가연구 Vol.26 No.3

본 연구는 네 개의 인지진단 평가 모델을 데이타 분석결과를 토대로 비교하였는데, 하위모델을 표현할 수 있는 포괄적 구조의 일반 모델인 LCDM과 세 개의 하위모델인 C-RUM, DINA, DINO을 기능적인 면과 모델 적합도 지수면에서 비교 분석하였다. 분석 자료는 유도 모방 과제로 추출된 세 세트의 불어 말하기 데이터였고 세 개의 큐행렬이 각각의 데이터와 연결되어 분석되었다. 분석 결과로는, 문항 상관 평균제곱근 오차(RMSE)에 근거한 절대 적합도 지수와 정보기준(informationcriteria)에 근거한 상대 적합도 지수, 그리고 세 가지 측면에서 살펴본 모델의 기능적인 측면에서 LCDM과 C-RUM이 가장 적합하다고 판명되었다. 논문의 결론에서는 이러한 결과의 이유가 모델의 구조적인 특성과 연결되어 논의되었으며, 더불어 현 연구의 제한점과 이를 바탕으로 한 후속 연구에 대한 제안도 제시되었다. Pour cognitive diagnostic models were empirically compared under a unified general modeling framework (the log-linear cognitive diagnostic mood, LCDM) that can represent specific moods (DNA, DINO, and C-RUM) with proper constraints. The study used three sets of French speech data that were collected using elicited imitation tasks. In the analysis, three Q-matrices representing grammatical features of French were linked to the relevant sets of data. The four models were examined in terms of their functioning and model fit to the data. The overall estimation results of the four models were analyzed in three different aspects, which all supported the general model (LCDM) and the compensatory-RUM (C-RUM) behaved in a very similar way, yielding very congruent estimation results. This finding was quite consistent with the fact that C-RUM fit the most number of items in the two sets of the data. The relative mood fir indices of information criteria also confirmed this finding, indicating the general model and the C-RUM were the best fir to all three test data sets. In the conclusion, the probable reasons for these findings were discussed and suggestions for future research were made based on the limitations of the study.

혼합 연구 방법에 의한 온라인 영어뉴스 청취수업에 대한 효용성 연구

이연숙(Yi, Yeon-Sook) 학습자중심교과교육학회 2019 학습자중심교과교육연구 Vol.19 No.2

본 연구는 영어 뉴스 청취를 위한 여러 듣기 활동의 효과와 뉴스 청취 수업 전반에 대한 만족도를 알아보는 것을 목적으로 한다. 이를 위해 영어 뉴스 청취 수업을 수강한 사이버대학교 재학생 113명을 대상으로 설문 조사를 실시하여 그 결과를 분석하였다. 수업은 반복듣기와 함께 청취를 위한 발음 학습, 부분 받아쓰기 연습을 포함하였다. 또한 과제를 통해 뉴스 듣고 요약하기, 번역하기, 따라하기(쉐도잉) 활동을 하도록 유도되었다. 설문은 총 18문항으로, 듣기 활동의 효과를 묻는 문항, 뉴스 청취에 있어 어려운 부분과 분야를 묻는 문항, 수업이 영어 능력 향상에 도움이 되었는지를 묻는 문항, 시험 문항의 효용성을 묻는 문항, 수업에 대해 좋은점과 제안점을 자유응답으로 서술하는 문항들로 구성되었다. 수업 중 활동과 과제의 효과에 대한 응답으로는 발음설명과 연습, 쉐도잉 활동이 도움되었다는 응답이 많았는데, 특히 사이버대학에서 비중이 높은 30대 이상 학습자들의 쉐도잉에 대한 인식이 20대와 유의미한 차이를 보였다. 뉴스청취를 어렵게 하는 요인은 어휘와 표현, 뉴스 속도, 복잡한 구문, 발음, 배경지식 부족 순으로 확인되었고, 전체 수업의 효과로는 듣기와 읽기 능력 배양을 꼽았다. 자유 응답은 받아쓰기와 발음 연습과 같은 수업활동과 수업구성과 내용에 관한 피드백, 수업에 바라는 사항 등으로 정리할 수 있었는데, 수업에 대해서는 대체로 긍정적인 반응을 보여주었지만 직장을 가진 학생으로서 난이도가 높은 뉴스청취를 듣는 것에 대한 부담과 과제량의 많음을 토로하였다. 결론에는 분석결과가 시사하는 점과 연구의 제한점을 밝히고 이를 바탕으로 한 향후 연구를 제안한다. This study aims to examine students perception on an online news listening course and the effectiveness of the listening activities or tasks. To this end, a 18-item survey was administered to a total of 113 students enrolled at a cyber university in Seoul, Korea. The most salient point in their responses was that pronunciation practice and shadowing tasks helped their listening comprehension the most. Particularly, in terms of shadowing activity, the perception of learners over 30, who account for more proportion compared to offline colleges, was significantly different from that of 20’s. It was identified that the most difficult factors while listening to English news were vocabulary/expressions, speed of speech, complex sentences, ways of pronunciation (elision, liaison, linking, assimilation, reduction), and lack of background information. The skills they could improve the most were listening as well as reading, revealing that the course content was not sufficient to improve their productive skills. For the open ended questions, many students left positive comments about the effectiveness of dictation and shadowing tasks and the overall structure of the class. They also expressed their thoughts freely about the workload of the class. The article also discusses how to utilize the research findings in future class and the limitations of the study, suggesting avenues for further research

Examining Multiple-choice Items in High School English Tests-Linking to Assessment Literacy

이연숙 ( Yeon Sook Yi ) 한국현대영어영문학회 2015 현대영어영문학 Vol.59 No.3

In the research context with no prior studies that investigated the adequacy of multiple-choice (MC) items in high school English exams administered regularly each semester, the present study puts them under scrutiny. The MC items in 33 tests are first examined according to item format and frequency of item types. Findings about the way they were used in the tests are also presented. Moreover, based on the multiple-choice item writing guidelines that are frequently cited and utilized internationally, the study looks into the items to see how well the items comply with these guidelines along with distinctive examples under each category. The study further advances the discussion, linking the problem to promoting assessment literacy for in-service teachers. (Sang Myung University)

Perceptions on EFL Writing Peer Feedback by Two Proficiency Groups: Focusing on Personality and Comparing Giving and Receiving Feedback

Yeon-Sook Yi(이연숙) 언어과학회 2019 언어과학연구 Vol.0 No.89

Based on a lack of research about the impact of personality on peer feedback in EFL writing, this study investigated relevant research questions with two proficiency groups of college students. The study also focused on the differing effects when giving and receiving peer feedback. The findings included the following: First, lower proficiency seemed to play a role in giving higher satisfaction when receiving feedback. Second, motivation was significantly correlated with effectiveness in the Lower group. Third, participants strengthened their ability and confidence more when receiving feedback. Fourth, the effectiveness of extroverts was higher when receiving feedback, particularly in the Higher group. Lastly, personality affected the preference of peer type of Higher group. Based on these findings, pedagogical implications and suggestions for future research are discussed.

대학생 대상 TETE에 대한 다각적인 연구

이연숙(Yi Yeon-Sook) 학습자중심교과교육학회 2018 학습자중심교과교육연구 Vol.18 No.15

본 연구는 대학생 대집단을 대상으로 영어로 영어 가르치기(TETE) 수업의 효과와 L1의 필요성, 수업에 대한 불안감을 묻는 30개 문항으로 이루어진 설문조사를 실시하여 학습자의 인식을 조사하였다. 또 이들의 수능영어 등급 점수와 이들 요인들의 상관관계를 살펴보았다. 분석 결과, 이들은 TETE 에 대해 전반적으로 긍정적인 인식을 보였지만 수업의 효과는 세부기술면에서 차이가 있었다. 또한 L1의 필요성에 대해 서도 높은 응답을 보였으며 구체적인 언어기술면에서는 문법에서 가장 많이 한국어의 필요성을 느끼는 것으로 조사되었다. 불안감 수치도 평균적으로 중간 이상의 수준을 보였다. 이들의 수능 영어 수준과 여러 변인들의 상관관계는 거의 모두 통계적으로 유의미하게 산출되었다. 즉, 수능수준이 높을수록 TETE에 대한 인식과 효과에 대한 긍정적인 인식이 높았다. 또한 한국어 필요성과 불안감과는 반비례하는 상관관계를 보였다. 이런 분석 결과를 바탕으로 초중고등 학생 대상 연구보다 연구가 부족한 대학생 대상 TETE 수업의 운영에 대한 관심을 촉구하고, 일괄적이고 전면적인 시행이 아닌 학습자 수준과 목표 기술에 따른 단계적이고 유연한 TETE 시행을 제안한 다. This study aims to examine college students perception on the TETE course, its effectiveness, their perceived need for L1, and degree of anxiety in the TETE class. For this purpose, a 30-item survey was administered to a large group of college students after a semester of TETE class. Their responses showed that they had positive attitudes toward TETE but their perceived effectiveness was different across specific skills of speaking, listening, reading, writing, vocabulary and grammar. They revealed high level of need for using L1, especially when learning grammar. Their degree of anxiety was also shown to be considerably high. Correlation analyses were conducted between these variables and most correlations turned out statistically significant. One of the results was that the better their CSAT rank, the higher their perceived effectiveness of TETE was. However, the CSAT was reversely correlated with both L1 need and anxiety. Based on the findings, attention is called for to further research on TETE for college students and it is suggested that TETE be implemented in multiple phases considering the proficiency levels of learners and the target skills of English.

인지진단평가 영어 시험 적용에서 Q행렬 구축 방법과 후속 학습 효과 연구

이연숙(Yeon-Sook Yi) 한국영어평가학회 2023 영어평가 Vol.18 No.2

In cognitive diagnostic modeling research, one area that has not had enough research interests is remedial learning or instruction based on the information provided by cognitive diagnostic assessment (CDA). The present study tries to address this research gap by exploring how effective subsequent self-directed learning will be after applying cognitive diagnostic assessment to diagnosing English grammar ability, which was not addressed in the existing CDA research. The significance of the present study is that it provides an in-depth self-study guide to investigate its effectiveness. To this end, the study administers a diagnostic test and provides detailed descriptions of their mastery levels of grammar for a small group of students who will follow through with the guided self-regulated learning. The post-test reveals that the guided self-study was notably effective with the improved total score and heightened skill mastery probabilities. The current research also examines the effectiveness of distractor analysis in optimally modifying Q-matrix for CDA. The result indicates the usefulness of distractor analysis for Q-matrix refinement when the response data are available. The article ends with some suggestions for future research based on the implications of the findings and limitations of the study.

중국어 평가 문항 작성 기법 연구

金錫永(Kim, Sok-yong),宋紅玲(Song, Hong-ryung),李康齊(Lee, Kang-jae),李美京(Lee, Mi-kyoung),李衍淑(Yi, Yeon-sook) 중국어문학연구회 2015 중국어문학논집 Vol.0 No.90

This paper critically examines the multiple-choice items in three major standardized Chinese language tests currently administered in Korea, namely, the new HSK, SNULT, and FLEX. Based on the analysis of the items, this article presents 15 different kinds of not-good items grouped under five major categories and discusses the problems in the illustrated items. Generalizing from these item examples and based on the existing literature of item writing guidelines, this study suggests 24 guidelines of seven different types for writing Chinese language items. The authors believe that the 15 invalid item patterns and the 24 item writing guidelines will significantly contribute to the betterment of test items in many standardized language tests including these three exams. It is also expected that these guidelines will provide useful criteria for classroom language assessment. The paper concludes with a suggestion that more research be conducted about item formats and optimal combination of sections in a test battery, which will help produce further guidelines pursuant to the possible new findings.