통역 자동 평가의 가능성과 한계 고찰 - MTQE 적용 사례를 중심으로 -

다국어 초록 (Multilingual Abstract)

This study investigated the potential benefits and limitations of automated interpreting assessment by reviewing various approaches, including machine translation (MT) quality metrics and quality estimation (QE), followed by an experimental application of MTQE to a small-scale dataset of English-Korean consecutive interpretations by three interpreting students. CometKiwi, one of the state-of-the-art MTQE methods that show strong evaluation performance without requiring reference translations, was employed to compare automated evaluation with human evaluation. The findings reveal that automated evaluations exhibited a strong correlation with human evaluations and achieved complete agreement in ranking interpreting outputs, confirming the feasibility of QE-based automated evaluation for interpretations. At both the text and segment levels, higher-quality interpreting outputs showed greater alignment between human and automated evaluations, while lower-quality outputs tended to receive relatively higher scores from the QE model, highlighting discrepancies with human evaluations. It was noted that CometKiwi returned scores above zero for uninterpreted segments, possibly overestimating the final overall quality scores. The study suggests that automated evaluation scores could serve as useful resources for interpreting educators and students when an independent reference source is needed to support human evaluation.

목차 (Table of Contents)

• 1. 서론
• 2. 통역 자동 평가를 위한 접근법
• 2.1. MT 품질 평가 지표
• 2.2. MT 품질 예측 (MTQE)
• 3. 연구
• 3.1. 연구 자료
• 3.2. 연구 방법
• 4. 결과 및 논의
• 4.1. 인간 평가와의 일치도
• 4.2. 인간 평가자 간 불일치와 자동 평가
• 4.3. 원문 누락에 대한 자동 평가
• 5. 결론
• 참고문헌