문맥 유사도와 수식 동치성 검증을 결합한 수학식 답안 자동 판별 모델 = An Automatic Assessment Model for Mathematical Expression Answers Combining Contextual Similarity and Formula Equivalence Verification

다국어 초록 (Multilingual Abstract)

An Automatic Assessment Model for Mathematical Expression Answers Combining Contextual Similarity and Formula Equivalence Verification Mun Hee Suk Advisor : Prof. Shin JuHyun, Ph.D. Department of Software Convergence Engineering Graduate School of Future Human Resources Convergence National technical qualification practical examinations that include descriptive mathematical expression problems can assess not only examinees’ final answ ers but also their calculation processes. However, manual scoring requires co nsiderable time and manpower, and maintaining consistency among raters re mains a challenge. This study proposes an automatic assessment model for mathematical expr ession answers by combining contextual similarity and formula equivalence ve rification. In the first stage, a mathematics-specialized pretrained language m odel is used to embed both model answers and examinee responses, and the cosine similarity between the two embeddings is calculated. Based on this si milarity score, responses are preliminarily classified as correct or incorrect. In the second stage, formula equivalence verification is selectively applied to res ponses that are difficult to determine confidently through the first-stage class ification. This verification process consists of formula normalization, exact ma tching after normalization, symbolic equivalence checking for expressions and equations, and numerical approximation checking. The proposed model does not apply formula equivalence verification to all responses. Instead, it identifies a verification interval around the first-stage cl assification threshold using validation data and applies the second-stage verif ication only to responses within this boundary region. The verification interval is selected through validation data–based performance comparison, and the s elected decision rule is then applied to the test dataset. The experimental data are drawn from the “Mathematics Automatic Solution Data” provided by AI-Hub. The experimental results show that the proposed model consistently outperforms the first-stage MathBERT classifier on both va lidation and test datasets in terms of Accuracy, Balanced Accuracy, and Macro F1-score. In comparison with baseline models, including TF-IDF, KoSBERT, a nd GPT-based approaches, the proposed model achieves the best performan ce across these three evaluation metrics. This study is meaningful in that it demonstrates the feasibility of automatic assessment for mathematical expression answers in an environment where ha ndwritten responses are converted into text through OCR. In particular, the re sults show that combining contextual similarity with formula equivalence verific ation improves both overall accuracy and balanced classification performance. The proposed model provides a practical foundation for applying automatic gr ading to national technical qualification practical examinations. Its performance is expected to improve further as computer-based testing environments and structured mathematical input tools, such as equation editors, become more widely available.

목차 (Table of Contents)

• I. 서론 1
• A. 연구 배경 및 목적 1
• B. 연구 내용 및 구성 3
• II. 관련 연구 4
• A. 텍스트 유사도 기반 연구 5
• 1. TF-IDF 기반 모델 5
• 2. BERT 모델 6
• B. LLM 활용 연구 8
• C. 수식 동치성 검증 연구 8
• III. 문맥 유사도와 수식 동치성 검증을 결합한 답안 판별 10
• A. 연구 구성도 10
• B. 1차 문맥 유사도 기반 판별 12
• 1. 데이터셋 구성 12
• 2. MathBERT 학습 13
• 3. 정오 임계값 선정 및 1차 판별 16
• C. 2차 수식 동치성 검증 17
• 1. 수식 정규화 18
• 2. 수식 동치 검증 19
• 3. 수치 근사 검증 23
• D. 수학식 답안 최종 판별 24
• 1. 수식 동치성 검증 적용구간 선정 24
• 2. 검증 적용구간 정오 판별 규칙 26
• 3. 문맥 유사도와 수식 동치성 결합 판별 27
• IV. 실험 및 결과 28
• A. 실험 데이터셋 28
• B. 실험 평가 및 분석 31
• 1. 실험 평가 방법 31
• 2. 실험 결과 분석 33
• V. 결론 및 향후 연구 39
• 참고문헌 41