한문고전문헌의 기계번역 평가방안 탐색

정성훈 ( Jung¸ Sunghoon ),하지영 ( Ha¸ Jiyoung ),김우정 ( Kim¸ Woojeong ) 근역한문학회 2021 한문학논집(漢文學論集) Vol.60 No.-

이 글은 기계번역을 이용한 한문고전 번역물의 품질평가 방법을 살펴보고, 품질평가의 객관성을 제고하는 동시에 번역품질 향상에 기여할 수 있는 방안을 제안한 것이다. 고립어인 한문 고전문언문은 문체가 다양하고 문법상의 변화도 복잡하다. 또한 기계번역은 평가기준·평가목적·평가비용·텍스트의 종류 등도 함께 고려하여야 하므로 신뢰성이 높고 간편한 번역 품질 평가모델을 개발하기가 쉽지 않다. 자동평가는 기계번역의 어떤 요소가 번역 품질에 영향을 미치는지는 알 수 없으며, 점수가 가장 높은 기계번역 모델을 보여줄 수는 있지만 기계번역 품질에 대한 타당성을 보장하지는 못한다. 그리고 평가기준도 평가모델에 따라 달라질 수 있고 대량의 데이터를 필요로 하는 경우도 있다. 이런 문제점을 보완하기 위해서는 수동평가가 필요한데, 평가자 각각의 경험이나 수준이 존재하고, 평가기준에 대한 이해가 다를 수 있으며, 평가 환경이나 차수에 따른 차이 등 주관에 치우칠 우려도 불식하기 어렵다. 따라서 자동평가와 수동평가의 장단점을 고려하여 기계번역기의 성격과 목적에 맞는 평가방법을 찾아 적용하되, 기계번역 모델의 성능을 객관적으로 평가할 수 있는 척도를 개발하여야 하며, 궁극적으로 이러한 평가방법이 기계번역 모델의 문제점을 찾아 개선해나가는 데 도움이 될 수 있도록 해야 한다. The purpose of this paper is to examine several methods of translation quality evaluation on the classical chinese using machine translation, and suggest some ways to increase the objectivity of quality evaluation and improve the quality of translation. The classical chinese, an isolated language, have diverse styles and complicated grammatical changes. In addition, it is not easy to develop a reliable and easy translation quality evaluation model because machine translation should also consider evaluation standards, evaluation purposes, evaluation costs, and types of text. Automatic evaluation does not know which elements of machine translation affect translation quality, and although it can show the highest scoring machine translation model, it does not guarantee validity for machine translation quality. In addition, evaluation criteria may vary depending on the evaluation model and may require a large amount of data. To compensate for this problem, manual evaluation is required, which may have different results depending on the experience or level of the appraiser, understanding of the criteria, and the environment or number of evaluations. Therefore, considering the advantages and disadvantages of automatic and manual evaluation, an evaluation method suitable for the purpose of the machine translator shall be found and applied, but a measure shall be developed to objectively evaluate the performance of the machine translation model. And ultimately, these evaluation methods should help identify and improve the problems of the machine translation model.

기계번역 프로그램 품질에 대한 사용자 평가와 사용자의 L2 수준 간 상관관계 고찰 -한중 언어 쌍을 중심으로-

공수 한국통역번역학회 2019 통역과 번역 Vol.21 No.3

While the quality of machine translation is getting better and better, it is still not perfect. In this case, how the user treats the imperfect machine translation results determines whether the user achieves the purpose for using machine translation. In this process, user evaluations are a key factor. User evaluations are not objective nor accurate. User evaluations are a subjective evaluation of the user and is related to the user. Therefore, this paper attempts to analyze the relationship between the level of the user's L2 and user evaluations. This paper surveyed 69 Chinese users to understand the current status of their use of machine translation, including frequency of use, purpose of use, favorite machine translation, reasons for preferences, and satisfaction with the quality of the machine translation used. At the same time, for specific translation articles, it lets users evaluate the accuracy and fluency of machine translation articles. The survey results show that users with low L2 levels used machine translation at a higher frequency and were more inclined to evaluate machine translation from the perspective of ease of use. In addition, statistical analysis of the evaluation results found that users’ evaluation of machine translation was related to the user's own level L2. The lower the L2 level, the higher the evaluation of the adequacy and fluency of machine translation, and the higher the assessment of the overall quality of the machine translation.

Evaluation of Y2O3 surface machinability using ultra-precision lapping process with IED

차지완,황성철,이은상 대한기계학회 2009 JOURNAL OF MECHANICAL SCIENCE AND TECHNOLOGY Vol.23 No.4

Prospects of Y2O3 have been more extended as a great promising and creditable material for optical, electronic and mechanical purposes. Y2O3 has been more observed as a fine ceramic which has great material properties: high light transparency, excellent thermal resistance and chemical inertness. But in terms of effective application of Y2O3, its hard and brittle nature needs to be overcome during the surface machining process. Therefore, the surface machining control of Y2O3 should be conducted carefully. The evaluation for stable and continuous machining should also be investigated in various industrial fields as there are only limited studies on the subject. The lapping process with in-process electrolytic dressing (IED) is widely used for surface machining of hard and brittle materials. In this study, Y2O3 surface machinability was evaluated by using the ultra-precision lapping process with IED method by changing three major variables: applied force, wheel speed and machining time. The most suitable value of Ra 92nm surface roughness was acquired with smooth surface quality from the following machining condition: 7kg of applied force, 60rpm of wheel speed and 30minutes of machining time. After the lapping process, the machining tendency and surface characteristics were analyzed with fracture toughness and Vickers hardness for the evaluation of Y2O3 surface machinability.

특허 기계번역 결과물의 평가 - KIPRIS의 무료 한영 기계번역을 중심으로

최효은,이지은 한국통역번역학회 2017 통역과 번역 Vol.19 No.1

This study aims at assessing the quality of patent translations by K2E-PAT, a Korean-English machine translation system run by Korean Intellectual Property Office, based on the analysis of machine translations of 38 semiconductor-related patent abstracts. In the study, we’ve conducted both automated evaluation widely used in the machine translation evaluation and human evaluation which can complement the shortcomings of automated evaluation. For the automated evaluation, case insensitive BLEU was adopted as an automated score, as it is the most widespread in the intellectual property field. For human evaluation, two examiners examined the quality of machine translation in terms of fidelity and readability. Both automated and human evaluations revealed the machine translations are mostly not satisfactory according to the criteria. Human evaluation indicates that the patent abstract translations contain numerous semantic and syntactic errors as well as terminological ones, severely hampering fidelity and readability. The findings highlight the need to improve the quality standards of K2E-PAT machine translation by fostering collaboration between translation experts and computational linguists.

문법 정확도 평가(GAE): 기계 번역 모델의 정량화된 정성 평가

박도준,장영진,김학수 한국정보과학회 2022 정보과학회논문지 Vol.49 No.7

자연어 생성은 시스템의 계산 결과를 사람의 언어로 표현하는 작업을 의미한다. 이와 같은 자연어 생성 모델은 정량 평가만으로 생성된 문장의 품질을 대변할 수 없기 때문에 사람이 주관적인 기준에 따라 문장의 의미나 문법 점수를 매기는 정성 평가도 같이 사용하여 생성된 문장의 품질을 평가한다. 기존의 정성 평가는 주로 문법 적합도, 의미 적합도를 지표로 사용했으나, 평가자의 기준에 따라 큰 점수 편차가 발생하는 문제점이 존재했다. 따라서 본 논문에서는 구체적인 점수 기준을 제공해 줄 수 있는 문법 정확도 평가(Grammar Accuracy Evaluation, GAE) 방법을 제안한다. 본 논문에서는 기계 번역 모델의 번역 품질을 BLEU와 GAE를 통해 분석하였다. 분석 결과 BLEU 지표로 측정된 점수가 모델의 절대적인 성능을 대변하지 않음을 확인하였으며, GAE 지표를 통해 동의어로 대체된 어휘 및 문장 구조의 변화를 오답으로 평가한 BLEU 지표의 단점이 보완됨을 확인하였다. Natural Language Generation (NLG) refers to the operation of expressing the calculation results of a system in human language. Since the quality of generated sentences from an NLG model cannot be fully represented using only quantitative evaluation, they are evaluated using qualitative evaluation by humans in which the meaning or grammar of a sentence is scored according to a subjective criterion. Nevertheless, the existing evaluation methods have a problem as a large score deviation occurs depending on the criteria of evaluators. In this paper, we propose Grammar Accuracy Evaluation (GAE) that can provide the specific evaluating criteria. As a result of analyzing the quality of machine translation by BLEU and GAE, it was confirmed that the BLEU score does not represent the absolute performance of machine translation models and GAE compensates for the shortcomings of BLEU with flexible evaluation of alternative synonyms and changes in sentence structure.

인공지능 기반 멀티태스크를 위한 비디오 코덱의 성능평가 방법

김신,이예지,윤경로,추현곤,임한신,서정일 한국방송∙미디어공학회 2022 방송공학회논문지 Vol.27 No.3

MPEG-VCM(Video Coding for Machine) aims to standardize video codec for machines. VCM provides data sets and anchors, which provide reference data for comparison, for several machine vision tasks including object detection, object segmentation, and object tracking. The evaluation template can be used to compare compression and machine vision task performance between anchor data and various proposed video codecs. However, performance comparison is carried out separately for each machine vision task, and information related to performance evaluation of multiple machine vision tasks on a single bitstream is not provided currently. In this paper, we propose a performance evaluation method of a video codec for AI-based multi-tasks. Based on bits per pixel (BPP), which is the measure of a single bitstream size, and mean average precision(mAP), which is the accuracy measure of each task, we define three criteria for multi-task performance evaluation such as arithmetic average, weighted average, and harmonic average, and to calculate the multi-tasks performance results based on the mAP values. In addition, as the dynamic range of mAP may very different from task to task, performance results for multi-tasks are calculated and evaluated based on the normalized mAP in order to prevent a problem that would be happened because of the dynamic range.

Support Vector Machine을 이용한 암호화된 신용평가 모델 학습

이은민(Eunmin Lee),이주희(Joohee Lee) 한국정보통신학회 2023 한국정보통신학회 종합학술대회 논문집 Vol.27 No.1

최근 빅데이터를 다루기 위한 기계학습과 클라우드 컴퓨팅 기술이 발전함에 따라 개인정보를 보호하는 기계학습(Privacy-Preserving Machine Learning)이 화두가 되고 있다. 동형암호는 암호화된 상태에서 데이터의 연산이 가능하며, 양자컴퓨터를 이용한 공격에도 안전한 차세대 암호 기술이다. 본 연구에서는 개인정보보호를 위한 동형암호화된 기계학습 시나리오 중, 금융 데이터를 바탕으로 채무 불이행 확률을 예측하고 대출 여부를 결정하기 위한 신용평가 모델을 학습하는 방법을 다룬다. 먼저, 신용평가 모델을 학습하고 활용하는 일련의 과정에 대해 구체적인 시나리오를 구성하고, 안전성 요구조건을 정의한다. 또한, 신용평가에서 분류 정확도가 높은 Support Vector Machine(SVM) 학습 알고리즘을 사용하여 신용평가에 최적화된 분류 모델을 학습시킨다. 이때 SVM 학습 알고리즘으로는 동형암호 연산 적용에 적합하게 변환할 수 있는 LS(Linear Square)-SVM 모델을 적용하여 효율적인 신용평가 모델 학습 시스템을 제안한다. 본 연구를 통해 데이터 소유자의 민감한 개인정보를 보호하는 암호화된 신용평가 모델 학습이 가능하며, 학습 결과로 얻은 SVM 모델을 사용하여 신용평가 예측 결과의 정확도를 높일 수 있다. Recently, the development of machine learning and cloud computing has led to the rise of Privacy-Preserving Machine Learning (PPML). Homomorphic encryption enables computations over encrypted data without decryption, and it is secure against adversaries that use quantum computers. This study focuses on learning a credit evaluation model to determine loan eligibility using homomorphic encryption. The study outlines a concrete scenario for learning and using the credit evaluation model in a privacy-preserving way and defines the security requirements. To optimize the credit evaluation model, we use a Support Vector Machine (SVM) training algorithm with high classification accuracy. This paper proposes an efficient credit evaluation model learning system using LS(Linear Square)-SVM model, which is recomposed to an HE-friendly computation. It enables learning of a credit evaluation model over encrypted data while protecting user’s sensitive information and increases the accuracy of credit evaluation predictions.

고속가공 시스템의 정밀도 평가방법에 관한 연구

손덕수(Deuk-Soo Son),이안호(Ahn-Ho Lee),이정길(Jung-Kil Lee),이우영(Woo-Young Lee) 한국생산제조학회 2004 한국생산제조시스템학회 학술발표대회 논문집 Vol.2004 No.4

KS and ISO have proposed several evaluation methods of conventional machine tools. Even though the accuracy of the tools can be evaluated with those methods, there are still no proper evaluation methodsofhigh speed machining. Because it is hard to evaluate characteristics of high speed machining such as decrease of cutting temperature, cutting force, and reduced machining time. Therefore, new evaluation method for high speed machine should be developed. In this paper, several shapes of model have been proposed to evaluate cutting accuracy of high speed machine.

머신러닝 기반 고온 부품 열화 평가 프로그램 개발

최우성,장성호,이상민,강해수,방명환,배용채 대한기계학회 2020 大韓機械學會論文集A Vol.44 No.1

Replication is the most commonly used method in industry to assess the extent of damage and degradation of high temperature components. However, it is quite difficult to distinguish degradation levels because of the high level of uncertainty associated with the subject of the evaluator. Therefore, a more quantitative and accurate objective degradation evaluation method is necessary. In this paper, we propose a machine learning-based degradation evaluation program that evaluates the degradation grade of high temperature components using a support vector machine, which has excellent effect in classification problem among machine learning method. Also, open source image processing library is used. We verified the accuracy of the developed program by performing a degradation evaluation using image data of which the degradation grade was known. We demonstrate that the machine learning-based degradation evaluation program enables objective, quick and accurate deterioration evaluation a level that surpasses the capability of existing expert judgment. 표면 조직 복제법은 고온 부품의 손상 정도 및 열화 등급을 평가하기 위해 산업 현장에서 가장 많이 사용되는 방법이다. 그러나 평가자의 주관에 따른 불확실도가 높기 때문에 등급 간 명확한 구별이 불가하다. 따라서 보다 정량적이며 정확한 객관적인 열화 평가 방법이 필요하다. 본 논문에서는 오픈소스 이미지 처리 라이브러리와 머신러닝 기법 중 분류 문제에서 탁월한 효과를 보이는 SVM(Support Vector Machine)을 이용하여 고온 부품의 열화 등급을 평가하는 머신러닝 기반 고온 부품 열화 평가 프로그램을 소개한다. 열화 등급을 알고 있는 이미지 데이터를 활용한 열화 평가를 통해 개발된 프로그램의 유효성을 검증하였다. 머신러닝 기반 열화 평가 프로그램을 통해 기존 전문가의 판단보다 객관적이고 빠르면서 정확한 열화 평가가 가능할 것이다.

진동 신호를 이용한 캠 프로파일 CNC 연삭기의 실험적 평가에 관한 연구

이춘만,임상헌 한국공작기계학회 2005 한국공작기계학회 춘계학술대회논문집 Vol.2005 No.-

A cam profile grinding machine is a mandatory machine tool for manufacture of high precision contoured cam. Experimental evaluation of modal analysis is an effective tool to investigate dynamic behavior of a machine. This paper presents the measurement system and experimental investigation on the modal analysis of a grinding machine. The weak part of the machine is found by the experimental evaluation. The results provide structure modification data for good dynamic behaviors. And safety of the machine was confirmed by the modal analysis of modified machine design. Finally, the cam profile grinding machine was successfully developed.