기계 독해 성능 개선을 위한 데이터 증강 기법

다국어 초록 (Multilingual Abstract)

Machine reading comprehension is a method of understanding the meaning and performing inference over a given text by computers, and it is one of the most essential techniques for understanding natural language. The question answering task yields a way to test the reasoning ability of intelligent systems. Nowadays, machine reading comprehension techniques performance has significantly improved following the recent progress of deep neural networks. Nevertheless, there may be challenges in improving performance when data is sparse. To address this issue, we leverage word-level and sentence-level data augmentation techniques through text editing, while minimizing changes to the existing models and cost. In this work, we propose data augmentation methods for a pre-trained language model, which is most widely used in English question answering tasks, to confirm the improved performance over the existing models.

참고문헌 (Reference)

1 S. Edunov, "Understanding Back-Translation at Scale" 489-500, 2018

2 X. Jiao, "TinyBERT: Distilling BERT for Natural Language Understanding" 4163-4174, 2020

3 A. Riabi, "Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering"

4 P. Rajpurkar, "SQuAD: 100, 000+ Questions for Machine Comprehension of Text" 2383-2392, 2016

5 L. Wan, "Regularization of Neural Networks using DropConnect" 1058-1066, 2013

6 A. W. Yu, "QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension" 2018

7 D. C. Ciresan, "Multi-column deep neural networks for image classification" 3642-3649, 2012

8 T. Nguyen, "MS MARCO:A Human Generated Machine Reading Comprehension Dataset"

9 A. Asai, "Logic guided data augmentation and regularization for consistent question answering" ACL 5642-5650, 2020

10 Z. Yang, "HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering" 2369-2380, 2018

11 J. Pennington, "Glove:Global vectors for word representation" 1532-1543, 2014

12 J. Wei, "EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks" 6381-6387, 2019

13 A. Anaby-Tavor, "Do Not Have Enough Data? Deep Learning to the Rescue!" AAAI 7383-7390, 2020

14 V. Karpukhin, "Dense Passage Retrieval for Open-Domain Question Answering" 6769-6781, 2020

15 V. Kumar, "Data augmentation using pre-trained transformer models"

16 A. Abdelkawi, "Complex Query Augmentation for Question Answering Over Knowledge Graphs" 571-587, 2019

17 P. Y. Simard, "Best practices for convolutional neural networks applied to visual document analysis" 958-963, 2003

18 J. Devlin, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" 4171-4186, 2019

19 S. Longpre, "An exploration of data augmentation and sampling techniques for domain-agnostic question answering"

20 I. Sato, "APAC:Augmented PAttern Classification with Neural Networks"

21 Z. Lan, "ALBERT: A Lite BERT for Self-supervised Learning of Language Representations" 2020