Syntactic priming in the L2 neural language model

다국어 초록 (Multilingual Abstract)

In recent years, the increasing abilities of neural (-network) language models (NLMs) have led to examining their representation of syntactic structures. To assess the linguistic knowledge that NLMs acquire, researchers have leveraged the traditional syntactic priming paradigm to investigate the potentials of NLMs in learning abstract structural information. In this study, we concentrated on investigating the extent to which the L2 NLM is sensitive to syntactic priming in psycholinguistic. Following Sinclair et al. (2022), we adopted a novel metric with which we controled various linguistic factors. Based on this adoption, we implemented the L2 NLM trained on the L2 corpus and explored which factors influence the strength of priming effects. In so doing, we discovered that the L2 NLM is also sensitive to various linguistic factors but displays irregular syntactic priming performances depending on experiments with different types of controlled materials.

참고문헌 (Reference)

1 Prasad, G., "Using priming to uncover the organization of syntactic representations in neural language models"

2 김유희, "The Ability of L2 LSTM Language Models to Learn the Filler-Gap Dependency" 한국컴퓨터정보학회 25 (25): 27-40, 2020

3 Bock, J. K., "Syntactic persistence in language production" 18 (18): 355-387, 1986

4 최선주 ; 박명관, "Syntactic Priming by L2 LSTM Language Models" 한국현대언어학회 37 (37): 475-489, 2022

5 Sinclair, A., "Structural persistence in language models : Priming as a window into abstract language representations" 10 : 1031-1050, 2022

6 Brown, T., "Language models are few-shot learners" 3 : 1877-1901, 2020

7 최선주 ; 박명관 ; 김유희, "How are Korean Neural Language Models ‘surprised’ Layerwisely?" 한국언어과학회 28 (28): 301-317, 2021

8 Bhattacharya, D., "Filler-gaps that neural networks fail to generalize" 486-495, 2020

9 Sanh, V., "DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter"

10 Zhang, Y., "Dialogpt : Large-Scale generative pre-training for conversational response generation"

11 Gulordava, K., "Colorless green recurrent networks dream hierarchically"

12 Warstadt, A., "BLiMP : The benchmark of linguistic minimal pairs for English" 8 : 377-392, 2020

13 Tenney, I., "BERT rediscovers the classical NLP pipeline"

14 Linzen, T., "Assessing the ability of LSTMs to learn syntax-sensitive dependencies" 4 : 521-535, 2016

15 최선주 ; 박명관, "An L2 Neural Language Model of Adaptation to Dative Alternation in English" 현대영미어문학회 40 (40): 143-159, 2022

16 최선주 ; 박명관, "An L2 Neural Language Model of Adaptation" 한국영어학회 22 : 547-562, 2022

17 Van Schijndel, M., "A neural model of adaptation in reading"