Sentence-Level Neural Language Decoding based on Speech Imagery from EEG Signals

다국어 초록 (Multilingual Abstract)

This study focuses on the importance of communication in the healthcare field and the challenges faced by patients who are unable to speak due to medical conditions or treatments in expressing their needs and concerns. It highlights the potential for providing assistive solutions in such situations. To address this issue, we propose a novel approach using speech imagery of sentences, a technique where one imagines speeching without actually producing sound. The study collected electroencephalography(EEG) data from healthy participants and compared it with data collected during speech imagery generation. The study employed a dataset comprising four affirmative class sentences and four negative class sentences to conduct the experiment, utilizing two classifiers and two deep learning techniques for analysis. The results revealed that the classification accuracy for the affirmative class sentences was highest when employing regularized linear discriminant analysis(RLDA), while the classification accuracy for the negative class sentences was highest when using support vector machine(SVM). Although the study was conducted with a sample of healthy participants, it underscores the potential of speech imagery as a bidirectional communication modality for individuals who are unable to speak. Furthermore, this research represents a promising avenue for future investigations, focusing on decoding the intended messages of a select population with communication impairments.

참고문헌 (Reference)

1 N. Hashim, "Word-based classification of imagined speech using EEG" 488 : 195-204, 2018

2 E. C. Leuthardt, "Using the electrocorticographic speech network to control a brain–computer interface in humans" 8 (8): 2011

3 S. -H. Lee, "Towards an EEG-based intuitive BCI communication system using imagined speech and visual imagery" 4409-4414, 2019

4 S. -H. Lee, "Towards an EEG-based Intuitive BCI Communication System Using Imagined Speech and Visual Imagery" 4409-4414, 2019

5 M. D’Zmura, "Toward EEG sensing of imagined speech" 5610 : 40-48, 2009

6 S. Iqbal, "Time domain analysis of EEG to classify imagined speech" 380 : 793-800, 2015

7 F. Nijboer, "The influence of psychological state and motivation on brain–computer interface performance in patients with amyotrophic lateral sclerosis–a longitudinal study" 4 : 55-, 2010

8 D. S. Weisberg, "The Seductive Allure of Neuroscience Explanations" 20 (20): 470-477, 2008

9 C. Cortes, "Support-vector networks" 20 : 273-297, 1995

10 H. -y. Li, "Speech Enhancement Algorithm Based on Independent Component Analysis" 598-602, 2009

11 C. S. DaSalla, "Single-trial classification of vowel speech imagery using common spatial patterns" 22 (22): 1334-1339, 2009

12 G. S. Meltzner, "Silent speech recognition as an alternative communication device for persons with laryngectomy" 25 (25): 2386-2398, 2017

13 G. Pereyra, "Regularizing neural networks by penalizing confident output distributions, arXiv:1701.06548"

14 J. -H. Jeong, "Real-time deep neurolinguistic learning enhances noninvasive neural language decoding for brain– machine interaction" 1-14, 2022

15 E. M. Holz, "Long-term independent brain-computer interface home use improves quality of life of a patient in the locked-in state : a case study" 96 (96): S16-S26, 2015

16 S. Hochreiter, "Long short-term memory" 9 (9): 1735-1780, 1997

17 C. H. Nguyen, "Inferring imagined speech using EEG signals : a new approach using Riemannian manifold features" 15 (15): 2018

18 C. -H. Han, "Electroencephalography-based endogenous brain–computer interface for online communication with a completely locked-in patient" (18) : 1-13, 2019

19 S. Iqbal, "EEG based classification of imagined vowel sounds" 1591-1594, 2015

20 J. Choi, "Developing a motor imagery-based real-time asynchronous hybrid BCI controller for a lower-limb exoskeleton" 20 (20): 2020

21 D. -H. Lee, "Design of an EEG-based drone swarm control system using endogenous BCI paradigms" 1-5, 2021

22 L. Deng, "Deep learning : methods and applications" 7 (7): 197-387, 2014

23 Y. LeCun, "Deep learning" 521 : 436-444, 2015

24 S. Martin, "Decoding spectrotemporal features of overt and covert speech from the human cortex" 7 : 2014

25 J. -H. Jeong, "Decoding of multi-directional reaching movements for EEG-based robot arm control" 511-514, 2018

26 N. Yoshimura, "Decoding of covert vowel articulation using electroencephalography cortical currents" 10 : 2016

27 J. -H. Jeong, "Decoding movement-related cortical potentials based on subject-dependent and section-wise spectral filtering" 28 (28): 687-698, 2020

28 D. -Y. Lee, "Decoding imagined speech based on deep metric learning for intuitive BCI communication" 29 : 1363-1374, 2021

29 I. Käthner, "Comparison of eye tracking, electrooculography and an auditory brain–computer interface for binary communication : a case study with a participant in the locked-in state" 12 (12): 1-13, 2015

30 J. -H. Jeong, "Brain-controlled robotic arm system based on multi-directional CNN-biLSTM network using EEG signals" 28 (28): 1226-1238, 2020

31 J. R. Wolpaw, "Brain-computer interfaces for communication and control" 113 (113): 767-791, 2002

32 J. Zhang, "Application of back-propagation neural network in the post-blast re-entry time prediction" 3 (3): 128-148, 2023

33 J. Ye, "A two-stage linear discriminant analysis via QR-decomposition" 27 (27): 929-941, 2005