뇌 신호를 이용한 시각 정보 재구성: 멀티모달 데이터셋과 모델의 도전과제 = Visual Information Reconstruction Using Brain Signals: Challenges of a Multimodal Dataset and Model

다국어 초록 (Multilingual Abstract)

This study explores an alternative possibility for brain-computer interface (BCI) technology to support real-time activities such as online learning for the visually impaired. Acknowledging the limitations of existing technologies, which primarily focus on replacing hand functions in visually-enabled environments, the study raises the need for research into image generation and visual information reconstruction using brainwavesto visualize a user's thoughts. To this end, a multimodal dataset including speech-text pairs and corresponding brain signals was constructed, and the validity of the model was evaluated. Data were collected from three groups of subjects—the visually impaired, those with visual impairments, and those with normal vision—using an EPOC X-14 channel wireless EEG headset. The experimental results showed high training accuracy in signal-to-text alignment, but confirmed that the model's generalization ability was limited due to the small size of the dataset. This study demonstrates the potential of this approach while emphasizing the need for future research to build large-scale datasets and delve into complex contextual understanding.