정서 조절 관련 fMRI 연구의 메타분석

왕동평 전북대학교 일반대학원 2022 국내박사

Emotion regulation can promote academic engagement, cognitive functions, and learning abilities among students. The use of habitual emotion regulation strategies may affect real-time emotion regulation. Moreover, emotion regulation strategies are more prone to recruit appropriate brain regions in situations that need to regulate emotions. Different emotion regulation strategies might engage different neural networks, and these networks might be recruited to a different extent depending on the emotion regulation goal. Therefore, the purpose of this study is to explore the brain regions and networks correlated to emotion regulation, emotion regulation strategies, and emotion regulation goals through a meta-analysis of fMRI neuroimaging studies, and to derive educational implications. In order to collect fMRI data on emotion regulation, emotion regulation strategies, and emotion regulation goals electronic journals were used to search for literature reporting Talairach or MNI standard coordinates of healthy subjects from the year 2000 to 2021. Finally, 106 studies and 132 experiments were included. The coordinate-based meta-analysis of Activation Likelihood Estimation(ALE) was used to detect the significant activation regions related to emotion regulation, emotion regulation strategies, and emotion regulation goals. MACM(Meta-Analytic Connectivity Modeling) analysis was also performed to measure functional connectivity in activated regions related to the overall emotion regulation. Data analysis by using GingerALE 3.0.2, and the activated brain regions and connectivity networks were visualized by Mango 4.0.1 and BrainNet Viewer 1.7. The results are summarized as follows. First, the brain regions related to the overall emotion regulation showed a distribution pattern of the frontoparietal network (FPN) and the prefrontal cortex-limbic system network (PFC-limbic system network). They were mainly concentrated in the left medial frontal gyrus (BA6), left inferior frontal gyrus (BA45), left parahippocampal gyrus, right inferior parietal lobule (BA40), left inferior parietal lobule (BA40), and right middle frontal gyrus (BA9). The brain network of emotion regulation was interconnected with other cognitive functions (e. g. perception, attention, memory, learning, decision making, and language abilities) brain networks. Second, emotion regulation strategies reflect the diversity of emotion regulation in specific brain regions. The distraction was single activated in the left inferior frontal gyrus (BA45), left medial frontal gyrus (BA32), left medial frontal gyrus (BA6), and right middle frontal gyrus (BA9). The reappraisal was single activated in the left medial frontal gyrus (BA6), left inferior frontal gyrus (BA7), and right precentral gyrus (BA9). The suppression was single activated in the left middle frontal gyrus (BA6) and left superior frontal gyrus (BA8). The concepts of distraction, reappraisal, and suppression were likely to include specific cognitive processes. The distraction, reappraisal, and suppression co-activation areas are the left medial frontal gyrus (BA6). This result indicated that the left medial frontal gyrus (BA6) plays an important role in emotion regulation strategies. Third, emotion regulation goals reflect the diversity of emotion regulation in specific brain regions. The down-regulation was single activated in the left medial frontal gyrus (BA6), left inferior frontal gyrus (BA45), and right precentral gyrus (BA9). The maintenance was single activated in the left inferior frontal gyrus (BA47), left middle frontal gyrus (BA6), left inferior frontal gyrus (BA6), and right middle frontal gyrus (BA47). The up-regulation was single activation in the left medial frontal gyrus (BA6) and left inferior frontal gyrus (BA47). The concepts of down-regulation, maintenance, and up-regulation were likely to include specific cognitive processes. The down-regulation, maintenance, and up-regulation co-activation areas are the left medial frontal gyrus (BA6), left inferior frontal gyrus (BA45), and left supramarginal gyrus (BA40), left parahippocampal gyrus. Fourth, During the distraction, conjunction analysis for down-regulation showed that the left medial frontal gyrus (BA6), left middle frontal gyrus (BA6), left parahippocampal gyrus, right middle frontal gyrus (BA6), right inferior parietal lobule (BA40), and right parahippocampal gyrus were activated, conjunction analysis for maintenance showed that the left medial frontal gyrus (BA6) and right inferior parietal lobule (BA40) were activated, and conjunction analysis for up-regulation showed that the left medial frontal gyrus (BA6) was activated. During the reappraisal, conjunction analysis for down-regulation showed that the left medial frontal gyrus (BA6), left inferior frontal gyrus (BA45), left inferior temporal gyrus (BA20), and right inferior parietal lobule (BA40) were activated, conjunction analysis for maintenance showed that the left inferior frontal gyrus (BA45) was activated, and conjunction analysis for up-regulation showed that the right inferior parietal lobule (BA40) was activated. During the suppression, conjunction analysis for down-regulation showed that the left supramarginal gyrus (BA40) was activated, conjunction analysis for maintenance showed that the left superior frontal gyrus (BA6) was activated, conjunction analysis for up-regulation showed that the left medial frontal gyrus (BA6) and left middle frontal gyrus (BA6) were activated. Fifth, a relatively fixed brain network was formed between brain regions that activate overall emotion regulation. The left medial frontal gyrus (BA6), left inferior frontal gyrus (BA45), left parahippocampal gyrus, right inferior parietal lobule (BA40), left inferior parietal lobule (BA40), and right middle frontal gyrus (BA9) were the 6 important nodes represent the main concentrated areas of the brain where emotion regulation is activated. By combining the results of this study, we proposed an educational strategy considering corresponding functions of brain regions associated with emotion regulation, emotion regulation strategies, and emotion regulation goals. 정서 조절은 학생의 학습 참여, 인지기능, 학습 능력을 촉진시킬 수 있다. 정서 조절의 정서 조절의 목표 및 전략에 따라 다른 양상으로 나타난다. 이는 신경망 차원에서도 유사할 수 있다. 즉, 정서 조절의 목표 및 전략에 따라 활성화되는 뇌 부위가 다르게 나타날 수 있다. 이에 따라 본 연구에서는 fMRI 신경영상 연구 메타분석을 통하여 정서조절, 정서조절 전략, 정서조절 목표와 관련된 대뇌 영역과 신경망을 탐색하고 교육적 의미를 도출하 고자 하였다. 정서조절, 정서조절 전략, 정서조절 목표에 대한 fMRI 데이터를 수집하기 위하여 2000년부터 2021년까지 건강한 피험자를 대상으로 연구한 후, 뇌 영역 좌표를 보고한 문헌을 검색하였다. 최종적으로 106개의 연구와 132개의 실험 자료를 수집하였다. ALE(약자 포함)를 통해 정서 조절 전략, 목표 별로 활성화되는 뇌 영역을 확인하였으며, 이후 과 MACM (Meta-analytic connectivity modeling) 을 통해 정서 조절과 관련된 활성화 영역의 기능적 연결을 분석하였다.. 분석에는 GingerALE 3.0.2, Mango 4.0.1, BrainNet Viewer 1.7을 사용하였다. 주요 결과를 요약하면 다음과 같다. 첫째, 전반적인 정서 조절과 관련해서는 전두·두정 네트워크와 전전두엽피질-변연계 네트워크의 활성화가 나타났다. 이는 주로 좌측 내측 전두회 (left medial frontal gyrus, BA6), 좌측 하전두 회 (left inferior frontal gyrus, BA45), 좌측 해마회 (left parahippocampal gyrus), 우측 하두정소엽 (right inferior parietal lobule, BA40), 좌측 하두정소엽 (left inferior parietal lobule, BA40) 과 우측 중전두회 (right middle frontal gyrus, BA9) 에 집중되어 있었다. 둘째, 정서 조절 전략에 따른 활성화 뇌 부위를 분석한 결과, 정서 조절 전략 별로 다양한 뇌 부위가 활성화됨을 확인할 수 있었다. 주의 전환 전략의 경우 좌측 하전두 회 (left inferior frontal gyrus, BA45), 좌측 내측 전두회 (left medial frontal gyrus, BA32), 좌측 내측 전두회 (left medial frontal gyrus, BA6), 우측 중전두회 (right middle frontal gyrus, BA9)가 활성화되었고, 재평가 전략과 관련해서는 좌측 내측 전두회(left medial frontal gyrus, BA6), 좌측 하전두 회 (left inferior frontal gyrus, BA7), 과 우측 중심전회 (right precentral gyrus, BA9)이 활성화되었다. 억제 전략의 경우 좌측 중전두회 (left middle frontal gyrus, BA6) 과 좌측 위 이마회 (left superior frontal gyrus, BA8) 영역의 활성화되었다. 이를 통해 주의 전환, 재평가, 억제 전략 각각이 고유한 인지 과정과 관련 있음을 확인하였다. 세 전락 모두 활성화된 뇌 영역은 좌측 내측 전두회 (left medial frontal gyrus, BA6)로, 이는 좌측 내측 전두회 (left medial frontal gyrus, BA6) 영역이 정서 조절 전략에 있어 중요한 역할을 담당함을 뜻한다. 셋째, 정서 조절 목표별로 활성화되는 뇌 영역이 달랐다. 하향 조절의 경우 좌측 내측 전두회(left medial frontal gyrus, BA6), 좌측 하전두 회 (left inferior frontal gyrus, BA45), 과 우측 중심전회 (right precentral gyrus, BA9) 영역이 활성화되었고, 유지 조절은 좌측 하전두 회 (left inferior frontal gyrus, BA47), 좌측 중전두회 (left middle frontal gyrus, BA6), 좌측 하전두 회 (left inferior frontal gyrus, BA6), 과 우측 중전두회 (right middle frontal gyrus, BA47) 영역이 활성화되었다. 상향 조절의 경우, 좌측 내측 전두회 (left medial frontal gyrus, BA6) 과 좌측 하전두 회 (left inferior frontal gyrus, BA47) 영역이 활성화되었다. 이는 하향 조절, 유지 조절, 상향 조절 각각이 서로 구분되는 인지 과정임을 의미한다. 세 목표 모두에서 활성화되는 뇌 영역은 좌측 내측 전두회 (left medial frontal gyrus, BA6), 좌측 하전두 회 (left inferior frontal gyrus, BA45), 과 좌측 모서리위회 (left supramarginal gyrus, BA40), 좌측 해마회 (left parahippocampal gyrus) 이었다. 넷째, 주의 전환 전략에 있어서 하향 조절에 대해 결합 분석을 실시한 결과 좌측 내측 전두회 (left medial frontal gyrus (BA6), 좌측 중전두회 (left middle frontal gyrus, BA6), 좌측 해마회 (left parahippocampal gyrus), 우측 중전두회 (right middle frontal gyrus, BA6), 우측 하두정소엽 (right inferior parietal lobule, BA40), 과 우측 해마회 (right parahippocampal gyrus) 영역이 활성화됨을 확인하였다. 유지 조절에 대한 분석에서는 좌측 내측 전두회 (left medial frontal gyrus, BA6) 과 우측 하두정소엽 (right inferior parietal lobule (BA40) 영역의 활성화가, 상향 조절에 대한 분석에서는 좌측 내측 전두회 (left medial frontal gyrus, BA6) 영역의 활성화가 나타났다. 재평가 전략에 대한 결합 분석에서는 하향 조절의 경우 좌측 내측 전두회 (left medial frontal gyrus, BA6), 좌측 하전두 회 (left inferior frontal gyrus, BA45), 좌측 아래관자회 (left inferior temporal gyrus, BA20), 과 우측 하두정소엽 (right inferior parietal lobule, BA40) 영역, 유지 조절의 경우 좌측 하전두 회 (left inferior frontal gyrus, BA45) 영역, 상향 조절의 경우 우측 하두정소엽 (right inferior parietal lobule, BA40) 영역이 활성화되었다. 억제 전략에 대해 동일한 분석을 실시한 결과, 하향 조절에서는 좌측 모서리위회 (left supramarginal gyrus, BA40), 유지 조절에서는 좌측 위 이마회 (left superior frontal gyrus, BA6), 상향 조절에서는 좌측 내측 전두회 (left medial frontal gyrus, BA6) 과 좌측 중전두회 (left middle frontal gyrus, BA6) 영역이 활성화되었다. 다섯째, 뇌 네트워크를 분석한 결과 좌측 내측 전두회 (left medial frontal gyrus, BA6), 좌측 하전두 회 (left inferior frontal gyrus, BA45), 좌측 해마회 (left parahippocampal gyrus), 우측 하두정소엽 (right inferior parietal lobule, BA40), 좌측 하두정소엽 (left inferior parietal lobule, BA40), 과 우측 중전두회 (right middle frontal gyrus, BA9) 사이에 정소절과 관련하여 중요하게 집중된 6개의 노드를 확인하였다. 본 연구 결과를 바탕으로 학교 현장에서 정서조절을 교육할 때 필요한 시사점에 대해 논의하였다.

Emotion, fiction and rationality : cognitivism vs. non-cognitivism

최진희 University of Wisconsin-Madison 1999 해외박사

The focus of this dissertation is on the rationality of emotion directed toward fiction. The launch of the cognitive theory of emotion in philosophy of mind and in psychology provides us with a way to show how emotion is not, by nature, opposed to reason and rationality. However, problems still remain with respect to emotion directed toward fiction, because we are emotionally involved with a story about people that do not exist and events that did not happen. This is called the paradox of fiction. The current debate in relation to emotion and fiction in aesthetics revolves around the paradox of fiction. I believe that the paradox of fiction consists of two different problems: an explicatory problem and a classificatory problem. The former concerns how we apprehend fiction if we do not believe that what is described in fiction happened, while the latter concerns the problem concerning if our emotional responses toward fiction do not require belief of the relevant sort, how should we classify them? examine four theories on the first issue: the make-believe theory, the simulation theory, the revisionist theory, and the thought theory. I believe that the revisionist theory and the thought theory provide us with the most plausible view on how we apprehend fiction. For the latter issue, I divide cognitivism in general into two different kinds: narrow cognitivism and broad cognitivism. I defend broad cognitivism anginst narrow cognitivism. However, I believe that cognitivism in general falls short of giving a full account of emotion directed toward fiction, since it neglects the role of non-representational features of fiction in arousing emotion in the reader. In this regard, I attempt to go beyond cognitivism. Last, I examine how we can attribute rationality to emotion directed toward fiction, gi that not all emotional responses are accompanied by cognition, strictly called. By discussing various warranting conditions, I try to show how emotions can be warranted not only in terms of their cognitive component, but also in terms of the perceptual element involved in emotion.

Developmental Pathways Linking Maternal and Child Emotion Regulation: The Mediating Role of Emotion Socialization

Kim, Jee Eun 연세대학교 일반대학원 2026 국내석사

본 연구는 어머니의 정서조절이 유아기의 자녀 정서조절로 어떻게 이어지는지를 탐색하며, 이 과정에서 어머니의 정서사회화 반응과 유아의 정서조절 전략이 어떤 간접적 역할을 하는지 검증하고자 하였다. 연구에는 한국의 170쌍의 어머니–자녀가 참여하였으며, 아동은 53~72개월(평균 = 61.9개월)로 모두 4~5세였다. 어머니의 정서조절 전략(재평가, 반추)은 RESS를 통해, 정서사회화 반응(지지적·비지지적 반응)은 CCNES를 통해 측정하였다. 아동의 정서조절 유능성은 ERC로, 아동의 정서조절 전략 사용은 SIRE 이야기 완성 과제를 통해 실험적으로 평가하였다. SPSS 29.0을 사용하여 기술통계와 상관분석을 실시하였으며, AMOS 26.0을 사용한 경로분석 결과, 제시된 모형은 적합한 것으로 나타났다(χ²(31) = 34.94, p = .29, TLI = .95, CFI = .97, RMSEA = .03, 90% CI [.00, .07]). 주요 결과를 살펴보면, 어머니의 재평가 전략은 지지적 정서사회화 반응과 정적으로 관련되었으며, 이러한 지지적 반응은 아동의 적응적 정서조절 전략 사용을 예측하였다. 더 나아가, 어머니의 재평가 → 지지적 반응 → 아동의 적응적 전략 → 아동의 정서조절 유능성으로 이어지는 간접효과 또한 유의하게 나타났다. 반면, 어머니의 반추는 비지지적 반응과는 관련이 있었으나, 비지지적 반응은 아동의 부적응 전략 사용을 유의하게 예측하지 못하였고, 부적응 전략 역시 아동의 정서조절 유능성과 연결되지 않았다. 즉, 부정적 경로는 본 연구에서 지지되지 않았다. 한편, 지지적·비지지적 정서사회화 반응은 아동의 정서조절 유능성에 각각 유의한 직접효과를 보였다. 종합하면, 어머니의 적응적 정서조절은 지지적 정서사회화를 통해 아동의 적응적 전략 사용과 정서조절 유능성으로 이어지는 간접 경로를 갖는 것으로 나타났다. 본 연구는 유아기가 적응적 정서조절을 강화하기에 중요한 시기임을 시사하며, 부모의 정서조절 능력과 정서사회화 행동을 목표로 한 조기 개입의 필요성을 강조한다. The present study aimed to investigate the developmental pathways linking mothers’ emotion regulation and preschool children’s emotion regulation. A path analysis was conducted to examine how maternal emotion regulation strategies – cognitive reappraisal and rumination – were associated with children’s different emotion regulation strategies and overall regulation competence through the indirect effects of maternal emotion socialization behaviors to children’s negative emotional expressions. Participants were 170 mother–child dyads from South Korea. Children were between 53 and 72 months of age (M = 61.9, SD = 5.29), corresponding to 4 to 5 years old. Specifically, 63 children (37.1%) were 4 years old, and 107 children (62.9%) were 5 years old. The sample included 89 girls (52.4%) and 81 boys (47.6%). Mothers were between 30 and 51 years of age (M = 38.7, SD = 3.55). Maternal emotion regulation strategies were assessed using the Korean version of the Regulation of Emotion Systems Survey (RESS), focusing on reappraisal and rumination subscales. Maternal emotion socialization behaviors were measured with the Korean version of the Coping with Children’s Negative Emotions Scale (CCNES), comprising supportive and non-supportive responses. Children’s overall emotion regulation competence was reported by mothers using the Korean version of the Emotion Regulation Checklist (ERC). Finally, children’s emotion regulation strategies were experimentally assessed using the Korean version of the Emotion Regulation Strategies Story Stems (SIRE). Descriptive statistics and correlations were conducted in SPSS 29.0. Path analysis was conducted in AMOS 26.0, and the model demonstrated satisfactory fit, χ² (31) = 34.94, p = .29, TLI = .95, CFI = .97, RMSEA = .03, 90% CI [.00, .07]. Results demonstrated that maternal cognitive reappraisal was positively associated with supportive responses, which in turn predicted more frequent use of adaptive emotion regulation strategies among children and stronger regulation competence. The indirect effect from maternal reappraisal to children’s regulation competence through supportive responses and children’s adaptive strategy use was significant. Maternal rumination was positively associated with non-supportive responses; however, non-supportive responses did not significantly predict children’s maladaptive strategy use, nor did maladaptive strategies predict children’s regulation competence. Thus, the negative pathway was not supported in this study. Additionally, maternal supportive and non-supportive responses showed significant direct effects on children’s emotion regulation competence, highlighting the important role of parental emotion socialization beyond children’s strategy use. In sum, the findings suggest that maternal adaptive emotion regulation influences children’s regulation competence through indirect effects involving supportive emotion socialization and children’s adaptive strategies. These results emphasize early childhood as a crucial developmental period for strengthening adaptive emotion regulation. Consequently, it further highlights the importance of targeting parental regulatory strategies and emotion socialization behaviors in early interventions designed to promote children’s emotional development.

Fine-Grained Emotion-Controllable Text-to-Speech via Acoustic?Emotion interaction and Bidirectional State Space Model

Insung Ham 고려대학교 대학원 2026 국내석사

Recent advances in Text-to-Speech (TTS) have enabled emotionally expressive speech synthesis; however, fine-grained control of emotional intensity without degrading speech quality remains challenging. Many existing systems exhibit a trade-off between expressiveness and naturalness, especially at extreme intensity levels, largely due to simplistic feature-fusion strategies. To address this issue, this thesis presents a fine-grained emotion-controllable TTS framework built on a Bidirectional State-Space Model (BiMamba). By replacing Transformer based self-attention with a linear-time Bidirectional state-space backbone, the proposed system improves computational efficiency while maintaining strong context modeling for speech generation. Two components are introduced to enhance controllability and synthesis quality. First, Emotion-Guided Cross Attention (EGCA) is designed to model emotion–acoustic interactions by using emotion representations to selectively attend to relevant acoustic regions, producing stable and discriminative intensity representations beyond additive or concatenative conditioning. Second, a Dual Discriminative Learning strategy is adopted using a Joint Conditional and Unconditional (JCU) discriminator to jointly enforce overall realism and speaker-consistent emotion-intensity fidelity. Experiments on the Emotional Speech Dataset (ESD) demonstrate that the proposed model outperforms competitive baselines in perceptual quality, achieving higher naturalness (NMOS) and speaker similarity (SMOS) while maintaining strong emotion recognition accuracy and improved efficiency. Notably, the system remains robust at maximum intensity, substantially mitigating the quality degradation commonly observed in prior emotion-controllable TTS approaches. These results indicate that interaction-aware conditioning and dual supervision provide an effective path toward practical, high-fidelity emotional TTS with reliable intensity control.

라디오 청취자 문자 사연을 활용한 KoBERT 기반 한국어 다중 감정 분석 연구

이재아 서울과학기술대학교 2023 국내석사

최근 딥러닝 기술 연구의 발전으로 감정 분석에 관한 다양한 연구가 진행되고 있다. 초기 자연어처리 분야에서는 인공지능이 인간의 감정 또는 감성을 단순 극성인 긍/부정으로 분류하는 연구가 다수 존재하였다. 그러나 최근에는 긍/부정으로 감정 극성을 분류하는 이진 감성 분석을 넘어서 더 복잡하고 어려운 태스크인 다중 감정 분석에 관한 연구로 발전하고 있다. 이러한 다중 감정 분석 기술은 방송 분야와 융합하여 새로운 결과 창출을 기대할 수 있다. 그러나 방송 분야에서의 감정 분석 연구는 높은 관심에도 불구하고 아직 부족한 실정이다. 특히, 방송 매체 중 라디오에서 청취자 문자 사연은 실제 인간이 가질 수 있는 다양한 감정이 담겨 있는 텍스트 데이터임에도 불구하고 관련 연구는 미흡할 뿐만 아니라 실제 사람들이 사용하는 문장에 대한 한국어 다중 감정 분석에 관한 연구는 부족하다. 이에 실제 환경에서 수집한 라디오 청취자 문자 사연을 활용하여 감정 분석을 수행하는 시스템을 제안하고, 이를 통해 한국어 다중 감정 분석에 관하여 연구를 진행하였다. 본 논문에서는 실제 환경에서 수집한 라디오 청취자 문자 사연을 활용하여 한국어 다중 감정 분석 성능을 향상하는 방안을 연구하였다. 기존의 감정 분석 연구에서 보편적으로 이용한 개방 데이터셋이 아닌 실제 라디오 방송의 청취자 문자 사연을 직접 수집하여 감정 분석을 위한 한국어 데이터셋으로 활용했다는 점에서 차별성이 있다. 실제 환경에서 수집한 라디오 청취자 문자 사연을 분석함으로써 한국어 감정 분석이 어려운 언어학적 특성에 대하여 고찰해보았다. 또한, 한국어 다중 감정 분석의 정확도를 높일 수 있는 데이터셋 구성에 관한 고찰과 분석을 위해 설문조사와 실험을 수행하였다. 실험을 진행하기에 앞서, 실험을 위한 한국어 말뭉치를 구축하기 위해 감정 레이블링의 보편적인 기준을 정의하기 위하여 설문조사를 진행하였다. 또한, 한국어 및 문어체에 특화된 KoBERT 언어 모델로 한국어 다중 감정 분석 시스템을 구축하여 두 가지 실험을 진행하였다. 정제된 데이터와 정제되지 않은 데이터를 감정 분석 모델에 각각 테스트 데이터로서 주입하여 비교함으로써 비문법적인 요소들이 KoBERT 기반 한국어 다중 감정 분석 시스템 성능에 어떤 영향을 끼치는지 고찰해보았으며, 개방 데이터셋과 직접 구축한 한국어 말뭉치를 비교 분석하여 한국어 다중 감정 분석 시스템의 정확도 향상을 위한 전이학습용 데이터셋 구성 방안을 제안하였다. 본 연구에서는 한국어 감정 분석 정확도가 높다고 검증된 KoBERT 언어모델을 이용한 다중 감정 분석 시스템을 구축하여 감정 분석이 수행되는 과정에서 한국어 다중 감정 분석이 어떠한 이유로 어려운지 분석하고 데이터셋 조성에 대한 방향성을 제시하였다. 이를 통하여 한국어 텍스트 감정 분석의 정확도를 향상할 자료로 쓰이는 데에 의미가 있으며, 방송 분야에서의 감정 분석 기술 활용에 도움이 되고자 한다. With the recent development of Deep Learning technology research, various studies on Emotion Analysis are being conducted. In the early Natural Language Processing field, there were many studies in which Artificial Intelligence classified into various human emotions or into positive/negative emotion. However, recently, beyond Binary Sentiment Analysis, which classifies emotional polarity as positive/negative, it has evolved into a study on Multi-class Emotion Analysis, a more complex and difficult task. Such Multi-class Emotion Analysis technology can be expected to generate new results by converging with the broadcasting field. However, despite high interest in the field of broadcasting, research on Multi-class Emotion Analysis is still insufficient. In particular, although the Radio listeners' text messages are textual data that contains various emotions that humans can have, related studies are insufficient and studies of Korean Multi-class Emotion Analysis on sentences used by real people are insufficient. Accordingly, a system for performing Emotion Analysis using radio listeners’ text messages collected in the actual environment was proposed and through this, a study on Korean Multi-class Emotion Analysis was conducted. In this paper, a method of improving the performance of Korean Multi-class Emotion Analysis was studied by using radio listeners’ text messages collected in a real environment. It is differentiated in that it directly collects listeners’ text messages of actual radio broadcasts and uses them as a Korean dataset for Emotion Analysis, rather than an open dataset commonly used in existing Emotion Analysis studies. By analyzing the radio listeners' text messages collected in the actual environment, the linguistic characteristics that are difficult to analyze Korean emotions were examined. In addition, a survey and experiment were conducted to consider and analyze the composition of a dataset that can increase the accuracy of Korean Multi-class Emotion Analysis. Prior to conducting the experiment, a survey was conducted to define a universal standard for emotional labeling in order to build a Korean corpus for the experiment. In addition, two experiments were conducted by establishing a Korean Multi-class Emotion Analysis system with a KoBERT Language Model specialized in Korean and literary styles. We investigated how non-grammatical factors affect the performance of the KoBERT-based Korean Multi-class Emotion Analysis system by injecting refined data and unrefined data each into the Emotion Analysis model as test data, and proposed a method of constructing a dataset for Fine-tuning to improve the accuracy of the Korean Multi-class Emotion Analysis system. In this study, a Multi-class Emotion Analysis system using the KoBERT Language Model, which was proven to have high accuracy in Korean Emotion Analysis, was established to analyze Korean emotion in the process of Emotion Analysis, and to present the direction for creating a dataset. Through this, it is meaningful to be used as a material to improve the accuracy of Korean Text Emotion Analysis, and it is intended to help apply Emotion Analysis technology in the broadcasting field.

서비스 산업의 효과를 증강시키기 위한 그룹 감성 기반의 다감각 자극 기술에 대한 연구

김영주 상명대학교 일반대학원 2018 국내박사

Emotion plays an important role in enhancing interpersonal relationships and determine groups behavior. Emotion has been reported to influence on not only decision making and work efficiency but also a predictor of user behavior. Individual emotion has been studied for developing emotion recognition. Although group emotion has observed to be important of applying industry service and marketing, individual emotion has attempted to be studied for recognition and still has not been enough to describe public emotion behavior. Therefore, group emotion is demanding to be studied for recognition and application of industry. Group emotion is defined according to both the intensity of relationship and individual level of emotion. A leader's emotion of the group rather than a collection of individual emotion has been observed to be main contributor of determining a group emotion. Ripple effect describes mechanism of emotion contagion related to group emotion and unconscious mimicking emotional response each other including body movement, facial expression, and verbal expression and other physiological response. Therefore, response synchronization has been studied to determine group emotion in this study. Body movement in the group has been analyzed according to synchronization to a leader's and the group emotion has been classified into four emotion domain in Russell's emotion model. The service industry has shown much interest on customers’emotion as they are a significant factor in purchase intention or service experience. To understand the needs of customers, group emotion should be researched as customers could experience service or product differently depending on the quality of group emotion. Multi-sensory stimulation also plays an important role in how we feel and behave. For example, research on cure for learning disabilities and dementia has applied multi-sensory stimulation in recently years. Also, the service industry has invested to develop multi-sensory stimulation such as fragrance, mood lights, background music, and physical architecture to induce consumers’ positive emotions. Sensory marketing enhancing customer's interest, satisfaction and motivation according to multi-sensory stimulation has been developed by understanding the group emotion in the service domains. For verification and application of group emotion stimulated by multi-sensory to enhancement of achieving business goal. The three service domains were selected in this study based on the service process matrix: education, welfare, and marketing. First, the service effect in the kindergarten was to enhance children's attention in the class through presenting the multi-sensory stimulation. Second, the service effect in the welfare services was to improve mental health with increased physical activity levels caused by multi-sensory stimulation during the Tai Chi class. Lastly, the marketing services was to increase customer satisfaction for increasing sales in the coffee shop when presenting the multi-sensory stimulation. Ten children aged from 5 to 7 were exposed to the multi-sensory stimulation in the kindergarten class and their body movement were measured through camera. Their body movement was analyzed to verify the effects of multi-sensory stimulation on the education service domain. Nine elderly people aged from 60 to 70 were asked to participate a Tai Chi class at a welfare center. They were also exposed to multi-sensory stimulation and their body movement was analyzed to verify the effect of multi-sensory stimulation in the welfare service domain. 234 customers aged form 20 to 40 were exposed to multi-sensory stimulation which intended to induce certain emotion in specific time. Participants in the three service domains were also asked to compare a multi-sensory stimulation and a non-stimulus condition to identify enhancement of group emotion through subjective assessment. The Wilcoxon signed-rank test was performed to find the statistical significance of the mean difference (p <.05). Service satisfaction and body movement increased significantly when participants experience the multi-sensory stimulation. The amount of movement was larger when participants felt pleasant-relaxation than in pleasure-arousal emotion by the multi-sensory stimulation. On the other hand, the frequency power of movement was larger in when participants felt pleasant-relaxation. These results showed that group emotion was enhanced by the multi-sensory stimulation. In conclusion, this study proposed new method of recognizing group emotion according to measurement and analysis of body movement in the group. Group emotion has been modulated by multi-sensory stimulation for achieving service business goals effectively. Multi-sensory stimulation has been observed to improve group emotion and its proper design has accelerated customer's experience for benefit of service industry. The methods and results in this study will contribute the development of designing sensory marketing and service marketing. key words: Group emotion, Multi-sensory stimulation, Body-movement, Augmenting emotion

Adaptive Virtual Reality System for Emotion Detection and Induction Using Multimodal Feedback and Character Animation

Afzal, Sitara 세종대학교 대학원 2025 국내박사

Emotion detection and regulation have emerged as pivotal areas of research, with applications spanning mental health, education, entertainment, and human-computer interaction. This research presents a novel framework that seamlessly integrates real- time emotion detection with personalized audio-video recommendations to effectively influence emotional states. Utilizing advanced facial expression analysis, the system achieves outstanding accuracy through two custom-designed detection models. The first model delivers an accuracy of 92%, balancing performance with computational efficiency. The second model, Squeeze-SparrowNet, achieves a remarkable accuracy of 98%, combining the compression efficiency of SqueezeNet with the optimization power of the Sparrow Search Algorithm. This lightweight architecture ensures exceptional performance while maintaining suitability for real-time applications. The framework transitions effortlessly from emotion detection to the recommendation of tailored audio or video stimuli designed to induce positive emotions. By ensuring contextually relevant and impactful interventions, the system enhances both user engagement and emotional well-being. To validate its effectiveness, a 3D PC-VR virtual environment was developed, incorporating an interactive virtual character with expressive facial animations, natural gestures, and real-time communication capabilities. This immersive environment dynamically adapts to the user’s emotional state, playing content-based audio-video content in the background when negative emotions are detected. The integration of interactive virtual elements with adaptive recommendations offers a empathetic user experience. The system’s evaluation, conducted through both quantitative and subjective analyses, revealed high user satisfaction with its ability to accurately detect emotions and provide contextually appropriate content. Users particularly appreciated the immersive 3D environment and the dynamic interactions of the virtual character, which fostered realistic and engaging experiences. By seamlessly combining state-of-the-art emotion detection models with personalized audio-video recommendations, this framework demonstrates its potential to revolutionize emotional regulation. This user-centric approach to emotional engagement represents a significant advancement in emotion-aware technologies, with far-reaching implications for transforming how users interact with digital environments. Keywords: Virtual Reality, Real-Time Emotion Recognition, Affective Computing, Emotion Induction Techniques, Recommender System, Digital Healthcare Systems.

A Modular Framework for Visual Emotion Analysis and Captioning with Psychological Insights : 심리학적 통찰을 활용한 시각 감정 분석 및 캡셔닝을 위한 모듈형 프레임워크

Amirian Varnousefaderani, Bahar 경북대학교 대학원 2025 국내석사

시각적 감정 분석은 이미지에 대해 인간이 느낄 수 있는 가장 가능성이 높은 감정을 예측하는 것을 목표로 한다. 반면, 감정 기반 캡셔닝은 이미지의 시각적 내용을 설명함과 동시에 해당 이미지가 유발하는 정서적 반응을 언어로 표현하는 데 중점을 둔다. 시각적 감정 분석은 일반적으로 분류 문제로 다루어지지만, 인간 감정의 주관성과 심리적 복합성으로 인해 예측 성능과 해석 가능성에서 고유한 도전 과제가 존재한다. 본 연구는 심리학적 통찰을 바탕으로 비전-언어 모델을 활용해 이미지로부터 감정적으로 중요한 텍스트 단서를 추출하고, 이를 텍스트 기반 분류 파이프라인에 입력하여 감정을 예측하였다. 또한, 예측된 감정과 텍스트 단서를 결합해 감정 기반 캡션 생성을 위한 토대를 마련하였다. 이를 위해 대규모 언어 모델을 사용해 과업 특화 소규모 데이터셋을 구축하고, 이를 바탕으로 소형 언어 모델을 미세 조정하였다. 비전-언어 모델은 수정하지 않고 텍스트 분류기만 미세 조정하였음에도 불구하고, 제안하는 접근법은 시각적 감정 분석 성능을 30% 이상 향상시켰다. 또한, 생성된 감정 기반 캡션은 문맥적으로 자연스러우면서 다양한 감정 간 구분에도 효과적임을 확인하였다. 제안 프레임워크는 모듈형 구조로 각 구성 요소의 개선이 전체 성능 향상으로 이어지며, 파운데이션 모델의 빠른 진화에도 유연하게 호환된다. Visual Emotion Analysis (VEA) aims to predict the emotion that humans are most likely to experience when viewing an image, while emotion-aware captioning focuses on generating captions that not only describe image content but also explain the emotional response it evokes. VEA is typically framed as a classification task; however, unlike standard visual classification, it must contend with the inherently subjective and psychologically complex nature of human emotions. Despite progress in this field, this characteristic poses unique challenges for both performance and interpretability. This work explores using vision-language models (VLMs) to extract emotionally relevant textual cues from images, guided by psychological insights. These cues are then used in a simple text-based classification pipeline for emotion prediction. The extracted information and predicted emotion also serve as the foundation for generating emotion-aware captions that explain the rationale behind each emotional interpretation. To achieve this, a small task-specific dataset is first constructed using large language model APIs, and a smaller language model is then fine-tuned on this dataset for the same task. Without modifying the underlying VLM and solely by fine-tuning the selected text classifier, this approach improves VEA performance by over 30%. Moreover, the generated captions are not only contextually relevant but also more effective at distinguishing between different emotions. Furthermore, the proposed framework is inherently plug-and-play, allowing enhancements to any individual component to directly translate into overall performance gains, naturally aligning with the rapid evolution of foundation models.

Exploring changes of engineering students’ emotion through affect-aware feedbacks in physics problem-solving

Lee, Sungeun Sungkyunkwan university 2020 국내박사

The purpose of the research was to design appropriate affect-aware feedbacks to support students’physics problem-solving. For the purpose, a research was conducted in 2018. Participants were 12 freshmen at an engineering university in 2018 in Gyeonggi-do, South Korea. To conduct the research, what were the emotions experienced by the students through affect-aware feedbacks, which changes in students’ emotions were through affect-aware feedbacks and which appropriate affect-aware feedbacks supports physics problem-solving were investigated. Students’ emotions were influenced through affect-aware feedbacks. Two students could realize their emotions through affect-aware feedbacks. Students experienced positive emotions through affect-aware feedbacks mirroring positive emotion. Students experienced negative emotions through affect-aware feedbacks mirroring negative emotions. It means that students’ emotions can be regulated through affect-aware feedbacks In all problems, lower physics level students’ emotions were changed from negative to positive through affect-aware feedbacks even though they could not solve physics problems. Higher physics performance level class students’ emotions were not related with affect-aware feedback when physics problems were easy. When physics problems were difficult, higher physics performance level class students were impact during solving physics problems through affect-aware feedbacks. Emotions of all students were changed from negative to positive through affect-aware feedbacks. A few students were disappointed when they could not solve physics problems even though they were provided with affect-aware feedbacks. Students who were sensitive to emotions and were of lower physics performance level class showed negative emotions to the affect-aware feedbacks which were related to praise. Students who were sensitive to emotions and were of higher physics performance level class preferred the affect-aware feedbacks which were related to praise. Lower physics performance level class students preferred “I will help”. However, higher physics performance level class students showed negative emotion to the affect-aware feedback “I will help”. Therefore, affect-aware feedbacks related with emotions would be useful to lower physics performance level students. Instructive feedback would be useful to higher physics performance level students.

안면부 미동 분석을 통한 사회 감성 인식 및 심장 정보 추출

황성택 상명대학교 일반대학원 2018 국내박사

Emotional communication involves interactions through gestures, language, and facial expressions. Interactions are divided into non-verbal and verbal elements. Non-verbal elements have a high influence on effective emotional communication. Among the non-verbal elements, facial expression occupies the largest proportion. Therefore, emotion recognition and evaluation are proceeding using facial movement. Most emotion recognition studies using facial expressions use cameras. However, the movement generated by facial expressions can express the fake emotion. To do this, researchers studied micro-expression. The micro-expression is a very fast and small expression. These micro-expressions are used to determine the real and fake emotions. Existing facial recognition researches have recognized facial muscle movements as macro-expressions. However, in the case of micro expression, it is recognized as a form containing a macro expression. Therefore, both macro- and micro- should be classified correctly in the face. Thus, the accuracy of emotion recognition using facial movement can be improved. Therefore, this study focuses on the elimination and application of macro-movement. The basic objective and two application methods are the main goals in this study. The basic objective is to develop a system that removes the macro-movements and extracts the correct micro-movement. (1) The first objective is to based on micro-movement infer social emotion recognition. (2) The second objective is to based on micro-movement infer a cardiac response. The basic section is to detect and remove macro-movement in facial movement. In the first experiment, it creates a strong facial movement in the face and detects and removes the macro-movement. Strong expression movements were induced by performing Ekman's 6 basic. In the performed facial expression, we identified the origin of the macro movement and removed it. As a result, the strong movement of the macro-movement was removed. In the second experiment, it creates a natural facial movement on the face and detects and removes the macro-movement. Natural movements were induced by speaking. In addition, we observed the time when the macro movement occurred and removed it. As a result, we confirmed that the movements were the same when talking and that the macro-movement was removed. This confirms that the correct micro-movement has been extracted. Part 1 was social emotion recognition using micro-movement. Based on previous studies, four types of intimacy, empathy, competition-cooperation, and focus were selected for social emotion. In the first intimacy induction experiment was conducted by 30 participants. The intimacy was that the couple (friend) and the non-couple (stranger) performed mutual stimulation. As a result, it showed 78.84% (intimacy: 84.61%, non-intimacy: 73.07%), accuracy for intimacy and non-intimacy. In the second empathy induction experiment, 15 participants proceeded. Empathy proceeded with the couple (friend) seeing and executing the partner stimulus and the monitor stimulus. As a result, it showed 100% (empathy: 100%. non-empathy: 100%) accuracy for empathy and non-empathy. In the third competition-cooperation induction experiment, 52 participants proceeded. The competition-cooperation was conducted by a couple (friend) competing against the other and score competition with the computer. As a result, it showed 95.18% (competition: 92.30%, cooperation: 98.07%) accuracy for competition and cooperation. In the fourth focus induction experiment was conducted by 52 participants. The focus was on the couple (friend) to score the target according to the degree of difficulty. As a result, it showed 94.23% (low-focus: 88.46%, high-focus: 100%) accuracy for low-focus and high-focus. Part 2 was based on micro-movement inferred a cardiac response. In the expression condition experiment, 14 participants in the comparative experiment between the photoplethysmograph sensor and the camera. Photoplethysmograph and camera were simultaneously measured for 3 minutes for signal comparison. As a result, correlation coefficient 0.783 for non expression and 0.770 for expression. In the natural facial movement condition experiment, 50 participants in the comparative experiment between the electrocardiogram sensor and the camera. Electrocardiogram and camera were simultaneously measured for 5 minutes for signal comparison. As a result, correlation coefficient 0.855 for non natural and 0.832 for natural. keyword: micro-movement, macro-movement, social emotion, cardiac response, facial movement, noise rejection