Adaptive Virtual Reality System for Emotion Detection and Induction Using Multimodal Feedback and Character Animation = 다중 모달 피드백과 캐릭터 애니메이션을 활용한 감정 감지 및 유도를 위한 적응형 가상 현실 시스템|RISS 상세보기

다국어 초록 (Multilingual Abstract)

Emotion detection and regulation have emerged as pivotal areas of research, with applications spanning mental health, education, entertainment, and human-computer interaction. This research presents a novel framework that seamlessly integrates real- time emotion detection with personalized audio-video recommendations to effectively influence emotional states. Utilizing advanced facial expression analysis, the system achieves outstanding accuracy through two custom-designed detection models. The first model delivers an accuracy of 92%, balancing performance with computational efficiency. The second model, Squeeze-SparrowNet, achieves a remarkable accuracy of 98%, combining the compression efficiency of SqueezeNet with the optimization power of the Sparrow Search Algorithm. This lightweight architecture ensures exceptional performance while maintaining suitability for real-time applications. The framework transitions effortlessly from emotion detection to the recommendation of tailored audio or video stimuli designed to induce positive emotions. By ensuring contextually relevant and impactful interventions, the system enhances both user engagement and emotional well-being. To validate its effectiveness, a 3D PC-VR virtual environment was developed, incorporating an interactive virtual character with expressive facial animations, natural gestures, and real-time communication capabilities. This immersive environment dynamically adapts to the user’s emotional state, playing content-based audio-video content in the background when negative emotions are detected. The integration of interactive virtual elements with adaptive recommendations offers a empathetic user experience. The system’s evaluation, conducted through both quantitative and subjective analyses, revealed high user satisfaction with its ability to accurately detect emotions and provide contextually appropriate content. Users particularly appreciated the immersive 3D environment and the dynamic interactions of the virtual character, which fostered realistic and engaging experiences. By seamlessly combining state-of-the-art emotion detection models with personalized audio-video recommendations, this framework demonstrates its potential to revolutionize emotional regulation. This user-centric approach to emotional engagement represents a significant advancement in emotion-aware technologies, with far-reaching implications for transforming how users interact with digital environments.

Keywords: Virtual Reality, Real-Time Emotion Recognition, Affective Computing, Emotion Induction Techniques, Recommender System, Digital Healthcare Systems.

번역하기

목차 (Table of Contents)

1. INTRODUCTION 15
1. Research Background 15
2. Scope of Research 19
3. Research Questions 20
4. Research Gap 21

1. INTRODUCTION 15
1. Research Background 15
2. Scope of Research 19
3. Research Questions 20
4. Research Gap 21
5. Research Contributions 22
6. Organization of the Thesis 23
7. Publications 25
8. Summary of the chapter 26
2. LITERATURE REVIEW 27
1. Emotion Detection 27
1.1. Machine Learning Based Facial Emotion Recognition 28
1.2. Virtual Reality Based Emotion Recognition 33
2. Emotion Induction 42
2.1. Emotion Induction in Immersive Environments 42
2.2. Emotion Induction using Machine Learning 50
3. Recommendation Systems 60
4. Summary of the Chapter 64
3. LIGHTWEIGHT CONV-NET FOR EMOTION DETECTION 66
1. Research Background 66
2. Proposed Methodology 69
2.1. Data Acquisition70
2.2. Data Preprocessing.71
2.3. Proposed Convolutional Neural Network72
2.4. Evaluation Matrix76
3. Experimentation and Results 78
4. Comparison with state-of-the-art approaches 80
5. Summary of the chapter 82
4. REAL TIME EMOTION DETECTION AND INDUCTION VIA
AUDIO-VIDEO RECOMMENDATION 84
1. Research Background 84
2. Proposed Methodology 86
2.1. Data Collection86
2.2. Induction Mapping88
2.3. Data Preprocessing.89
2.4. Lightweight Squeeze Neural Network Architecture90
2.5. Audio-Video Recommendation94
3. Experimentation and Results 99
4. Comparison with state of the art approaches 105
5. Summary of the chapter 106
5. 3D APPLICATION DEVELOPMENT 107
2. Research Background 107
3. Proposed Methodology 110
3.1. Creating An Immersive Environment.110
3.2. Key Elements in the Environment.111
4. 3D Character Creation 112
5. Real-Time Positive Emotion Induction 112
5.1. Emotion Induction Methods113
6. Subjective Analysis 115
7. Physiological Validation Using Bio-Well GDV Device 120
8. Summary of the chapter 121
6. CONCLUSION AND FUTURE WORK. 122
References 125

참고문헌 (Reference)

1. Basic Emotions, Ekman, P., Handbook of Cognition and Emotion 98, 45–60, 16 Preprint not peer reviewed, , 1999

2. Random forests, Breiman, L., Machine learning 45: p. 5-32, , 2001

3. Gesture and thought, McNeill, D., University of Chicago press, , 2008

4. Facial action coding system, Ekman, P., W. V. Friesen, , 1978

5. Bodily expression of emotion, Wallbott, H. G., 28(6): p. 879-896, , 1998

6. Positive emotion and honesty, C. N. Noussair, Medai, E., Frontiers in psychology 12: p. 694841, , 2021

7. The proximal experience of awe, Nelson-Coffey, S. K., 14(5): p. e0216780, , 2019

8. Emotion elicitation using films, Gross, J. J., R. W. Levenson, 9(1): p. 87-108, , 1995

9. Experimental study of affect bursts, Schröder, M., 40(1-2): p. 99-116, , 2003

10. Affective computing for hci in HCI (1), Picard, R. W., Citeseer, , 1999