The Smart Home Middleware based on Pattern Recognition of Physiological and Environmental Context : 생체 및 환경 컨텍스트의 패턴 인식에 기반한 스마트 홈 미들웨어

다국어 초록 (Multilingual Abstract)

This thesis describes the smart home middleware that provides an automatic home service based on a user’s home service pattern in a smart home. Our smart home was divided into four parts-living room, bed room, study room, and DVD room-and the smart home middleware could recognize an occupant's location using location-tracking-system-based UWB. The smart home middleware utilizes 7 basic data values to predict the home service pattern of the occupant: pulse, body temperature, facial expression value, room temperature, time, occupant location, and occupant motion. We defined the two-layer context model presented in this study as Layer 1, which defines 7 context data inputs that are acquired from the occupant, home environment, and home appliance, and Layer 2, which consists of the person's widget, the environment widget, and the device widget. Each widget manages context in the HHIML-based(human-home interaction mark-up language) XML tree structure. The context manager creates a context model and manages all contexts using the HHIML. All contexts created in HHIML format are transmitted to the Home Service Predictor. The home service predictor consists of three processes, the Supervised Algorithm-based Pattern Analyzer, the Rule-based Pattern Analyzer, and the Human Emotion Analyzer. The Supervised Algorithm-based Pattern Analyzer and the Rule-based Pattern Analyzer learn multi-contexts acquired from all sensor devices predict home services according to the learned pattern model. The Human Emotion Analyzer evaluates differences in the occupant's emotion or stress levels while the occupant enjoys home services. We applied a SVM(Support Vector Machine) as the SAPA's pattern algorithm and applied C 4.5 as the RPA's pattern algorithm. To recognize an occupant's emotion pattern, we applied K-means-based EM as the HEA's pattern recognition.

목차 (Table of Contents)

• 1. Introduction and Motivation 1
• 1.1. What is Smart Home 1
• 1.2. Elements of a Smart Home 3
• 1.2.1. Intelligent Control 3
• 1.3. Context-Aware Middleware for Smart Home 16
• 1.4. Biometrics and Emotion Recognition 18
• 1.4.1. Biometrics 18
• 1.4.2. Biometric System 20
• 1.4.3. Affective Computing 21
• 1.5. Thesis Outline 24
• 2. Background and Related Works 25
• 2.1. Smart Home Project 25
• 2.1.1. Aware Home 25
• 2.1.2. Mav Home 28
• 2.1.3. Neural Network House 31
• 2.1.4. House_n 32
• 2.1.5. NIST Smart Space Project 33
• 2.1.6. Easy Living 35
• 2.1.7. PRIMA 36
• 2.2. Context-Aware Studies 37
• 2.3. Emotion Recognition using Biometrics 40
• 3. A Conceptual Framework for Smart Home Middleware based Physiological and Environmental Context 44
• 3.1. Smart Home Middleware Scenario 44
• 3.2. Context Model 46
• 3.2.1. Context Definition 46
• 3.2.2. Context Model 46
• 3.3. The Smart Home Middleware 49
• 3.3.1. Architecture of the Smart Home Middleware 49
• 3.3.2. Flowchart of the Smart Home Middleware 54
• 3.3.3. Context Manager 56
• 3.3.4. Home Service Predictor 58
• 3.3.5. HHIML(Human-Home Interaction Markup Language) and User Profile Manager 70
• 4. Human Emotion Analyzer 78
• 4.1. Physiological Signals 78
• 4.1.1. ECG 78
• 4.1.2. EMG 80
• 4.1.3. SC(Skin Conductivity) 83
• 4.1.4. Skin Temperature 83
• 4.1.5. BVP(Blood Volume Pulse) 83
• 4.1.6. EEG 84
• 4.2. Human Emotion Analyzer 88
• 5. Implementation of the Smart Home Middleware Toolkit 90
• 5.1. Smart Home Server and Smart Home Browser 90
• 5.2. Human Emotion Analyzer 94
• 6. Using the Smart Home Toolkit as a Research Test-bed for Smart Home 99
• 6.1. Sensor Device and Appliance for Smart Home 99
• 6.2. Demo Simulation for Smart Home 101
• 7. Performance Evaluation & Issues 106
• 7.1. Performance Evaluation of the Smart Home Middleware 106
• 7.2. Human Behavior Pattern Recognition 107
• 7.2.1. Human Behavior Pattern Recognition using the Supervised Algorithm based Pattern Analyzer 107
• 7.2.2. Human Behavior Pattern Recognition using the Rule based Pattern Analyzer 114
• 7.3. Human Stress and Emotion Recognition 115
• 7.3.1. Experiment Environment for Recognition of Human Stress 115
• 7.3.2. Human Emotion Analyzer's Performance 116
• 7.3.3. Performance Analysis of the Human Emotion Recognition 116
• 8. Conclusion 119
• 9. Reference 121