Exploring Mixed Reality Human-Machine Interfaces for Enhanced Driving Experience and Safety = 향상된 주행 경험과 안전을 위한 혼합현실 기반 휴먼-머신 인터페이스 연구

다국어 초록 (Multilingual Abstract)

In vehicle infotainment systems (IVIS) are becoming more complex, which makes it harder to prototype realistic interactions and evaluate input techniques under consistent conditions. This thesis investigates mixed reality (MR) as a controlled test bench for IVIS tasks and presents a reusable MR IVIS toolchain built on Meta Quest 3 and Unity. The system renders world anchored WebView panels at fixed poses and maps modality specific inputs to a shared interaction vocabulary under the same access and closure rules. Timing anchors and interaction telemetry are logged on a single monotonic clock to support traceable analysis.

The toolchain is evaluated in a seated, non driving expert study with a within subjects design (N = 6). Three input modalities are compared, Controller Ray, Hand Ray, and a desk anchored Tangible. Participants complete four IVIS tasks, Music, Navigation, HVAC, and Phone, under fixed access and completion rules and identical success conditions. Objective measures include total completion time and task specific interaction measures derived from telemetry, complemented by SUS and CTAM-Lite ratings of usability, perceived control or precision, effort, safety or trust, and intention to use. Results show strong task dependence. Hand Ray produced the longest completion times overall, with the largest gap in Navigation. Tangible was fastest in Navigation, while Controller Ray was fastest in tasks dominated by pointing and selection. Subjective ratings were similar in overall level, with Tangible rated highest for perceived control and safety or trust, and Hand Ray rated highest for intention to use. Overall, the findings suggest that modality choice should be guided by task demands and interpreted using both end to end time and task specific interaction measures.

국문 초록 (Abstract)

차량 내 인포테인먼트 시스템 (IVIS) 은 점점 더 복잡해지고 있으며, 그 결과 현실적인 상호작용을 빠르게 프로토타이핑하고 일관된 조건에 서 입력 기법을 평가하는 일이 어려워지고 있습니다. 본 논문은 IVIS 과 제를 위한 통제된 평가 테스트벤치로서의 혼합현실 (MR) 을 검토하고, Meta Quest 3 와 Unity 기반의 재사용 가능한 MR IVIS 툴체인을 제안합 니다. 시스템은 고정된 자세의 월드 앵커드 WebView 패널을 렌더링하 고, 모달리티별 입력을 공통 상호작용 어휘로 매핑하며, 동일한 접근 및 종료 규칙을 적용합니다. 또한 단일 모노토닉 클록에서 타이밍 앵커와 상호작용 텔레메트리를 기록하여 추적 가능한 분석을 가능하게 합니다.

제안한 툴체인은 착석 상태의 비주행 전문가 실험에서 within subjects 설 계로 평가되었습니다 (N = 6). 세 가지 입력 모달리티, Controller Ray, Hand Ray, 그리고 책상 고정형 Tangible 을 비교하였습니다. 참가자는 고 정된 접근 및 완료 규칙과 동일한 성공 조건 하에서 네 가지 IVIS 과제, Music, Navigation, HVAC, Phone 을 수행하였습니다. 객관 지표로는 총 완료 시간과 텔레메트리 기반의 과제별 상호작용 지표를 사용하였고, 주관 지표로는 SUS 와 CTAM Lite 를 통해 사용성, 지각된 제어 또는 정 밀도, 노력, 안전 또는 신뢰, 사용 의도를 수집하였습니다.

결과는 과제 요구에 따른 강한 의존성을 보였습니다. Hand Ray 는 전 체적으로 가장 긴 완료 시간을 보였으며, 특히 Navigation 에서 차이가 가장 컸습니다. Tangible 은 Navigation 에서 가장 빠른 성능을 보였고, Controller Ray 는 포인팅과 선택이 중심인 과제에서 가장 빠른 성능을 보였습니다. 주관 평가는 전반적으로 유사한 수준이었으나, Tangible 은 지각된 제어와 안전 또는 신뢰에서 가장 높게 평가되었고, Hand Ray 는 사용 의도에서 가장 높게 평가되었습니다. 종합하면, 모달리티 선택은 과제 요구에 기반해 이루어져야 하며, 결과 해석에는 총 완료 시간과 과 제별 상호작용 지표를 함께 사용하는 것이 유용함을 시사합니다.

목차 (Table of Contents)

• Abstract i
• Table of Contents ii
• List of Figures v
• 1 Introduction 1
• 2 Fundamentals and State of the Art 3
• 2.1 Foundations of Vehicle HMI and IVIS 3
• 2.1.1 Definitions 3
• 2.1.2 Driver Vehicle Context & Task Taxonomy 6
• 2.1.3 Design Guidelines & Standards 8
• 2.1.4 Interaction Modalities & Control Elements 10
• 2.1.5 Usability and measurement concepts in IVIS evaluation 13
• 2.2 MR Technologies and Applications 14
• 2.2.1 Conventional In-Vehicle Displays 14
• 2.2.2 Head Up Displays and AR HUD Basics 15
• 2.2.3 Mixed Reality Passthrough HMDs 17
• 2.2.4 Cross Context Placement Conventions 17
• 2.3 Interaction Concepts and Modalities 18
• 2.3.1 Discrete Selection on Spatially Separated Targets 18
• 2.3.2 Continuous Manipulation: Pan, Zoom, and Bounded Value Adjustment 19
• 2.3.3 Access, Closure, and Feedback Timeliness 20
• 2.3.4 Presentation Adjacent Findings 21
• 2.3.5 Design Expectations from Prior Work 21
• 2.4 Evaluation and Gaps in Current HMI Research 22
• 2.5 Summary 24
• 3 Methodology and Study Tools 25
• 3.1 Terminology and reporting conventions 25
• 3.1.1 Compared modalities and action vocabulary 26
• 3.2 Research Need and Goals 27
• 3.2.1 Evaluation of the State of the Art 27
• 3.2.2 Identification of the Research Gap 29
• 3.2.3 Goals and Intended Contribution 31
• 3.3 Research Approachn 32
• 3.3.1 Objective and research questions 32
• 3.3.2 Methodology 33
• 4 Design and Implementation of the MR-HMI 34
• 4.1 Tasks & Global Controls 34
• 4.2 System & Physical Setup 37
• 4.2.1 Purpose, Constraints, and Scene Layout 38
• 4.2.2 Hardware Components 39
• 4.3 Software Stack & Interaction Policy 40
• 4.3.1 Interaction policy and focus management 42
• 4.3.2 Timing, errors, and validity 43
• 4.4 Application Surfaces 43
• 4.4.1 Launcher and closure 43
• 4.4.2 Application surfaces 44
• 4.4.3 Implementation and integrity 45
• 5 Execution and Evaluation of the User Study 46
• 5.1 Study Overview 46
• 5.2 Task Protocol and Counterbalancing 47
• 5.2.1 Design Overview 47
• 5.2.2 Counterbalancing Scheme 48
• 5.2.3 Practice and View Recentering 49
• 5.2.4 Session Flow and Timing 50
• 5.3 Metrics, Errors, and Trial Status 50
• 5.3.1 Objective outcomes and timing anchors 51
• 5.3.2 Derived metric computation from telemetry 51
• 5.4 Results 53
• 5.4.1 Participants 54
• 5.4.2 Data quality and trial accounting 54
• 5.4.3 Total completion time 55
• 5.4.4 Task specific interaction metrics 55
• 5.4.5 Subjective Measures (Questionnaires) 63
• 5.4.6 Questionnaire Outcomes 64
• 5.5 Limitations 71
• 5.6 Summary and Discussion 71
• 5.6.1 Interpreting Modality Differences 73
• 5.6.2 Implications for MR-IVIS Prototyping and Toolchain First Evaluation 75
• 6 Summary and Outlook 77
• 6.1 Summary and Discussion 77
• 6.2 Limitations and Outlook 80
• Bibliography 83
• 7 Appendix 89
• 7.1 System Usability Scale (SUS) 89
• 7.2 CTAM-Lite (Post-Modality) 90
• 7.3 Overall Interaction Control (Qualitative) 91