Ergonomic Evaluation and Human-Centered Design of Augmented Reality Assembly Instructions = 증강현실 기반 조립 정보 제시 방식의 인간공학적 평가와 인간 중심 설계

국문 초록 (Abstract)

증강현실(AR)은 가상 정보를 현실 세계에 실시간으로 중첩하는 기술로서, 최근 제조 현장에서 기존 종이 매뉴얼을 대체하여 작업자의 조립 작업을 지원하는 핵심 도구로 자리매김하고 있다. 특히 광학 투과형 헤드마운트 디스플레이(OST-HMD)를 활용한 AR 시스템은 작업자의 양손 자유도를 보장하며 조립 지침을 동시에 제공함으로써 수동 조립 공정의 효율성을 극대화할 것으로 기대된다. 실제로 선행 연구들은 AR 매뉴얼이 전통적 방식 대비 작업 오류를 줄이고 완료 시간을 단축하며 훈련 효율을 제고한다는 점을 입증해 왔다.
그러나 인간공학적 관점에서 최적의 AR 가이드라인을 수립하기에는 몇 가지 한계가 존재한다. 기존 연구들은 주로 작업 성과(tasks performance)와 주관적 평가 지표(subjective rating)에 의존하여, 매뉴얼 유형별로 발생하는 객관적인 신경생리학적 비용, 즉 실제 인지적 소비(cognitive cost)를 규명하는 데 미흡하였다. 또한 조립 공정이 정보 획득, 부품 선택, 목표 위치 탐색 등 유기적인 세부 단계들로 구성됨에도 불구하고, 이를 단일 활동으로 간주하여 특정 매뉴얼 유형을 일괄 적용하는 정적 매뉴얼 유형 비교 연구에 머물러 왔다. 이에 본 연구는 신경생리학적 평가와 세부 작업 단계별 분석을 수행하고, 이를 바탕으로 인간공학(HF/E) 원리에 기반한 차세대 '단계 적응형(Phase-adaptive)' 시스템을 개발하여 그 효용성을 검증하였다.

다국어 초록 (Multilingual Abstract)

Augmented Reality (AR) is defined as a technology that superimposes computer-generated virtual information onto the real world in real-time, bridging the gap between physical and digital spaces. With the advent of Industry 4.0, AR is gaining increasing popularity in the manufacturing sector as a key technology to support human operators in complex tasks. Among various display interfaces, Optical See-Through Head-Mounted Displays (OST-HMDs) are being actively adopted/developed specifically for manual assembly contexts, as they provide a hands-free functionality that allows operators to use both two hands while simultaneously viewing instructions. Reflecting this industrial trend, a substantial body of literature has focused on evaluating the effectiveness of HMD-based AR systems, consistently demonstrating that AR support outperforms traditional instructional methods by reducing error rates, shortening task completion times, and enhancing training efficiency.
While extensive research has investigated the effectiveness of AR in manufacturing, earlier studies have limitations in that 1) they predominantly relied on task performance and subjective workload assessments, leaving a gap in understanding the objective neurophysiological costs (i.e. actual cognitive consumption) associated with AR manuals; 2) they evaluated assembly as a homogeneous activity using aggregate-level metrics, thereby overlooking the phase-dependent interaction between manual type and specific task phases; and 3) they focused on comparing existing static manual types rather than systematically developing and validating a new innovative systems grounded in Human Factors and Ergonomics (HF/E) principles.
To address these limitations, this dissertation pursues a comprehensive investigation through a neurophysiological evaluation, phase-specific analysis to the development of a new innovative phase-adaptive system, comprising three interconnected studies.
Study 1 conducted a multidimensional evaluation of paper-based, user-fixed AR, and world-fixed AR manuals to quantify their distinct cognitive costs. By utilizing functional Near-Infrared Spectroscopy (fNIRS) to monitor hemodynamic responses in the dorsolateral prefrontal cortex (dlPFC), this study empirically verified that AR manual types impose significantly higher cognitive control demands than the traditional manual, despite the superior task performance of the world-fixed AR type. This finding highlights a hidden cognitive cost of AR (e.g., visual-attentional conflict) that is often undetected by subjective assessments alone.
Study 2 performed a phase-specific analysis to identify the optimal manual type for each assembly phase. By decomposing the assembly cycle into distinct assembly task phases and interpreting user behaviors through Wickens’ display design principles, the study demonstrated that the optimality of a manual type is not static but phase-dependent. Specifically, the findings indicated that the Information Acquisition phase benefits from the immediate accessibility of the user-fixed AR manual, whereas the Task Execution phase (such as Target Localization and Part Selection) requires the spatial registration of the world-fixed AR manual to minimize visual clutter and task efficiency.
Study 3 developed and validated a novel ‘phase-adaptive’ AR manual based on the empirical and theoretical foundations established in the preceding studies. This system employs a hybrid system, dynamically transitioning the reference frame of instructions to match the user's real-time operation state. A controlled experiment comparing the phase-adaptive AR manual against static AR manual types demonstrated that the adaptive approach successfully integrated the benefits of both two static manual types. The system showed consistent trends toward reduced cognitive workload, improved usability, and efficient task completion, effectively mitigating the switching costs typically associated with dynamic interfaces.
The findings of this dissertation significantly advance the understanding of HF/E in AR-based assembly manuals. By providing evidence-based design guidelines for phase-specific optimization, this research contributes to the realization of human-centered AR manuals that enhance both operator performance and cognitive well-being in industrial environments.

목차 (Table of Contents)

• Abstract i
• Contents iv
• List of Tables vi
• List of Figures vii
• Chapter 1 Introduction 1
• 1.1 Research Background 1
• 1.2 Literature Review 4
• 1.3 Research Gaps 18
• 1.4 Research Objectives 19
• 1.5 Dissertation Outline 20
• Chapter 2 Comparing Paper- and AR-Based Assembly Manuals in Task Performance, dlPFC fNIRS Hemodynamic Responses and Perceived Workload 22
• 2.1 Research Hypotheses 22
• 2.2 Method 24
• 2.3 Results 34
• 2.4 Discussion 38
• 2.5 Conclusion 47
• Chapter 3 Phase-Dependent Optimality: Investigating the Effectiveness of AR Manual Types across Assembly Sub-Phases 48
• 3.1 Research Hypothesis 48
• 3.2 Method 49
• 3.3 Results 53
• 3.4 Discussion 57
• 3.5 Conclusion 63
• Chapter 4 Development and Validation of a Phase-Adaptive Augmented Reality Instruction System for Manual Assembly: Design, Implementation, and Evaluation 65
• 4.1 Introduction 65
• 4.2 System Implementation 67
• 4.3 User Evaluation 74
• 4.4 Results 75
• 4.5 Discussion 78
• 4.6 Conclusion 82
• Chapter 5 Conclusion 84
• 5.1 Summary of Key Findings 84
• 5.2 Contributions 86
• Bibliography 88
• 국문초록 111
• 감사의 글 114