Flow theory (4-channel model)를 적용한 자율 주행에서 운전자의 몰입도와 정신적 작업부하 평가를 통한 생리학적 이해

박진상 고려대학교 대학원 2020 국내석사

As autonomous driving technology develops, the driver’s role is gradually changing to passengers. Especially, in a semi-autonomous driving environment(level 3), drivers can engage with Non-Driving-Related-Tasks (NDRTs) more. The aim of this study is to examine how the condition change of drivers performing NDRTs is related to mental workload. Investigating how psychological signals change as the engagement that causes mental workload. In addition, this study investigates the effect of driver's engagement conducting NDRTs on driving performance. This study applied a 4-channel model (Quadrant model), one of the models of flow theory, to the semi-autonomous driving environment. To investigate the degree of engagement in the 4 conditions presented in the 4-channel model, the challenge of the NDRTs was composed 2 levels (Simple, Complex), the driver's skill was decided to depending on whether motivation is provided or not. 2 types of NDRTs (Addition, Drawing) demanding cognitive load were given to participants. They used the Flow Show Scale (FSS) to assess their engagement about NDRTs. The mental workload was also measured through NASA-TLX. During the experiment, Heart Rate (HR) and LF/HF ratio were measured by Electrocardiography (ECG). Take Over Reaction time (TOrt) was measured using a driving simulator. As a result, the subjects showed different degrees of engagement in 4 conditions. They also represented different mental workloads under these conditions. In particular, the LF/HF ratio was found to be the most relevant variable for mental workload and engagement. This shows that the highest LF/HF ratio is shown in flow conditions with a moderate mental workload. Also, Take Over reaction time (TOrt) was the slowest in the flow condition compared to other conditions.

EEG를 활용한 건설소음이 건설 작업자의 인지능력에 미치는 영향 평가

OLATUNBOSUN SAMUEL OLUWADAMILARE 충북대학교 2024 국내석사

In recent years, the construction industry has witnessed a rise in occupational fatalities, highlighting its status as one of the most hazardous sectors. Construction workers are exposed to diverse risks while engaging in dynamic onsite activities. Despite extensive research into the causes of these fatalities, unsafe behavior remains a significant concern, often stemming from impaired cognitive performance. Understanding the factors influencing cognitive performance is therefore paramount for ensuring construction safety. While noise exposure is recognized as a prominent factor affecting cognitive performance, its impact can be modulated by task difficulty levels and the acoustic properties of noise. Despite its importance, noise frequency has received relatively less attention in construction safety research. To this end, this study aims to investigate the combined effects of task difficulty and noise frequency on mental workload of construction workers. A 3-level n-back memory task is conducted under three noise conditions while EEG data of participants are recorded simultaneously. In addition, behavioral and subjective assessments are also conducted, and comparisons are made with findings from EEG assessment. The findings reveal that subjective assessment aligns with the hypothesis of a combined effect of task difficulty and noise frequency on workers' mental workload. Moreover, significant findings include the identification of brain regions and EEG frequency powers more responsive to noise- induced mental workload. Notably, low-frequency noise was found to have a more detrimental effect on cognitive performance compared to high-frequency noise. Overall, this research sheds light on the complex relationship between task difficulty, noise frequency, and cognitive performance among construction workers, offering valuable insights for improving safety protocols and interventions in the industry.

Quantitative prediction of mental workload with the ACT-R cognitive architecture

조성식 Graduate School of Information Management & Securi 2012 국내박사

A cognitive architecture, such as ACT-R, generally does not have any built-in functions to predict mental workloads even though it can directly represent an operator?s cognitive process and predict the operator?s performance at the sub-second scale. In this paper, a methodology to quantitatively predict the mental workload with a cognitive architecture, ACT-R, is proposed. A mathematical representation of the mental workload over time with respect to the activated time of the ACT-R modules was proposed in this paper. Experiments were performed on memorization tasks, visual-manual tasks, and menu selection tasks. In result, it was found that the predicted values of mental workload achieved by the proposed method were highly correlated with the mean of NASA-TLX subjective ratings from the participants. It was propsed that the way of predicting mental workload in this study can be possibly applied to alternative cognitive architectures that have similar attributes to those of ACT-R. In addtion, the method proposed in this study can be applied to quantitatively predict an operator?s mental workload over time early on in the design phase of dynamic systems.

Optimizing Mental Workload Detection in N-back Tasks using Physiological Signals with Empirical Mode Decomposition Feature Extraction and Explainable AI: A Machine Learning & CNN-Based Approach

DAYAL AAKANKSHA 인제대학교 일반대학원 2025 국내석사

In contemporary society, mental workload also referred to as stress has evolved into a ubiquitous issue affecting individuals across various domains of life, requiring precise identification and practical solutions. Recent technological advancements and data analytics have opened avenues for innovative approaches to stress detection, with a notable focus on utilizing physiological signals such as Electrocardiogram (ECG) and Photoplethysmography (PPG). Furthermore, monitoring mental workload (MW) has emerged as a critical necessity in numerous contexts, including safety measures and smart technology applications like driver awareness and Brain-Computer Interfacing (BCI). MW assessment offers insights into operator processing capabilities and subjective psychological experiences, crucial for minimizing errors and preventing "overload" conditions. Physiological signals, including ECG and PPG serve as valuable indicators for MW assessment. Most prior studies have focused on identifying stress via physiological signals. Several studies have used multiple physiological signals such as electrocardiogram (ECG), electroencephalogram (EEG), galvanic skin response (GSR), electromyogram (EMG), and arterial blood pressure (ABP) to identify stress, either in binary form (stress/no stress) or across multiple levels (e.g., low, moderate, and high). The low cost of sensors has permitted the development of commercial wearable devices that monitor and record a variety of physiological signals, including photoplethysmography (PPG), and electrocardiography (ECG). The MAUS database makes a significant contribution to meeting the demand for reliable datasets for MW evaluation. This dataset comprises various physiological signals, including single-lead ECG, GSR, fingertip PPG, and wristband PPG, collected under varying MW conditions induced through the Nback task. The N-back task provides a reliable, objective reference for MW assessment, with stimuli intensity correlating to MW levels. In our study, we have utilized ECG and PPG signals due to their established role as reliable physiological stress indicators. Integrating physiological signal processing technique, known as Empirical Mode Decomposition (EMD) for feature extraction, further enhances the understanding of stress responses. EMD facilitates extracting relevant features corresponding to stress-related components, enabling a comprehensive characterization of physiological responses. Machine learning classifiers and CNN use dataset to accurately distinguish between stress and non-stress conditions, allowing real-time monitoring and intervention. To improve the interpretability of stress detection algorithms, Shapley Additive explanations (SHAP) analysis provides insights into the contribution of individual features, enabling actionable insights from underlying physiological data. We opted for these signals for their ability to offer a comprehensive understanding of stress-related physiological changes. 현대 사회에서 정신적 작업 부하(스트레스)는 다양한 삶의 영역에서 사 람들에게 영향을 미치는 보편적인 문제로 발전하였으며, 정확한 식별과 실질적인 해결책이 필요합니다. 최근 기술 발전과 데이터 분석은 스트레 스 감지를 위한 혁신적인 접근 방식을 가능하게 했으며, 특히 심전도 (ECG) 및 광용적맥파(PPG)와 같은 생리학적 신호를 활용하는 것에 초 점을 맞추고 있습니다. 또한, 정신적 작업 부하(MW) 모니터링은 안전 조치 및 운전자 인식, 뇌-컴퓨터 인터페이싱(BCI)과 같은 스마트 기술 응용 분야에서 중요한 필요로 부상하였습니다. MW 평가는 작업자의 처 리 능력과 주관적인 심리적 경험에 대한 통찰력을 제공하며, 오류를 최 소화하고 과부하 상태를 방지하는 데 필수적입니다. 심전도(ECG) 및 광 용적맥파(PPG)와 같은 생리학적 신호는 MW 평가를 위한 중요한 지표 로 사용됩니다. 대부분의 이전 연구는 생리학적 신호를 통해 스트레스를 식별하는 데 중점을 두었습니다. 여러 연구에서 심전도(ECG), 뇌파(EEG), 피부 전 도(GSR), 근전도(EMG), 동맥혈압(ABP)과 같은 다양한 생리학적 신호 를 사용하여 스트레스를 이진 형식(스트레스/비스트레스) 또는 여러 수 준(예: 낮음, 중간, 높음)으로 식별하였습니다. 센서의 저렴한 비용 덕분 에 광용적맥파(PPG)와 심전도(ECG)를 포함한 다양한 생리학적 신호를 모니터링하고 기록하는 상용 웨어러블 장치의 개발이 가능해졌습니다. MAUS 데이터베이스는 MW 평가를 위한 신뢰할 수 있는 데이터셋에 대한 수요를 충족하는 데 중요한 기여를 하고 있습니다. 이 데이터셋은 N-back 작업을 통해 유도된 다양한 MW 조건에서 수집된 단일 리드 ECG, GSR, 손가락 PPG 및 손목 PPG를 포함한 다양한 생리학적 신호 로 구성되어 있습니다. N-back 작업은 MW 수준과 상관관계가 있는 자극 강도를 통해 MW 평가를 위한 신뢰할 수 있는 객관적 기준을 제 공합니다. 본 연구에서는 스트레스의 신뢰할 수 있는 생리학적 지표로 잘 알려진 ECG와 PPG 신호를 사용하였습니다. 생리학적 신호 처리 기술인 경험적 모드 분해(EMD)를 활용한 특징 추 출은 스트레스 반응에 대한 이해를 더욱 증진시킵니다. EMD는 스트레 스와 관련된 요소에 해당하는 관련 특징을 추출하여 생리학적 반응을 포괄적으로 설명할 수 있게 합니다. 머신러닝 분류기와 CNN을 사용하 여 스트레스와 비스트레스 상태를 정확하게 구분함으로써 실시간 모니 터링과 개입이 가능하게 합니다. 또한, 스트레스 감지 알고리즘의 해석 가능성을 향상시키기 위해 Shapley Additive 설명(SHAP) 분석을 사용 하여 개별 특징의 기여도를 이해하고, 기저 생리학적 데이터로부터 실 행 가능한 통찰력을 제공합니다. 이러한 신호들은 스트레스와 관련된 생리학적 변화를 포괄적으로 이해하는 데 기여하기 때문에 선택되었습 니다.

End-to-End Multimodal Mental Workload Classification via On-the-Fly Scalogram Generation and Mixture-of-Experts Gating

KENESBAEVA PERIYZAT ISMAYLOVNA 인제대학교 일반대학원 2025 국내석사

The demand for accurate mental workload (MW) monitoring and classification has intensified, particularly in high-stakes domains such as aerospace and healthcare. Traditional MW classification methods often rely on hand-crafted features and single-modality inputs or static fusion techniques, which offer limited accuracy and fail to fully leverage cross-sensor complementarity. Recent multimodal fusion methods—such as attention-based weighting, averaging, or majority voting—struggle to assess the relative informativeness of each modality, particularly when sensor reliability varies. To address these limitations, we propose CogniMoE, an end-to-end multimodal framework that learns directly from raw physiological signals. It introduces three key innovations: on-the-fly scalogram generation using FP16 arithmetic, which eliminates pre-computation and significantly reduces memory and processing overhead; parallel CNN-LSTM branches for each modality, incorporating attention mechanisms and dynamic dropout to extract robust spatiotemporal features; and a Mixture of Experts (MoE) gating network that adaptively fuses modalities based on real-time informativeness, maintaining performance even when a modality degrades. Trained in a subject-independent manner on diverse participants, CogniMoE demonstrates strong generalizability and scalability. Evaluations on the MAUS and CLAS datasets show that it outperforms both traditional and recent state-of-the-art approaches, achieving accuracies of 94% and 92%, respectively. Moreover, on-the-fly scalogram generation reduces memory usage and processing time by an order of magnitude, providing a lightweight and efficient solution. The MoE gating mechanism further boosts classification performance by approximately 5% on average over non-adaptive fusion strategies by dynamically adjusting modality importance based on individual participant characteristics. 정신 작업 부하(MW)의 정확한 모니터링 및 분류에 대한 수요가 특히 항공우주와 의료와 같은 고위험 분야에서 크게 증가하고 있습니다. 기 존의 MW 분류 방법은 수작업으로 추출된 특징과 단일 모달리티 입력 또는 정적 융합 기법에 의존하며, 이는 정확도가 제한적이고 센서 간 상호 보완적 정보를 충분히 활용하지 못합니다. 최근의 다중모달 융합 기법—예를 들어, 주의 기반 가중치 부여, 평균화, 다수결 투표 등—은 센서 신뢰도가 변동할 때 각 모달리티의 상대적 정보 가치를 평가하는 데 어려움을 겪고 있습니다. 이러한 한계를 극복하기 위해, 우리는 원시 생리 신호로부터 직접 학습 하는 종단간(end-to-end) 다중모달 프레임워크인 CogniMoE를 제안합 니다. CogniMoE는 세 가지 주요 혁신을 도입합니다: FP16 연산을 활 용한 실시간 스칼로그램(on-the-fly scalogram) 생성은 사전 연산을 제 거하고 메모리 및 처리 오버헤드를 크게 줄입니다; 각 모달리티에 대 해 병렬적으로 구성된 CNN-LSTM 분기는 주의 메커니즘과 동적 드 롭아웃을 통합하여 강건한 시공간 특징을 추출합니다; 그리고 전문가 혼합(MoE) 게이팅 네트워크는 실시간 정보성에 기반하여 모달리티를 적응적으로 융합하며, 특정 모달리티의 품질이 저하되더라도 성능을 유 지합니다. 다양한 피험자를 대상으로 피험자 독립적 (subject-independent) 방식으로 학습된 CogniMoE는 높은 일반화 성능 과 확장성을 입증하였습니다. MAUS 및 CLAS 데이터셋을 활용한 평 가에서 CogniMoE는 전통적 방법과 최근의 최첨단 접근법을 모두 능 가하며, 각각 94%와 92%의 정확도를 달성하였습니다. 또한, 실시간 스 칼로그램 생성은 메모리 사용량과 처리 시간을 한 자릿수 수준으로 줄 여 경량화되고 효율적인 솔루션을 제공합니다. MoE 게이팅 메커니즘 은 개별 피험자의 특성에 따라 모달리티 중요도를 동적으로 조정함으 로써 비적응형 융합 전략 대비 평균 약 5% 향상된 분류 성능을 제공 합니다.

A study on the student pilot’s mental workload due to personality types of both instructor and student

Jo, Yong Kwan 연세대학교 2002 국내석사

(A) study on the student pilot's mental workload due to personality types of both instructor and student

조용관 Graduate School, Yonsei University 2002 국내석사

입력매체가 자기개방에 주는 영향 : 인지 작업부하와 글쓰기 속도의 매개효과

김수영 연세대학교 대학원 2016 국내석사

In this study, we explored the influence of different cognitive process caused by writing via various input media to writing property. Specifically, we investigated whether the level of self-disclosure in articles can be varied depending on types of input medium and what mediation factors influence on this. For this, we compared keyboard typing to handwriting in experiment 1 and (hard) keyboard typing to smart phone touch typing in experiment 2. In experiment 2, real community site is used for external validity. The result of experiment 1 showed that participants of keyboard typing condition disclosed themselves more than handwriting condition. In experiment 2, the participants of hard keyboard typing condition disclosed themselves more than smart phone touch typing condition. The result of experiment 1 & 2 showed that when mental workload is lesser or writing speed is faster, it makes more self-disclosure. This study is the first study to find out the relation between input media and self-disclosure. 본 연구는 다양한 입력매체로 글을 쓸 때 입력매체에 따라 글을 쓰는 과정에서 수반되는 인지과정의 차이로 인해 글의 속성이 어떻게 달라지는지 알아보는 연구이다. 구체적으로, 글의 속성 중 ‘자기개방’정도가 입력 매체에 따라 차이가 있는지, 이 때 영향을 주는 매개 요인이 무엇인지 알아보았다. 이를 위해, 실험 1에서는 손글쓰기와 키보드타이핑을 비교하였고 실험 2에서는 하드 키보드타이핑과 스마트폰 터치 타이핑을 비교하였다. 실험 2에서는 실제 커뮤니티 사이트에서 실험을 진행하였다. 실험 1의 결과, 참가자들이 키보드 타이핑으로 글을 쓸 때에 손글쓰기로 글을 쓸 때보다 자기개방을 많이 하는 것으로 나타났다. 또한 실험 2에서, 하드 키보드 타이핑을 할 때가 스마트폰 터치 타이핑하여 글을 쓸 때보다 자기개방을 많이 하는 것으로 나타났다. 그리고 실험1과 2 모두에서 글을 쓸 때 요구되는 인지 작업부하가 적을수록, 속도가 빠를수록 자기개방 정도가 높은 것으로 나타났다. 실험 결과를 바탕으로 손글쓰기, 하드 키보드 타이핑, 터치 타이핑 중 하드 키보드 타이핑 시에 인지 작업부하가 가장 적고, 속도가 빨라 자기개방을 가장 많이 하게 되는 것으로 나타났다. 본 연구는 입력매체와 자기개방의 관계를 밝힌 최초의 연구이다.

Safe Workload Ergonomic Exposure Project

Schwartz, Adam H ProQuest Dissertations & Theses University of Minn 2018 해외박사(DDOD)

Measurement of Cognitive Load in Lower Limb Prosthesis Wearers: The P3 Event-Related Potential in Sitting, Standing, and Walking

Swerdloff, Margaret Marie Northwestern University ProQuest Dissertations & T 2024 해외박사(DDOD)

Elevated cognitive load is a hindrance commonly reported in lower limb prosthesis wearers, and it holds potential as a vital clinical measure for assessing prosthesis effectiveness. While previous works have focused on self-reports and dual task metrics such as reaction time for evaluating cognitive load, we lack a comprehensive neurophysiological measure that is robust to bias and precise in the time domain. We utilized the P3 (or P300) event-related potential, a neurophysiological signal derived from dry EEG data, to measure cognitive load during stationary and mobile activities of daily living. The P3 amplitude, which is inversely related to cognitive load, is well documented in EEG studies. Using an auditory oddball paradigm, we elicited P3 responses during five-minute trials of sitting, standing, and walking in individuals with transtibial and transfemoral amputation, as well as those with intact limbs. In the first study, we characterized the P3 potential in intact limb subjects and examined the impact of head motion using accelerometer data. In the second study, we compared the P3 values between participants with transtibial and transfemoral levels of amputation. Our results suggest that P3 is an effective marker of cognitive load in intact limb participants and that it is robust to head motion artifacts. In those with lower limb prostheses, we saw a greater degree of variance among the participants in the transfemoral group compared to the transtibial group. While the small sample sizes in both studies may limit the generalizability of the results, they point towards the potential of P3 as an effective rehabilitative outcome measure of cognitive load. Furthermore, the dry EEG's rapid setup time, negating the need for gel application, suggests its suitability in fast-paced clinical environments. Future work could examine the long-term or short-term changes in cognitive load, which may be instrumental in evaluating and enhancing rehabilitative approaches or prosthesis selection. While this research aimed primarily at enhancing the lives of prosthesis wearers, this work is broadly generalizable to those who use other rehabilitative assistive devices, such as wheelchairs and exoskeletons. The overarching goal is to improve the holistic well-being of all users of assistive devices by understanding and addressing the complexities of their physical and cognitive experiences and abilities.