Wearable Mobile-Based Emotional Response-Monitoring System for Drivers

Lee, Boon Giin,Chong, Teak Wei,Lee, Boon Leng,Park, Hee Joon,Kim, Yoon Nyun,Kim, Beomjoon IEEE 2017 IEEE transactions on human-machine systems Vol.47 No.5

<P>Negative emotional responses are a growing problem among drivers, particularly in countries with heavy traffic, and may lead to serious accidents on the road. Measuring stress-and fatigue-induced emotional responses by means of a wireless, wearable system would be useful for potentially averting roadway tragedies. The focus of this study was to develop and verify an emotional response-monitoring paradigm for drivers, derived from electromyography signals of the upper trapezius muscle, photoplethysmography signals of the earlobe, as well as inertialmotion sensing of the head movement. The relevant sensors were connected to a microcontroller unit equipped with a Bluetooth-enabled low-energy module, which allows the transmission of those sensor readings to a mobile device in real time. A mobile device application was then used to extract the data from the sensors and to determine the driver's current emotion status, via a trained support vector machine (SVM). The emotional response paradigm, tested in ten subjects, consisted of 10 min baseline, 5 min prestimulus, and 5 min poststimulus measurements. Emotional responses were categorized into three classes: relaxed, stressed, and fatigued. The analysis integrated a total of 36 features to train the SVM model, and the final stimulus results revealed a high accuracy rate (99.52%). The proposed wearable system could be applied to an intelligent driver's safety alert system, to use those emotional responses to prevent accidents affecting themselves and/or other innocent victims.</P>

Driver Alertness Monitoring Using Fusion of Facial Features and Bio-Signals

Boon-Giin Lee,Wan-Young Chung IEEE 2012 IEEE Sensors Journal Vol.12 No.7

<P>Driver drowsiness is among the leading causal factors in traffic accidents occurring worldwide. This paper describes a method to monitor driver safety by analyzing information related to fatigue using two distinct methods: eye movement monitoring and bio-signal processing. A monitoring system is designed in Android-based smartphone where it receives sensory data via wireless sensor network and further processes the data to indicate the current driving aptitude of the driver. It is critical that several sensors are integrated and synchronized for a more realistic evaluation of the driver behavior. The sensors applied include a video sensor to capture the driver image and a bio-signal sensor to gather the driver photoplethysmograph signal. A dynamic Bayesian network framework is used for the driver fatigue evaluation. A warning alarm is sounded if driver fatigue is believed to reach a defined threshold. The manifold testing of the system demonstrates the practical use of multiple features, particularly with discrete methods, and their fusion enables a more authentic and ample fatigue detection.</P>

3D Navigation Real Time RSSI-based Indoor Tracking Application

Lee, Boon-Giin,Lee, Young-Sook,Chung, Wan-Young The Institute of Electronics and Information Engin 2008 JUCT : Journal of Ubiquitous Convergence Technolog Vol.2 No.2

Representation of various types of information in an interactive virtual reality environment on mobile devices had been an attractive and valuable research in this new era. Our main focus is presenting spatial indoor location sensing information in 3D perception in mind to replace the traditional 2D floor map using handheld PDA. Designation of 3D virtual reality by Virtual Reality Modeling Language (VRML) demonstrates its powerful ability in providing lots of useful positioning information for PDA user in real-time situation. Furthermore, by interpolating portal culling algorithm would reduce the 3D graphics rendering time on low power processing PDA significantly. By fully utilizing the CC2420 chipbased sensor nodes, wireless sensor network was established to locate user position based on Received Signal Strength Indication (RSSI) signals. Implementation of RSSI-based indoor tracking method is low-cost solution. However, due to signal diffraction, shadowing and multipath fading, high accuracy of sensing information is unable to obtain even though with sophisticated indoor estimation methods. Therefore, low complexity and flexible accuracy refinement algorithm was proposed to obtain high precision indoor sensing information. User indoor position is updated synchronously in virtual reality to real physical world. Moreover, assignment of magnetic compass could provide dynamic orientation information of user current viewpoint in real-time.

Wearable Glove-Type Driver Stress Detection Using a Motion Sensor

Lee, Boon-Giin,Chung, Wan-Young IEEE 2017 IEEE transactions on intelligent transportation sy Vol.18 No.7

<P>Increased driver stress is generally recognized as one of the major factors leading to road accidents and loss of life. Even though physiological signals are reported as the most reliable means to measure driver stresses, they often require the use of unique and expensive sensors, which produce dynamic and varying readings within individuals. This paper presents a novel means to predict a driver's stress level by evaluating the movement pattern of the steering wheel. This is accomplished by using an inertial motion unit sensor, which is placed on a glove worn by the driver. The motion sensor selected for this paper was chosen because for its low cost and the fact that it is least affected by environmental factors as compared with a physiological signal. Experiments were conducted in three different environmental scenarios. The scenarios were classified as 'urban,' 'highway,' and 'rural,' and they were chosen to simulate contrasting stress conditions experienced by the driver. In this paper, skin conductance and driver self-reports served as a reference stress to predict the driver's stress level. Galvanic skin response, a well-known stress indicator, was captured along the driver's palm and the readings were transmitted to a mobile device via low energy Bluetooth for further processing. The results revealed that indirect measurement of steering wheel movement with an inertial motion sensor could obtain accuracies up to an average rate of 94.78%. This demonstrates the opportunity for inclusion of motion sensors in wireless driver assistance systems for ambulatory monitoring of stress levels.</P>

3D Navigation Real Time RSSI-based Indoor Tracking Application

Boon-Giin Lee,Young-Sook Lee,Wan-Young Chung 대한전자공학회 2008 JUCT : Journal of Ubiquitous Convergence Technolog Vol.2 No.2

Representation of various types of information in an interactive virtual reality environment on mobile devices had been an attractive and valuable research in this new era. Our main focus is presenting spatial indoor location sensing information in 3D perception in mind to replace the traditional 2D floor map using handheld PDA. Designation of 3D virtual reality by Virtual Reality Modeling Language (VRML) demonstrates its powerful ability in providing lots of useful positioning information for PDA user in real-time situation. Furthermore, by interpolating portal culling algorithm would reduce the 3D graphics rendering time on low power processing PDA significantly. By fully utilizing the CC2420 chip-based sensor nodes, wireless sensor network was established to locate user position based on Received Signal Strength Indication (RSSI) signals. Implementation of RSSI-based indoor tracking method is low-cost solution. However, due to signal diffraction, shadowing and multipath fading, high accuracy of sensing information is unable to obtain even though with sophisticated indoor estimation methods. Therefore, low complexity and flexible accuracy refinement algorithm was proposed to obtain high precision indoor sensing information. User indoor position is updated synchronously in virtual reality to real physical world. Moreover, assignment of magnetic compass could provide dynamic orientation information of user current viewpoint in real-time.

Smartwatch-Based Driver Vigilance Indicator With Kernel-Fuzzy-C-Means-Wavelet Method

Boon Giin Lee,Jae-Hee Park,Chuan Chin Pu,Wan-Young Chung IEEE 2016 IEEE SENSORS JOURNAL Vol.16 No.1

<P>A high-precision driver vigilance predictor could be a monetary countermeasure to reduce road accidents. Heart rate variability is a well-known measurement parameter to predict driver vigilance state, but the measurement is susceptible to motion artifact due to body movement where the electrocardiogram (ECG) sensor device had to be worn close to the heart. Thus, this paper presents a novel approach to measure the ECG from the driver palms while holding on the steering wheel. In addition, photoplethysmograms sensor attached on a driver finger can also measure the similar heart rate pattern, known as pulse rate variability. Another significant vigilance measurement parameter, respiratory rate variability, can be derived directly from the ECG with the squaring baseline method, without the usage of respiratory sensor. Furthermore, this paper is also focusing on the integration of age and gender as vigilance measurement parameter as each individual exhibits distinct signal pattern. Autonomous rules are derived from the data set that performs the kernel fuzzy c-means with if-then rules extraction, which subsequently classify the driver vigilance level into two predefined classes, that are drowsy and awake. The vigilance monitoring application is developed in smartwatch, able to perform the features extraction, and then predict the driver vigilance class based on the Kernel Fuzzy-C-Mean trained model. A vibration warning will be triggered to the driver if the driver is estimated as drowsy in five consecutive time frames. In fact, the experimental results stated that the prediction accuracy can be achieved at 97.28% on average across variant subjects.</P>

Smart Wearable Hand Device for Sign Language Interpretation System With Sensors Fusion

Lee, Boon Giin,Lee, Su Min IEEE 2018 IEEE SENSORS JOURNAL Vol.18 No.3

<P>Gesturing is an instinctive way of communicating to present a specific meaning or intent. Therefore, research into sign language interpretation using gestures has been explored progressively during recent decades to serve as an auxiliary tool for deaf and mute people to blend into society without barriers. In this paper, a smart sign language interpretation system using a wearable hand device is proposed to meet this purpose. This wearable system utilizes five flex-sensors, two pressure sensors, and a three-axis inertial motion sensor to distinguish the characters in the American sign language alphabet. The entire system mainly consists of three modules: 1) a wearable device with a sensor module; 2) a processing module; and 3) a display unit mobile application module. Sensor data are collected and analyzed using a built-in embedded support vector machine classifier. Subsequently, the recognized alphabet is further transmitted to a mobile device through Bluetooth low energy wireless communication. An Android-based mobile application was developed with a text-to-speech function that converts the received textinto audible voice output. Experiment results indicate that a true sign language recognition accuracy rate of 65.7% can be achieved on average in the first version without pressure sensors. A second version of the proposed wearable system with the fusion of pressure sensors on the middle finger increased the recognition accuracy rate dramatically to 98.2%. The proposed wearable system outperforms the existing method, for instance, although background lights, and other factors are crucial to a vision-based processing method, they are not for the proposed system.</P>

Standalone Wearable Driver Drowsiness Detection System in a Smartwatch

Lee, Boon-Leng,Lee, Boon-Giin,Chung, Wan-Young IEEE 2016 IEEE SENSORS JOURNAL Vol.16 No.13

<P>Drowsiness while driving is one of the main causes of fatal accidents, especially on monotonous routes such as highways. The goal of this paper is to design a completely standalone, distraction-free, and wearable system for driver drowsiness detection by incorporating the system in a smartwatch. The main objective is to detect the driver's drowsiness level based on the driver behavior derived from the motion data collected from the built-in motion sensors in the smartwatch, such as the accelerometer and the gyroscope. For this purpose, the magnitudes of hand movements are extracted from the motion data and are used to calculate the time, spectral, and phase domain features. The features are selected based on the feature correlation method. Eight features serve as an input to a support vector machine (SVM) classifier. After the SVM training and testing, the highest obtained accuracy was 98.15% (Karolinska sleepiness scale). This user-predefined system can be used by both left-handed and right-handed users, because different SVM models are used for different hands. This is an effective, safe, and distraction-free system for the detection of driver drowsiness.</P>

Stress Events Detection of Driver by Wearable Glove System

Lee, Dae Seok,Chong, Teak Wei,Lee, Boon Giin IEEE 2017 IEEE Sensors Journal Vol. No.

<P>This paper is focused to develop a wearable glove system to detect driver stress events in real time. The driver's stress is estimated by the use of physiological signals and steering wheel motion analysis. The steering wheel motion is analyzed by driver's hand moving characteristic. Principally, the sensors on the glove gathered the photoplethysmogram signal via fingertip, and hand motion signal via inertial motion unit. The sensor module readings are transmitted to an end terminal application via a Bluetooth low energy transmission module to compute the driver stress index. The studies are carried out in a simulated driving, which is composed of three distinct driving scenarios to study the subjects' behaviors that correlate with stress. Twenty-eight subjects are requested to perform three different driving sessions with random scenarios generated while performing various driving maneuvers to assess the dynamic of mental workloads. The stress assessments of driving test subjects are self-reported at pre- and post-stimulus as well as observed through facial expression recorded throughout the whole experiments. Moreover, this paper also aimed to investigate the correlation of stress events with different driving tasks. Stress index is computed by a support vector machine pattern classifier with extracted features from sensors reading. Notably, stress index differences were found among three driving scenarios and driving maneuvers. Results revealed the true accuracy of stress detection is greater than 95% in average.</P>

METHODS TO DETECT AND REDUCE DRIVER STRESS: A REVIEW

정완영,Teak-Wei Chong,Boon-Giin Lee 한국자동차공학회 2019 International journal of automotive technology Vol.20 No.5

Automobiles are the most common modes of transportation in urban areas. An alert mind is a prerequisite while driving to avoid tragic accidents; however, driver stress can lead to faulty decision-making and cause severe injuries. Therefore, numerous techniques and systems have been proposed and implemented to subdue negative emotions and improve the driving experience. Studies show that conditions such as the road, state of the vehicle, weather, as well as the driver’s personality, and presence of passengers can affect driver stress. All the above-mentioned factors significantly influence a driver’s attention. This paper presents a detailed review of techniques proposed to reduce and recover from driving stress. These technologies can be divided into three categories: notification alert, driver assistance systems, and environmental soothing. Notification alert systems enhance the driving experience by strengthening the driver’s awareness of his/her physiological condition, and thereby aid in avoiding accidents. Driver assistance systems assist and provide the driver with directions during difficult driving circumstances. The environmental soothing technique helps in relieving driver stress caused by changes in the environment. Furthermore, driving maneuvers, driver stress detection, driver stress, and its factors are discussed and reviewed to facilitate a better understanding of the topic.