얼굴 영상 기반의 심박수 추정을 위한 딥러닝 모델의 경량화 기법 = Lightweight Deep Learning Model for Heart Rate Estimation from Facial Videos

다국어 초록 (Multilingual Abstract)

This paper proposes a deep learning method for estimating the heart rate from facial videos. Our proposed method estimates remote photoplethysmography (rPPG) signals to predict the heart rate. Although there have been proposed several methods for estimating rPPG signals, most previous methods can not be utilized in low-power single board computers due to their computational complexity. To address this problem, we construct a lightweight student model and employ a knowledge distillation technique to reduce the performance degradation of a deeper network model. The teacher model consists of 795k parameters, whereas the student model only contains 24k parameters, and therefore, the inference time was reduced with the factor of 10. By distilling the knowledge of the intermediate feature maps of the teacher model, we improved the accuracy of the student model for estimating the heart rate.
Experiments were conducted on the UBFC-rPPG dataset to demonstrate the effectiveness of the proposed method.
Moreover, we collected our own dataset to verify the accuracy and processing time of the proposed method on a real-world dataset. Experimental results on a NVIDIA Jetson Nano board demonstrate that our proposed method can infer the heart rate in real time with the mean absolute error of 2.5183 bpm.

참고문헌 (Reference)

1 S. Bobbia, "Unsupervised Skin Tissue Segmentation for Remote Photoplethysmography" 124 : 82-90, 2019

2 E. M. Nowara, "The Benefit of Distraction: Denoising Camera-Based Physiological Measurements Using Inverse Attention" 4955-4964, 2021

3 X. Niu, "SynRhythm:Learning a Deep Heart Rate Estimator from General to Specific" 3580-3585, 2018

4 G. de Haan, "Robust Pulse Rate From Chrominance-Based rPPG" 60 (60): 2878-2886, 2013

5 Z. Yu, "Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-temporal Networks" 2019

6 Z. Yu, "Remote Heart Rate Measurement from Highly Compressed Facial Videos: An End-to-end Deep Learning Solution with Video Enhancement" 151-160, 2019

7 R. Song, "PulseGAN: Learning to Generate Realistic Pulse Waveforms in Remote Photoplethysmography" 25 (25): 1373-1384, 2021

8 Z. Yu, "PhysFormer: Facial Video-Based Physiological Measurement With Temporal Difference Transformer" 4186-4196, 2022

9 E. Lee, "Meta-rPPG: Remote Heart Rate Estimation Using a Transductive Meta-learner" 392-409, 2020

10 M. Lewandowska, "Measuring Pulse Rate with a Webcam — A Non-contact Method for Evaluating Cardiac Activity" 405-410, 2011

11 S. Lin, "Knowledge Distillation via the Target-Aware Transformer" 10915-10924, 2022

12 K. Zhang, "Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks" 23 (23): 1499-1503, 2016

13 G. E. Hinton, "Distilling the Knowledge in a Neural Network"

14 W. Chen, "DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks" 349-365, 2018

15 W. Wang, "Algorithmic Principles of Remote PPG" 64 (64): 1479-1491, 2017

16 M. Poh, "Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam" 58 (58): 7-11, 2011

17 P. Gupta, "Accurate Heart-rate Estimation from Face Videos Using Qualitybased Fusion" 4132-4136, 2017

18 F. Bousefsaf, "3d Convolutional Neural Networks for Remote Pulse Rate Measurement and Mapping from Facial Video" 9 (9): 4364-, 2019