Frontal Face Video Analysis for Detecting Fatigue States (피로도 측정을 위한 정면 얼굴 영상 분석)

Simyeong Cha (차시명); Jongwoo Ha (하종우); Soungwoong Yoon (윤성웅); Chang-Won Ahn (안창원)

doi:10.9708/jksci.2022.27.06.043

Frontal Face Video Analysis for Detecting Fatigue States

Journal of The Korea Society of Computer and Information
Abbr : JKSCI
2022, 27(6), pp.43-52
DOI : 10.9708/jksci.2022.27.06.043
Publisher : The Korean Society Of Computer And Information
Research Area : Engineering > Computer Science
Received : May 20, 2022
Accepted : June 13, 2022
Published : June 30, 2022

Simyeong Cha ¹, Jongwoo Ha ¹, Soungwoong Yoon ¹, Chang-Won Ahn ¹

¹바이브컴퍼니

Accredited

ABSTRACT

We can sense somebody's feeling fatigue, which means that fatigue can be detected through sensing human biometric signals. Numerous researches for assessing fatigue are mostly focused on diagnosing the edge of disease-level fatigue. In this study, we adapt quantitative analysis approaches for estimating qualitative data, and propose video analysis models for measuring fatigue state. Proposed three deep-learning based classification models selectively include stages of video analysis: object detection, feature extraction and time-series frame analysis algorithms to evaluate each stage's effect toward dividing the state of fatigue. Using frontal face videos collected from various fatigue situations, our CNN model shows 0.67 accuracy, which means that we empirically show the video analysis models can meaningfully detect fatigue state. Also we suggest the way of model adaptation when training and validating video data for classifying fatigue.

KEYWORDS

Fatigue measurement, Video analysis, Migration, Video classification, Deep learning, Machine learning

Citation status

* References for papers published after 2023 are currently being built.

[journal] Dinges, D. F / 1995 / An overview of sleepiness and accidents / Journal of sleep research 4 : 4~14

[journal] Smets, E / 1995 / The Multidimensional Fatigue Inventory (MFI) Psychometric Qualifies of an Instrument to Assess Fatigue / Journal of Psychosomatic Research 39(3) : 315~325

[confproc] Lee, Y / 2021 / An Experimental study on Self-rated Fatigue Assessment Tool for the Fatigue Risk Groups / Proceedings of 2021 KMIST Conference : 1755~1756

[journal] Choe, J.-H / 2014 / Trend of Convergence Technology between Healthcare and the IoT / Information and Communications Magazine 31(12) : 10~16

[journal] Kim, K / 2016 / Wearable Health Device Technology in the IoT Era / The Korean Institute of Electrical Engineers 65(11) : 18~22

[journal] Daeho Kim / 2020 / A Study on the Pilot Fatigue Measurement Methods for Fatigue Risk Management / 한국항공우주의학회지 / 한국항공우주의학협회 30(2) : 54~60

[other] Simonyan, K / 2014 / Very Deep Convolutional Networks for Large-Scale Image Recognition

[confproc] Szegedy, C / 2015 / Going deeper with convolutions / Proceedings of the IEEE conference on computer vision and pattern recognition : 1~9

[confproc] He, K / 2016 / Deep Residual Learning for Image Recognition / Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition : 770~778

[confproc] Chollet, F / 2017 / Xception: Deep Learning with Depthwise Separable Convolutions / Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition : 1251~1258

[confproc] Huang, G / 2017 / Densely Connected Convolutional Networks / Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition : 4700~4708

[other] Howard, A. G / 2017 / MobileNets:Efficient Convolutional Neural Networks for Mobile Vision Applications

[confproc] Tan, M / 2019 / EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks / International Conference on Machine Learning

[other] Dosovitskiy, A / 2020 / An Image is Worth 16x16 Words:Transformers for Image Recognition at Scale

[journal] Dai, Z / 2021 / CoAtNet: Marrying Convolution and Attention for All Data Sizes / Advances in Neural Information Processing Systems : 34~

[journal] Vaswani, A / 2017 / Attention Is All You Need / Advances in Neural Information Processing Systems : 30~

[journal] Hochreiter, S / 1997 / LONG SHORT-TERM MEMROY / Neural Computation 9(8) : 1735~1780

[other] Cho, K / 2014 / Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation

[confproc] Dalal / 2005 / Histograms of oriented gradients for human detection / 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05) / IEEE 1 : 886~893

[other] Soomro, K / 2012 / UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild

[other] Kay, W / 2017 / The Kinetics Human Action Video Dataset

[confproc] Girshick, R / 2014 / Rich feature hierarchies for accurate object detection and semantic segmentation / Proceedings of the IEEE conference on Computer Vision and Pattern Recognition : 580~587

[confproc] Redmon, J / 2016 / You Only Look Once: Unified, Real-Time Object Detection / Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition : 779~788

[confproc] Liu, W / 2016 / SSD: Single Shot MultiBox Detector / European conference on computer vision / Springer : 21~37

[confproc] Sagonas, C / 2013 / 300 faces in-the-wild challenge: The first facial landmark localization challenge / Proceedings of the IEEE international conference on computer vision work-shops : 397~403

[journal] 유승훈 / 2021 / Development of Acquisition System for Biological Signals using Raspberry Pi / 한국정보통신학회논문지 / 한국정보통신학회 25(12) : 1935~1941

[other] Redmon, J / 2018 / YOLOv3: An Incremental

[confproc] Yang, S / 2016 / WIDER FACE:A Face Detection Benchmark / Procedings of the IEEE conference on computer vision and pattern recognition : 5525~5533

[confproc] Kazemi, V / 2014 / One Millisecond Face Alignment with an Ensemble of Regression Trees / Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition : 1867~1874

[other] Lin, M / 2013 / Network in network

[other] Lilleholt, L / 2020 / Pandemic fatigue: Measurement, correlates, and consequences

[journal] Qiang, Y / 2017 / Dynamic fatigue measurement of human erythrocytes using dielectrophoresis / Acta biomaterialia 57 : 352~362

[journal] Escobar-Linero, E / 2022 / Worker’s physical fatigue classification using neural networks / Expert Systems with Applications 198 : 116784~

[journal] Zhuang, Q / 2020 / Driver fatigue detection method based on eye states with pupil and iris segmentation / Ieee Access 8 : 173440~173449

[journal] Yan, P / 2020 / Driver fatigue detection system based on colored and infrared eye features fusion / Computers, Materials & Continua 63(3) : 1563~1574

This paper was written with support from the National Research Foundation of Korea.

KJCKorea
Journal Central

Journal of The Korea Society of Computer and Information 2023 KCI Impact Factor : 0.65

Frontal Face Video Analysis for Detecting Fatigue States

ABSTRACT

KEYWORDS

Citation status

* References for papers published after 2023 are currently being built.

Journal of The Korea Society of Computer and Information 2023 KCI Impact Factor : 0.65

Frontal Face Video Analysis for Detecting Fatigue States

ABSTRACT

KEYWORDS

Statistics

Tools

Issue List

Citation status

KCI Citation Counts (0)

REFERENCES (35) * References for papers published after 2023 are currently being built.

Search PDF

Citation

* References for papers published after 2023 are currently being built.