Lightening of Human Pose Estimation Algorithm Using MobileViT and Transfer Learning (MobileViT와 전이학습을 활용한 사람 자세 추정 알고리즘의 경량화)

Kunwoo Kim (김건우); Jonghyun Hong (홍종현); Jonghyuk Park (박종혁)

doi:10.9708/jksci.2023.28.09.017

Lightening of Human Pose Estimation Algorithm Using MobileViT and Transfer Learning

Journal of The Korea Society of Computer and Information
Abbr : JKSCI
2023, 28(9), pp.17-25
DOI : 10.9708/jksci.2023.28.09.017
Publisher : The Korean Society Of Computer And Information
Research Area : Engineering > Computer Science
Received : July 19, 2023
Accepted : August 30, 2023
Published : September 27, 2023

Kunwoo Kim ¹, Jonghyun Hong ¹, Jonghyuk Park ¹

¹국민대학교

Accredited

ABSTRACT

In this paper, we propose a model that can perform human pose estimation through a MobileViT-based model with fewer parameters and faster estimation. The based model demonstrates lightweight performance through a structure that combines features of convolutional neural networks with features of Vision Transformer. Transformer, which is a major mechanism in this study, has become more influential as its based models perform better than convolutional neural network-based models in the field of computer vision. Similarly, in the field of human pose estimation, Vision Transformer-based ViTPose maintains the best performance in all human pose estimation benchmarks such as COCO, OCHuman, and MPII. However, because Vision Transformer has a heavy model structure with a large number of parameters and requires a relatively large amount of computation, it costs users a lot to train the model. Accordingly, the based model overcame the insufficient Inductive Bias calculation problem, which requires a large amount of computation by Vision Transformer, with Local Representation through a convolutional neural network structure. Finally, the proposed model obtained a mean average precision of 0.694 on the MS COCO benchmark with 3.28 GFLOPs and 9.72 million parameters, which are 1/5 and 1/9 the number compared to ViTPose, respectively.

KEYWORDS

Deep Learning, Computer Vision, Keypoint Detection, Human Pose Estimation, Vision Transformer

Citation status

* References for papers published after 2023 are currently being built.

[confproc] B. Xiao / 2018 / Simple baselines for human pose estimation and tracking / Proceedings of the European Conference on Computer Vision : 466~481

[journal] 박종혁 / 2022 / Deep-Learning Based Real-time Fire Detection Using Object Tracking Algorithm / 한국컴퓨터정보학회논문지 / 한국컴퓨터정보학회 27(1) : 1~8

[confproc] D. Hwang / 2021 / SKU-Net: Improved U-Net using Selective Kernel Convolution for Retinal Vessel Segmentation / Proceedings of The Korea Society of Computer and Information 26(4) : 29~37

[journal] 양성봉 / 2020 / Improved CNN Algorithm for Object Detection in Large Images / 한국컴퓨터정보학회논문지 / 한국컴퓨터정보학회 25(1) : 45~53

[confproc] A. Vaswani / 2017 / Attention Is All You Need / Proceedings of the Neural Information Processing Systems

[confproc] A. Dosovitskiy / 2021 / An Image Is Worth 16X16Words: Transformers For Image Recognition At Scale / Proceedings of the International Conference on Learning Representations

[web] H. Touvron / 2022 / Training data-efficient image transformers & distillation through attention / arXiv / arXiv:2012.12877

[confproc] A. . Krizhevsky / 2012 / ImageNet Classification with Deep Convolutional Neural Networks / Proceedings of the Neural Information Processing Systems : 84~90

[confproc] C. Szegedy / 2015 / Going Deeper with Convolutions / Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

[web] K. Simonyan / 2014 / Very Deep Convolutional Networks for Large-Scale Image Recognition / arXiv / arXiv:1409.1556

[confproc] K. He / 2016 / Deep Residual Learning for Image Recognition / Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition : 770~778

[confproc] G. Huang / 2017 / Densely Connected Convolutional Networks / Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition : 4700~4708

[confproc] J. Wang / 2021 / Deep High-Resolution Representation Learning for Visual Recognition / Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition : 3349~3364

[confproc] Y. Xu1 / 2022 / ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation / Proceedings of the Neural Information Processing Systems

[confproc] T.-Y. Lin / 2014 / Microsoft coco:Common objects in context / Proceedings of the European Conference on Computer Vision

[confproc] S. -H. Zhang / 2019 / Pose2seg: Detection free human instance segmentation / Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition : 889~898

[confproc] M. Andriluka / 2014 / 2d human pose estimation: New benchmark and state of the art analysis / Proceedings of the Conference on Computer Vision and Pattern Recognition : 3686~3693

[web] T. Xiao / 2021 / Early convolutions help transformers see better / arXiv / arXiv:2106.14881

[confproc] D. Mehta / 2022 / Mobilevit: Light-Weight, General-Purpose, And Mobile-Freindly Vision Transformer / Proceedings of the International Conference on Learning Representations

[confproc] Q. Cheng / 2023 / Drone Detection Method Based on MobileViT and CA-PANet / Proceedings of the Electronics : 223~239

[confproc] Y. Yang / 2023 / MMViT-Seg: A lightweight transformer and CNN fusion network for COVID-19 segmentation / Proceedings of the Computer Methods and Programs in Biomedicine

[confproc] J. Deng / 2009 / Imagenet: A large-scale hierarchical image database / Proceedings of the Conference on Computer Vision and Pattern Recognition : 248~255

[web] M. Contributors / Openmmlab pose estimation toolbox and benchmark / https://github.com/open-mmlab/mmpose

[web] O. Russakovsky / 2014 / ImageNet Large Scale Visual Recognition Challenge / arXiv / arXiv:1409.0575

[confproc] S. J. Reddi / 2019 / On the convergence of adam and beyond / Proceedings of the International Conference on Learning Representations

[confproc] Z. Yang / 2019 / Xlnet: Generalized autoregressive pretraining for language understanding / Proceedings of the Neural Information Processing Systems

This paper was written with support from the National Research Foundation of Korea.

KJCKorea
Journal Central

Journal of The Korea Society of Computer and Information 2023 KCI Impact Factor : 0.65

Lightening of Human Pose Estimation Algorithm Using MobileViT and Transfer Learning

ABSTRACT

KEYWORDS

Citation status

* References for papers published after 2023 are currently being built.

Journal of The Korea Society of Computer and Information 2023 KCI Impact Factor : 0.65

Lightening of Human Pose Estimation Algorithm Using MobileViT and Transfer Learning

ABSTRACT

KEYWORDS

Statistics

Tools

Issue List

Citation status

KCI Citation Counts (0)

REFERENCES (26) * References for papers published after 2023 are currently being built.

Search PDF

Citation

* References for papers published after 2023 are currently being built.