Generating Augmented Lifting Player using Pose Tracking (포즈 추적을 이용한 증강현실 리프팅 플레이어 생성)

Jong-In Choi (최종인); Jong-Hyun Kim (김종현)

doi:10.9708/jksci.2020.25.05.019

Generating Augmented Lifting Player using Pose Tracking

Journal of The Korea Society of Computer and Information
Abbr : JKSCI
2020, 25(5), pp.19-26
DOI : 10.9708/jksci.2020.25.05.019
Publisher : The Korean Society Of Computer And Information
Research Area : Engineering > Computer Science
Received : April 3, 2020
Accepted : April 27, 2020
Published : May 29, 2020

Jong-In Choi ¹, Jong-Hyun Kim ²

¹서울여자대학교
²강남대학교

Accredited

ABSTRACT

This paper proposes a framework for creating acrobatic scenes such as soccer ball lifting using various users' videos. The proposed method can generate a desired result within a few seconds using a general video of user recorded with a mobile phone. The framework of this paper is largely divided into three parts. The first is to analyze the posture by receiving the user's video. To do this, the user can calculate the pose of the user by analyzing the video using a deep learning technique, and track the movement of a selected body part. The second is to analyze the movement trajectory of the selected body part and calculate the location and time of hitting the object. Finally, the trajectory of the object is generated using the analyzed hitting information. Then, a natural object lifting scenes synchronized with the input user's video can be generated. Physical-based optimization was used to generate a realistic moving object. Using the method of this paper, we can produce various augmented reality applications.

KEYWORDS

Augmented Reality, Artificial Intelligence, Deep Neural Networks, Posture Tracking, Video Synthesis

Citation status

* References for papers published after 2023 are currently being built.

[journal] Daniel Holden / 2017 / Phase-functioned neural networks for character control / ACM Transactions on Graphics / Association for Computing Machinery (ACM) 36(4) : 1~13

[journal] Xue Bin Peng / 2017 / Deeploco: dynamic locomotion skills using hierarchical deep reinforcement learning / ACM Transactions on Graphics / Association for Computing Machinery (ACM) 36(4) : 1~13

[journal] Dushyant Mehta / 2017 / Vnect: Real-time 3d human pose estimation with a single rgb camera / ACM Transactions on Graphics / Association for Computing Machinery (ACM) 36(4) : 1~14

[other] D. Mehta / 2019 / Xnect:Real-time multiperson 3d human pose estimation with a single RGB camera / CoRR, vol. abs/1907.00837,arXiv:1907.00837

[other] Z. Cao / 2018 / Openpose : Realtime multi-person 2d pose estimation using part affinity fields / CoRR, vol. abs/1812.08008, arXiv:1611.08050

[confproc] M. Andriluka / 2009 / Pictorial structures revisited: People detection and articulated pose estimation / CVPR : 1014~1021

[confproc] S. Johnson / 2010 / Clustered pose and nonlinear appearance models for human pose estimation : 1~11

[confproc] A. Bulat / 2016 / Human pose estimation via convolutional part heatmap regression 9911

[confproc] V. Ramakrishna / 2014 / Pose machines: Articulated pose estimation via inference machines / Conference Paper, Proceedings of European Conference on Computer Vision 09 : 33~47

[confproc] M. Sun / 2011 / Articulated part-based model for joint object detection and pose estimation / ICCV : 723~730

[confproc] A. Jain / 2012 / Articulated people detection and pose estimation:Reshaping the future / CVPR 06 : 3178~3185

[confproc] U. Iqbal / 2016 / Multi-person pose estimation with local joint-toperson associations 9914

[journal] G. Papandreou / 2017 / Towards accurate multi-person pose estimation in the wild / CVPR : 3711~3719

[confproc] A. Toshev / 2013 / Deeppose: Human pose estimation via deep neural networks / Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

[confproc] W. Ouyang / 2014 / Multi-source deep learning for human pose estimation / CVPR 06 : 2337~2344

[other] J. Tompson / 2014 / Joint training of a convolutional network and a graphical model for human pose estimation / arXiv:1406.2984.

[confproc] A. Newell / 2016 / Stacked hourglass networks for human pose estimation 9912 : 483~499

[other] L. Pishchulin / 2016 / Deepcut: Joint subset partition and labeling for multi person pose estimation / arXiv:1511.06645 : 4929~4937

[other] E. Insafutdinov / 2016 / Deepercut: A deeper, stronger, and faster multi-person pose estimation model / arXiv: 1605.03170

[confproc] K. He / 2015 / Deep residual learning for image recognition / CVPR 7

[journal] C. Twigg / 2008 / L.: Backward steps in rigid body simulation / ACM Trans. Graph 27

[confproc] J. Popovic / 2001 / Interactive manipulation of rigid body simulations / Proceedings of the ACM SIGGRAPH Conference on Computer Graphics

[journal] Raanan Fattal / 2004 / Target-driven smoke animation / ACM Transactions on Graphics / Association for Computing Machinery (ACM) 23(3) : 441~448

[journal] Adrien Treuille / 2003 / Keyframe control of smoke simulations / ACM Transactions on Graphics / Association for Computing Machinery (ACM) 22(3) : 716~723

[confproc] C. Wojtan / 2006 / Keyframe control of complex particle systems using the adjoint method : 15~23

[journal] Jernej Barbič / 2008 / Real-time control of physically based simulations using gentle forces / ACM Transactions on Graphics / Association for Computing Machinery (ACM) 27(5) : 1~10

[confproc] S. Jain / 2009 / Interactive synthesis of human-object interaction : 47~53

[journal] Jong-In Choi / 2015 / Virtual ball player / The Visual Computer / Springer Science and Business Media LLC 31(6-8) : 905~914

[confproc] J. Chemin / 2018 / A physics-based juggling simulation using reinforcement learning / Proceedings of the 11th Annual International Conference on Motion, Interaction, and Games (MIG ’18) : 1~7

[journal] Seokpyo Hong / 2019 / Physics-based full-body soccer motion control for dribbling and shooting / ACM Transactions on Graphics / Association for Computing Machinery (ACM) 38(4) : 1~12

This paper was written with support from the National Research Foundation of Korea.

KJCKorea
Journal Central

Journal of The Korea Society of Computer and Information 2023 KCI Impact Factor : 0.65

Generating Augmented Lifting Player using Pose Tracking

ABSTRACT

KEYWORDS

Citation status

* References for papers published after 2023 are currently being built.

Journal of The Korea Society of Computer and Information 2023 KCI Impact Factor : 0.65

Generating Augmented Lifting Player using Pose Tracking

ABSTRACT

KEYWORDS

Statistics

Tools

Issue List

Citation status

KCI Citation Counts (0)

REFERENCES (30) * References for papers published after 2023 are currently being built.

Search PDF

Citation

* References for papers published after 2023 are currently being built.