Real-time Lip Region Detection for Lipreadingin Mobile Device (모바일 장치에서의 립리딩을 위한 실시간 입술 영역 검출)

김영운; Kang Sun-Kyung (강선경); Jung， Sung-Tae (정성태)

Real-time Lip Region Detection for Lipreadingin Mobile Device

Journal of The Korea Society of Computer and Information
Abbr : JKSCI
2009, 14(4), pp.39-46
Publisher : The Korean Society Of Computer And Information
Research Area : Engineering > Computer Science

김영운 ¹, Kang Sun-Kyung ¹, Jung， Sung-Tae ¹

¹원광대학교

Accredited

ABSTRACT

Many lip region detection methods have been developed in PC environment. But the existing methods are difficult to run on real-time in resource limited mobile devices. To solve the problem, this paper proposes a real-time lip region detection method for lipreading in Mobile device. It detects face region by using adaptive face color information. After that, it detects lip region by using geometrical relation between eyes and lips. The proposed method is implemented in a smart phone with Intel PXA 270 embedded processor and 386MB memory. Experimental results show that the proposed method runs at the speed 9.5 frame/sec and the correct detection rate was 98.8% for 574 images.

KEYWORDS

mobile device, lip detection, lip reading

Citation status

* References for papers published after 2023 are currently being built.

[confproc] E. D. Petajan / 1984 / Automatic lipreading to enhance speech recognition / Proc. of Global Telecommunications Conf. : 265~272

[journal] C. C. Chibelushi / 2002 / A review of speech based bimodal recognition / IEEE Trans. Multimedia 4(1) : 23~37

[journal] G. Potamianos / 2003 / Recent advances in the automatic recognition of audiovisual speech / Proceedings of the IEEE 91(9) : 1306~1326

[journal] J. S. Lee / 2008 / Robust Audio-Visual Speech Recognition Based on Late Integration / IEEE Trans. on Multimedia 10(5) : 767~779

[confproc] T. Wark / 1998 / An approach to statistical lip modelling for speaker identification via chromatic feature extraction / Proc. of Fourteenth International Conference on Pattern Recognition : 123~125

[confproc] P. Delmas / 2002 / Towards robust lip tracking / Proc. of 16th International Conference on Pattern Recognition : 528~531

[journal] S. Dupont / 2000 / Audio-visual speech modeling for continuous speech recognition / IEEE Trans. on Multimedia 2(3) : 141~151

[confproc] C. Bregler / 1994 / Eigenlips for robust speech recognition / Proc. IEEE ICASSP : 669~672

[confproc] P. Duchnowski / 1994 / See me, hear me: Integrating automatic speech recognition and lip-reading / Proc. ICSLP : 547~550

[confproc] G. Potamianos / 2000 / A cascade image transform for speaker independent automatic speechreading / Proc. 1st IEEE Int. Conf. Multimedia and Expo. : 1097~1100

[journal] 이지근 / 2004 / Design and Implementation of a Real-Time Lipreading System Using PCA & HMM / 멀티미디어학회논문지 / 한국멀티미디어학회 7(11) : 12~12

[confproc] J. M. Zhang / 2003 / Research and implementation of a real time approach to lip detection in video sequences / International Conference on Machine Learning and Cybernetics : 2795~2799

[confproc] K. Saenko / 2005 / Visual speech recognition with loosely synchronized feature streams / Proc. of IEEE International Conference on Computer Vision : 1424~1431

[confproc] P.Viola / 2001 / Rapid Object Detection using a Boosted Cascade of Simple Features / Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition : 511~518

[journal] 정중교 / 2005 / Real-Time face detection using the Skin color and Haar-like feature / 한국컴퓨터정보학회논문지 / 한국컴퓨터정보학회 10(4) : 113~121

[journal] 김형균 / 2008 / Real-time Face Detection System using YCbCr Information and AdaBoost Algorithm / 한국컴퓨터정보학회논문지 / 한국컴퓨터정보학회 13(5) : 19~26

[confproc] G.R. Bradski / 1998 / Real Time Face and Object Tracking as a Component of a Perceptual User Interface / Proc. of the 4th IEEE Workshop on Applications of Computer Vision : 214~219

[confproc] C. P. Papageorgiou / 1998 / A general framework for object detection / Proc. of Sixth International Conference on Computer Vision : 555~562

KJCKorea
Journal Central

Journal of The Korea Society of Computer and Information 2023 KCI Impact Factor : 0.65