Real-time Object Recognition and Translation App for Foreign Workers Using Artificial Intelligence (인공지능을 이용한 외국인 근로자를 위한 실시간 사물 인식 및 번역 앱)

Bum-Soo Kim (김범수); Young-Hyen Kim (김영현); Byung-Tae Chun (전병태)

doi:10.34163/jkits.2020.15.6.005

Real-time Object Recognition and Translation App for Foreign Workers Using Artificial Intelligence

Journal of Knowledge Information Technology and Systems
Abbr : JKITS
2020, 15(6), pp.963-971
DOI : 10.34163/jkits.2020.15.6.005
Publisher : Korea Knowledge Information Technology Society
Research Area : Interdisciplinary Studies > Interdisciplinary Research
Received : November 10, 2020
Accepted : December 11, 2020
Published : December 31, 2020

Bum-Soo Kim ¹, Young-Hyen Kim ¹, Byung-Tae Chun ¹

¹한경대학교

Accredited

ABSTRACT

The number of foreign workers entering Korea is increasing every year. Accordingly, Korean language education and education on understanding Korean culture are provided at training institutions of government departments. Despite these efforts, it can be seen that foreign workers face many difficulties when they first encounter Korean culture. In this paper, we introduce a public app developed to solve the cultural difficulties of foreign workers. Among the popular objects in Korea, a study was conducted on objects that foreign workers feel somewhat unfamiliar to them. We developed an app that recognizes an object in real time on a smartphone and translates its usage into several languages. Language selection was based on the languages of five countries with a large portion of entry into Korea. 7,900 copies of training images were collected and labeled. A total of 20,000 iterations were trained with the Yolov3-tiny learning model using Darknet, an AI neural network. The developed app can recognize and translate real-time objects on a smartphone with an average accuracy of 91.3%. It is a system that translates and provides the description of the detected object into five languages at the touch of a button. It is expected that this app will be able to bridge cultural and linguistic differences in daily life of foreign workers.

KEYWORDS

Yolov3-tiny training model, Darknet-53 AI neural networks, Android smartphones, Translation data, Real-time detection, OpenCV, Image dataset.

Citation status

* References for papers published after 2023 are currently being built.

[thesis] J-S. Han / 2018 / Koreans and korean culture viewed by foreign workers from cambodia / master / Seoul National University of Education Graduate School of Education

[thesis] J-D. Park / 2013 / A study on the actual conditions of foreign workers in livestock industry / master / Kon kuk University

[thesis] Y-H / 2019 / Effect of foreign workers’domestic travel motivation, attitude, and sociocultural adaptation on behavioral intention : applying leisure restriction negotiation theory / Doctor / Kyung Hee University

[thesis] J-Y. Ko / 2020 / A study on the life and adaptation of foreign workers-focusing on foreign workers engaged in fishery in Jeju island- / master / Gyeongin National University of Education

[journal] 김범준 / 2018 / The Impact of Acculturation Stress on Quality of Life among Migrant Workers / 한국케어매니지먼트 연구 / 한국통합사례관리학회 (28) : 57~90

[journal] 정현숙 / 2004 / A Qualitative Exploratory Study on Intercultural Communication Confict : A Case Analysis of Lived Experiences of Foreign Factory Workers in Korea / 커뮤니케이션학 연구 / 한국커뮤니케이션학회 12(3) : 27~45

[journal] 이순희 / 2009 / Social Support and Acculturative Stress in Migrant Workers / Journal of Korean Academy of Nursing / 한국간호과학회 39(6) : 899~910

[web] / 2020 / National Folk Museum of Korea / youtube channel / https://www.youtube.com/channel/UCUN2CadW_WjICFZfyNfqGXg/videos

[journal] 이동훈 / 2019 / A Study on the Analysis of Jeju Island Precipitation Patterns using the Convolution Neural Network / 한국소프트웨어감정평가학회 논문지 / 한국소프트웨어감정평가학회 15(2) : 59~66

[journal] 강현선 / 2019 / A Study on the Establishment of the IDS Using Machine Learning / 한국소프트웨어감정평가학회 논문지 / 한국소프트웨어감정평가학회 15(2) : 121~128

[other] J. Redmon / 2018 / Yolov3: An incremental improvement / : 1~

[web] Daddy Makers / 2020 / Data labelling based on yolov3 deep learning / http://daddynkidsmakers.blogspot.com/2020/05/yolo.html

[confproc] J. Redmon / 2016 / You only look once: unified, real-time object detection / IEEE Conference on Computer Vision and Pattern Recognition(CVPR) : 779~788

[journal] X. Zhang / 2019 / Real-time object detection algorithm based on improved YOLOv3 / Journal of Electronic Imaging 28(5) : 53022~

[journal] Y. Zhang / 2019 / An imporved tiny-yolov3 predestrian detection algorithm / Optik 183(3) : 17~23

[web] / 2020 / Galaxy S7 Smartphone(SM-G930S) / SAMSUNG / https://www.samsung.com/sec/support/model/SM-G930SZWASKOOct

[journal] Kari Pulli / 2012 / Anatoly Baksheev, Kirill Kornyakov, and Victor Eruhimov, An imporved Real-time computer vision with OpenCV / Communications of the ACM 55(6) : 61~

[web] Ivangrov / 2020 / Anroid-Deep_Learning-with-OpenCV / https://github.com/ivangrov/Android-Deep-Learning-with-OpenCV

KJCKorea
Journal Central

Journal of Knowledge Information Technology and Systems 2023 KCI Impact Factor : 0.89

Real-time Object Recognition and Translation App for Foreign Workers Using Artificial Intelligence

ABSTRACT

KEYWORDS

Citation status

* References for papers published after 2023 are currently being built.

Journal of Knowledge Information Technology and Systems 2023 KCI Impact Factor : 0.89

Real-time Object Recognition and Translation App for Foreign Workers Using Artificial Intelligence

ABSTRACT

KEYWORDS

Statistics

Tools

Issue List

Citation status

KCI Citation Counts (2)

REFERENCES (18) * References for papers published after 2023 are currently being built.

Search PDF

Citation

* References for papers published after 2023 are currently being built.