Improving the Classification of Population and Housing Census with AI: An Industry and Job Code Study (AI를 이용한 인구주택총조사의 자동 분류 개선: 산업 및 직업코드 연구)

Byung-Il Yun (윤병일); Dahye Kim (김다혜); Young-Jin Kim (김영진); Medard Edmund Mswahili (메다드); Young-Seob Jeong (정영섭)

doi:10.9708/jksci.2023.28.04.021

Improving the Classification of Population and Housing Census with AI: An Industry and Job Code Study

Journal of The Korea Society of Computer and Information
Abbr : JKSCI
2023, 28(4), pp.21-29
DOI : 10.9708/jksci.2023.28.04.021
Publisher : The Korean Society Of Computer And Information
Research Area : Engineering > Computer Science
Received : February 14, 2023
Accepted : March 21, 2023
Published : April 28, 2023

Byung-Il Yun ¹, Dahye Kim ¹, Young-Jin Kim ¹, Medard Edmund Mswahili ², Young-Seob Jeong ²

¹(주) 에프에스
²충북대학교

Accredited

ABSTRACT

In this paper, we propose an AI-based system for automatically classifying industry and occupation codes in the population census. The accurate classification of industry and occupation codes is crucial for informing policy decisions, allocating resources, and conducting research. However, this task has traditionally been performed by human coders, which is time-consuming, resource-intensive, and prone to errors. Our system represents a significant improvement over the existing rule-based system used by the statistics agency, which relies on user-entered data for code classification. In this paper, we trained and evaluated several models, and developed an ensemble model that achieved an 86.76% match accuracy in industry and 81.84% in occupation, outperforming the best individual model. Additionally, we propose process improvement work based on the classification probability results of the model. Our proposed method utilizes an ensemble model that combines transfer learning techniques with pre-trained models. In this paper, we demonstrate the potential for AI-based systems to improve the accuracy and efficiency of population census data classification. By automating this process with AI, we can achieve more accurate and consistent results while reducing the workload on agency staff.

KEYWORDS

Natural language process, Classification, Population survey, Ensemble model, Industrial and Occupation code

Citation status

* References for papers published after 2023 are currently being built.

[other] Thompson, Matthew / 2012 / Creating an automated industry and occupation coding process for the American Community Survey

[confproc] A. Bethmann / 2014 / Automatic Coding of Occupations / Proceedings of Statistics Canada Symposium

[confproc] Y. K. Kang / 2001 / Automatic coding system for industry and occupation classification / The Korean Association for Survey Research. Fall Conference 2001 : 33~45

[journal] 임희석 / 2004 / An automated Classification System of Standard Industry and Occupation Codes by Using Information Retrieval Techniques / 컴퓨터교육학회 논문지 / 한국컴퓨터교육학회 7(4) : 51~60

[journal] 임희석 / 2006 / An Example-based Korean Standard Industrial and Occupational Code Classification / 한국산학기술학회논문지 / 한국산학기술학회 7(4) : 594~601

[confproc] C. K. Woo / 2020 / Comparison of Korean Standard Industrial Classification Automatic Classification Model on Deep Learning / Proceedings of the Korea Information Processing Society Conference : 516~518

[journal] 임정우 / 2021 / An Automated Industry and Occupation Coding Systemusing Deep Learning / 한국융합학회논문지 / 한국융합학회 12(4) : 23~30

[confproc] Kim, San / 2021 / A model of cross-lingual knowledge-grounded response generation for open-domain dialogue systems / Findings of the Association for Computational Linguistics: EMNLP 2021

[web] Park, Sungjoon / 2021 / Klue: Korean language understanding evaluation / arXiv / arXiv:2105.09680

[web] SKTBrain / KoBERT / https://github.com/SKTBrain/KoBERT

[web] Kim, Ildoo / 2021 / Kogpt: Kakaobrain korean (hangul) generative pre-trained transformer / https://github.com/kakaobrain/kogpt

[web] SKT-AI / KoGPT2 / https://github.com/SKT-AI/KoGPT2

[web] Ha, Sangchun / 2021 / TUNiB-Electra / https://github.com/tunib-ai/tunib-electra

[web] Lassl / lassl/bert-ko-base / https://huggingface.co/lassl/bert-kobase

[confproc] Lee, Junbum / 2020 / Kcbert: Korean comments bert / Annual Conference on Human and Language Technology / Human and Language Technology

[web] Lee, Junbum / 2021 / KcELECTRA: Korean comments ELECTRA / GitHub repository / https://github.com/Beomi/KcELECTRA

[other] Jangwon Park / 2021 / KoBigBird: Pretrained BigBird Model for Korean

[other] Park, Jangwon / 2020 / KoELECTRA: Pretrained ELECTRA Model for Korean

[journal] Alqershi, Fattoh / A robust consistency model of crowd workers in text labeling tasks / IEEE Access 8(2020) : 168381~168393

[confproc] Draws, Tim / 2022 / The effects of crowd worker biases in fact-checking tasks / 2022 ACM Conference on Fairness, Accountability, and Transparency

[journal] Le, Quoc V / A tutorial on deep learning part 2: Autoencoders, convolutional neural networks and recurrent neural networks / Google Brain 20(2015) : 1~20

[confproc] He, Kaiming / 2016 / Deep residual learning for image recognition / Proceedings of the IEEE conference on computer vision and pattern recognition

This paper was written with support from the National Research Foundation of Korea.

KJCKorea
Journal Central

Journal of The Korea Society of Computer and Information 2023 KCI Impact Factor : 0.65

Improving the Classification of Population and Housing Census with AI: An Industry and Job Code Study

ABSTRACT

KEYWORDS

Citation status

* References for papers published after 2023 are currently being built.

Journal of The Korea Society of Computer and Information 2023 KCI Impact Factor : 0.65

Improving the Classification of Population and Housing Census with AI: An Industry and Job Code Study

ABSTRACT

KEYWORDS

Statistics

Tools

Issue List

Citation status

KCI Citation Counts (0)

REFERENCES (22) * References for papers published after 2023 are currently being built.

Search PDF

Citation

* References for papers published after 2023 are currently being built.