본문 바로가기
  • Home
Basic Search

Evaluating the Impact of Training Conditions on the Performance of GPT-2-Small Based Korean-English Bilingual Models

  • Journal of The Korea Society of Computer and Information
  • Abbr : JKSCI
  • 2024, 29(9), pp.69-77
  • DOI : 10.9708/jksci.2024.29.09.069
  • Publisher : The Korean Society Of Computer And Information
  • Research Area : Engineering > Computer Science
  • Received : August 5, 2024
  • Accepted : September 21, 2024
  • Published : September 30, 2024

Euhee Kim 1,  Keonwoo Koo 2

1신한대학교
2동국대학교

Accredited

ABSTRACT

This study evaluates the performance of second language acquisition models learning Korean and English using the GPT-2-Small model, analyzing the impact of various training conditions on performance. Four training conditions were used: monolingual learning, sequential learning, sequential-interleaved learning, and sequential-EWC learning. The model was trained using datasets from the National Institute of Korean Language and English from BabyLM Challenge, with performance measured through PPL and BLiMP metrics. Results showed that monolingual learning had the best performance with a PPL of 16.2 and BLiMP accuracy of 73.7%. In contrast, sequential-EWC learning had the highest PPL of 41.9 and the lowest BLiMP accuracy of 66.3%(p < 0.05). Monolingual learning proved most effective for optimizing model performance. The EWC regularization in sequential-EWC learning degraded performance by limiting weight updates, hindering new language learning. This research improves understanding of language modeling and contributes to cognitive similarity in AI language learning.

Statistics

  • 84 Viewed
  • 56 Downloaded

Tools

Issue List

Citation status

* References for papers published after 2023 are currently being built.

This paper was written with support from the National Research Foundation of Korea.
Basic Search Advanced Search