@article{ART003169015},
author={Gukbeom Yoon and Rhee,Jung-Soo},
title={Minimizing the Loss Values in the CBOW Models},
journal={Journal of The Korea Society of Computer and Information},
issn={1598-849X},
year={2025},
volume={30},
number={1},
pages={65-72}
TY - JOUR
AU - Gukbeom Yoon
AU - Rhee,Jung-Soo
TI - Minimizing the Loss Values in the CBOW Models
JO - Journal of The Korea Society of Computer and Information
PY - 2025
VL - 30
IS - 1
PB - The Korean Society Of Computer And Information
SP - 65
EP - 72
SN - 1598-849X
AB - The Continuous Bag of Words (CBOW) model is a popular technique in natural language processing (NLP) used to generate word Embeddings. It predicts a target word given its surrounding context words.
The model consists of an input layer, a hidden layer, and an output layer. The PTB, which is commonly used as a benchmark for evaluating the performance of the CBOW model, is a medium-sized corpus that plays an important role in natural language processing and computational linguistics. It consists of 2,499 articles extracted from the Wall Street Journal in 1989, containing approximately 1 million words and 49,208 sentences. This paper aims to improve the average loss value of the loss function by applying batch normalization to the CBOW model and training it on the PTB dataset. To achieve the objective of this paper, experiments were conducted in a notebook environment equipped with CuPy, comparing the original CBOW model with the batch normalized CBOW model. The results shows that average loss value decreased from 1.25 to 0.65. Therefore, this paper demonstrates the effectiveness of batch normalization in improving the performance of the CBOW model and is expected to contribute to refining distributional representation of words for transfer learning.
KW - Batch Normalization;CBOW;CNN;CuPy;Embedding;Negative Sampling;Word2Vec
DO -
UR -
ER -
Gukbeom Yoon and Rhee,Jung-Soo. (2025). Minimizing the Loss Values in the CBOW Models. Journal of The Korea Society of Computer and Information, 30(1), 65-72.
Gukbeom Yoon and Rhee,Jung-Soo. 2025, "Minimizing the Loss Values in the CBOW Models", Journal of The Korea Society of Computer and Information, vol.30, no.1 pp.65-72.
Gukbeom Yoon, Rhee,Jung-Soo "Minimizing the Loss Values in the CBOW Models" Journal of The Korea Society of Computer and Information 30.1 pp.65-72 (2025) : 65.
Gukbeom Yoon, Rhee,Jung-Soo. Minimizing the Loss Values in the CBOW Models. 2025; 30(1), 65-72.
Gukbeom Yoon and Rhee,Jung-Soo. "Minimizing the Loss Values in the CBOW Models" Journal of The Korea Society of Computer and Information 30, no.1 (2025) : 65-72.
Gukbeom Yoon; Rhee,Jung-Soo. Minimizing the Loss Values in the CBOW Models. Journal of The Korea Society of Computer and Information, 30(1), 65-72.
Gukbeom Yoon; Rhee,Jung-Soo. Minimizing the Loss Values in the CBOW Models. Journal of The Korea Society of Computer and Information. 2025; 30(1) 65-72.
Gukbeom Yoon, Rhee,Jung-Soo. Minimizing the Loss Values in the CBOW Models. 2025; 30(1), 65-72.
Gukbeom Yoon and Rhee,Jung-Soo. "Minimizing the Loss Values in the CBOW Models" Journal of The Korea Society of Computer and Information 30, no.1 (2025) : 65-72.
