A Study on Word Sense Disambiguation Using Bidirectional Recurrent Neural Network for Korean Language

Jihong Min; Joon-Woo Jeon; KwangHo Song; YOO SUNG KIM

Journal of The Korea Society of Computer and Information

Abbr : JKSCI
2017, 22(4), pp.41-49
Publisher : The Korean Society Of Computer And Information
Research Area : Computer Science

Jihong Min ¹, Joon-Woo Jeon ², KwangHo Song ³, YOO SUNG KIM ⁴

¹인하대학교
²인하대학교
³인하대학교
⁴인하대학교

ABSTRACT

Word sense disambiguation(WSD) that determines the exact meaning of homonym which can be used in different meanings even in one form is very important to understand the semantical meaning of text document. Many recent researches on WSD have widely used NNLM(Neural Network Language Model) in which neural network is used to represent a document into vectors and to analyze its semantics. Among the previous WSD researches using NNLM, RNN(Recurrent Neural Network) model has better performance than other models because RNN model can reflect the occurrence order of words in addition to the word appearance information in a document. However, since RNN model uses only the forward order of word occurrences in a document, it is not able to reflect natural language’s characteristics that later words can affect the meanings of the preceding words. In this paper, we propose a WSD scheme using Bidirectional RNN that can reflect not only the forward order but also the backward order of word occurrences in a document. From the experiments, the accuracy of the proposed model is higher than that of previous method using RNN. Hence, it is confirmed that bidirectional order information of word occurrences is useful for WSD in Korean language.
KEYWORDS

Word Sense Disambiguation, Neural Network Language Model, Context Vector, Bidirectional Recurrent Neural Network
REFERENCES (24)

[journal] M. Sundermeyer / 2012 / LSTM Neural Networks for Language Modeling / Interspeech : 194~197

[confproc] mykang, bgkim / 2015 / Word Sense Disambiguation using Word2Vec / Proceeding of 27th Conference on Human & Cognitive Language Technology : 81~84

[confproc] jcshin / 2016 / Homograph Word Sense Disambiguation using Korean Lexical Semantic Map(UWordMap) and Word-Embedding / Proceeding of Korean Computer Congress 2016 : 702~704

[other] D. Yuan / 2016 / Word Sense Disambiguation with Neural Language Models / arXivpreprint arXiv:1603.07012

[confproc] jhmin / 2017 / Study on Word Sense Disambiguation Using Recurrent Neural Network for Korean / Proceeding of Winter Conference on Korean Association of Computer Education 2017 : 93~96

[confproc] jsbae / 2015 / End-to-end Learning of Korean Semantic Role Labeling Using Bidirectional LSTM CRF / Proceeding of 42th Winter Conference on Korean Institute of Information Scientists and Engineers : 566~568

[confproc] C. Irsoy / 2014 / Opinion Mining with Deep Recurrent Neural Networks / Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing : 720~728

[other] H. Sak / 2014 / Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition / arXiv preprint arXiv:1402.1128

[other] Junyoung Chung / 2014 / Empirical evaluation of gated recurrent neural networks on sequence modeling / arXiv preprint arXiv:1412.3555

[journal] 정순호 / 2012 / Inference of Context-Free Grammars using Binary Third-order Recurrent Neural Networks with Genetic Algorithm / 한국컴퓨터정보학회논문지 / 한국컴퓨터정보학회 17(3) : 11~25

[confproc] hmkim / 2016 / Syllable-based Korean POS Tagging using POS Distribution and Bidirectional LSTM CRFs / Proceeding of 28th Conference on Human & Cognitive Language Technology : 3~8

[journal] 안성만 / 2016 / Deep Learning Architectures and Applications / 지능정보연구 / 한국지능정보시스템학회 22(2) : 127~142

[journal] Y. Bengio / 1994 / Learning Long-Term Dependencies with Gradient Descent is Difficult / IEEE Transactions on Neural Networks 5(2)

[journal] 강범모 / 2014 / Text Context and Word Meaning: Latent Semantic Analysis / 언어학 / 사단법인 한국언어학회 (68) : 3~34

[web] / Gensim / https://radimrehurek.com/gensim/models/word2vec.html

[other] T. Mikolov / 2013 / Efficient estimation of word representations in vector space / arXivpreprint arXiv: 1301.3781

[journal] 정명범 / 2015 / Color matching application which can help color blind people based on smart phone / 한국컴퓨터정보학회논문지 / 한국컴퓨터정보학회 20(5) : 65~72

[web] / National Institute of Korean Language / http://ithub.korean.go.kr

[confproc] hgkim / 2007 / 21st Century Sejong Modern Korean Corpora: Results and Expectations / Proceeding of 19th Conference on Human & Cognitive Language Technology : 311~316

[confproc] shchoi / 2016 / On Word Embedding Models and Parameters Optimized for Korean / Proceeding of 28th Conference on Human & Cognitive Language Technology : 252~256

[web] / Korean Wikipedia / https://ko.wikipedia.org/

[web] / Namuwiki / https://namu.wiki/

[web] / SENSEAVAL-2 / http://www.hipposmond.com/senseval2/

[web] / Keras / http://keras.io
KCI Citation (1)

KJC Korea
Journal Central

Statistics

Tools