An Experimental Study on Feature Selection Using Wikipedia for Text Categorization (위키피디아를 이용한 분류자질 선정에 관한 연구)

KIM, YONG HWAN (김용환); Young-Mee Chung (정영미)

doi:10.3743/KOSIM.2012.29.2.155

An Experimental Study on Feature Selection Using Wikipedia for Text Categorization

Journal of the Korean Society for Information Management
Abbr : JKOSIM
2012, 29(2), pp.155~171
DOI : 10.3743/KOSIM.2012.29.2.155
Publisher : 한국정보관리학회
Research Area : Interdisciplinary Studies > Library and Information Science
Received : May 21, 2012
Accepted : June 16, 2012
Published : June 30, 2012

KIM, YONG HWAN ¹, Young-Mee Chung ¹

¹연세대학교

Accredited

ABSTRACT

In text categorization, core terms of an input document are hardly selected as classification features if they do not occur in a training document set. Besides, synonymous terms with the same concept are usually treated as different features. This study aims to improve text categorization performance by integrating synonyms into a single feature and by replacing input terms not in the training document set with the most similar term occurring in training documents using Wikipedia. For the selection of classification features, experiments were performed in various settings composed of three different conditions: the use of category information of non-training terms, the part of Wikipedia used for measuring term-term similarity, and the type of similarity measures. The categorization performance of a kNN classifier was improved by 0.35~1.85% in F1 value in all the experimental settings when non-learning terms were replaced by the learning term with the highest similarity above the threshold value. Although the improvement ratio is not as high as expected, several semantic as well as structural devices of Wikipedia could be used for selecting more effective classification features.

KEYWORDS

text categorization, document classification, feature selection, Wikipedia, term similarity, non-learning term

Citation status

* References for papers published after 2023 are currently being built.

[book] Bird, S. / 2007 / Natural language processing in Python / O'ReillyMedia

[confproc] Gabrilovich, E. / 2005 / Feature generation for text categorization using world knowledge / Proceedings of the 19th international Joint Conference on Artificial intelligence : 1048~1053

[confproc] Gabrilovich, E. / 2006 / Overcoming the brittleness bottleneck using Wikipedia : enhancing text categorization with encyclopedic knowledge / Proceedings of the 21st National Conference on Artificial Intelligence : 1301~1306

[confproc] Gabrilovich, E. / 2007 / Computing semantic relatedness using Wikipedia-based explicit semantic analysis / Proceedings of the 20th International Joint Conference on Artificial Intelligence : 1606~1611

[confproc] Huang, A. / 2009 / Clustering documents using a Wikipediabased concept representation / Proceedings of the 13th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining (LNCS 5476/2009) : 628~636

[confproc] Milne, D. / 2007 / A knowledge-based search engine powered by Wikipedia / Proceedings of the 16th ACM Conference on Information and Knowledge Management : 445~454

[confproc] Milne, D. / 2008 / An effective, low-cost measure of semantic relatedness obtained from Wikipedia links / Proceedings of the First AAAI Workshop on Wikipedia and Artificial Intelligence, (WIKIAI 2008)

[confproc] Minier, Z. / 2007 / Wikipedia-based kernels for text categorization / Proceedings of the International Symposium on Symbolic and Numeric Algorithms for Scientific Computing : 157~164

[confproc] Ponzetto, S. P. / 2006 / Exploiting semantic role labeling, WordNet and Wikipedia for coreference resolution / Proceedings of the Main Conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics : 192~199

[journal] Ponzetto, S. P. / 2007 / Knowledge derived from Wikipedia for computing semantic relatedness / Journal of Artificial Intelligence Research 30(1) : 181~212

[confproc] Strube, M. / 2006 / WikiRelate! Computing semantic relatedness using Wikipedia / Proceedings of the 21st National Conference on Artificial Intelligence : 1419~1424

[confproc] Wang, P. / 2007 / Improving text classification by using encyclopedia knowledge / Proceedings of the 2007 Seventh IEEE International Conference on Data Mining : 332~341

[confproc] Wang, P. / 2008 / Building semantic kernels for text classification using wikipedia / Proceeding of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining : 713~721

KJCKorea
Journal Central

Journal of the Korean Society for Information Management 2023 KCI Impact Factor : 1.31

An Experimental Study on Feature Selection Using Wikipedia for Text Categorization

ABSTRACT

KEYWORDS

Citation status

* References for papers published after 2023 are currently being built.

Journal of the Korean Society for Information Management 2023 KCI Impact Factor : 1.31

An Experimental Study on Feature Selection Using Wikipedia for Text Categorization

ABSTRACT

KEYWORDS

Statistics

Tools

Issue List

Citation status

KCI Citation Counts (6)

REFERENCES (13) * References for papers published after 2023 are currently being built.

Search PDF

Citation

* References for papers published after 2023 are currently being built.