A Study of Unsupervised Summarization Using Topic Model (토픽 모델을 이용한 비지도 요약 기법의 연구)

Hyunjin Bae (배현진); Chulyun Kim (김철연)

doi:10.29056/jsav.2023.3.10

A Study of Unsupervised Summarization Using Topic Model

Journal of Software Assessment and Valuation
Abbr : JSAV
2023, 19(1), pp.77-88
DOI : 10.29056/jsav.2023.3.10
Publisher : Korea Software Assessment and Valuation Society
Research Area : Engineering > Computer Science
Received : March 15, 2023
Accepted : March 20, 2023
Published : March 31, 2023

Hyunjin Bae ¹, Chulyun Kim ¹

¹숙명여자대학교

Accredited

ABSTRACT

Automatic Document Summarization is a research field that aims to create short summaries of documents while maintaining their important content. So far, document summarization has relied on supervised deep neural network models trained on large datasets. However, despite increasing demand from industry, there is still a shortage of summarization datasets. Data scarcity problem is not only a challenge for summarization but also for natural language processing in general. To address this, techniques such as Zero-Shot Learning and Self-Supervised Learning have emerged, which aim to create good representations so that models can handle unseen data well. In this paper, we propose a topic model-based unsupervised extractive summarization model called TES(Topic model based Extractive Summarization). Through experiments, we confirm that it performs similarly to existing models while suggesting its advantages over existing models.

KEYWORDS

document summarization, extractive summarization, topic modeling, NLP, Deep Learning

Citation status

* References for papers published after 2023 are currently being built.

[other] Linguistic Data Consortium / 2008 / The New York Times Annotated Corpus / Linguistic Data Consortium

[journal] Hermann, Karl Moritz / 2005 / Teaching machines to read and comprehend / Advances in neural information processing systems 28

[journal] Yangqiu Song / 2019 / Toward any-language zero-shot topic classification of textual documents / Artificial Intelligence / Elsevier BV 274 : 133~150

[confproc] Ben Zhou / 2018 / Zero-Shot Open Entity Typing as Type-Compatible Grounding / Proceedings of the 2018Conference on Empirical Methods in Natural Language Processing : 2065~2076

[confproc] Omer Levy / 2017 / Zero-Shot Relation Extraction via Reading Comprehension / Proceedings of the 21st Conference on Computational Natural Language Learning : 333~342

[confproc] Devlin, Jacob / 2019 / BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding / Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies 1 : 4171~4186

[confproc] Mike Lewis / 2020 / BART:Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension / Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics : 7871~7880

[confproc] Genta Indra Winata / 2021 / Language Models are Few-shot Multilingual Learners / Proceedings of the 1st Workshop on Multilingual Representation Learning : 1~15

[journal] David M. Blei / 2012 / Probabilistic topic models / Communications of the ACM / Association for Computing Machinery (ACM) 55(4) : 77~84

[confproc] Rada Mihalcea / 2004 / TextRank: Bringing Order into Text / Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing : 404~411

[journal] David Blei / 2003 / Latent Dirichlet Allocation / Journal of Machine Learning Research 3 : 993~1022

[confproc] Akash Srivastava / 2017 / Autoencoding Variational Inference For Topic Models / International Conference on Learning Representations

[confproc] Hao Zheng / 2019 / Sentence Centrality Revisited for Unsupervised Summarization / Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics : 6236~6247

[confproc] Ramesh Nallapati / 2017 / SummaRuNNer: a recurrent neural network based sequence model for extractive summarization of documents / Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence : 3075~3081

[confproc] Kyunghyun Cho / 2014 / On the Properties of Neural Machine Translation: Encoder–Decoder Approaches / Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation : 103~111

[confproc] Yang Liu / 2019 / Text Summarization with Pretrained Encoders / Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing : 3730~3740

[confproc] Shusheng Xu / 2020 / Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers / Findings of the Association for Computational Linguistics: EMNLP 2020 : 1784~1795

[journal] Vaswani, Ashish / 2017 / Attention is All you Need / Advances in Neural Information Processing Systems 30 : 03762~

[confproc] Karl Moritz Hermann / 2015 / Teaching machines to read and comprehend / Proceedings of the 28th International Conference on Neural Information Processing Systems : 1693~1701

[confproc] Christopher Manning / 2014 / The Stanford CoreNLP Natural Language Processing Toolkit / Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations

[confproc] See, Abigail / 2017 / Get To The Point:Summarization with Pointer-Generator Networks / Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics 1 : 2073~1083

[web] Lin, Chin-Yew / 2004 / ROUGE: A Package for Automatic Evaluation of Summaries. Text Summarization Branches Out / https://aclanthology.org/W04-1013

[confproc] Guillaume Klein / 2017 / OpenNMT: Open-Source Toolkit for Neural Machine Translation / Proceedings of ACL 2017, System Demonstrations : 67~72

[confproc] Romain Paulus / 2018 / A Deep Reinforced Model for Abstractive Summarization / 6th International Conference on Learning Representations, ICLR 2018

[other] Kingma, Diederik P / 2014 / Adam: A method for stochastic optimization / arXiv preprint arXiv:1412.6980

[confproc] Thomas Wolf / 2020 / Transformers : State-of-the-Art Natural Language Processing / Proceedings of the 2020Conference on Empirical Methods in Natural Language Processing: System Demonstrations : 38~45

[confproc] Shashi Narayan / 2018 / Ranking Sentences for Extractive Summarization with Reinforcement Learning / Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies 1 : 1747~1759

KJCKorea
Journal Central

Journal of Software Assessment and Valuation 2023 KCI Impact Factor : 0.35

A Study of Unsupervised Summarization Using Topic Model

ABSTRACT

KEYWORDS

Citation status

* References for papers published after 2023 are currently being built.

Journal of Software Assessment and Valuation 2023 KCI Impact Factor : 0.35

A Study of Unsupervised Summarization Using Topic Model

ABSTRACT

KEYWORDS

Statistics

Tools

Issue List

Citation status

KCI Citation Counts (0)

REFERENCES (27) * References for papers published after 2023 are currently being built.

Search PDF

Citation

* References for papers published after 2023 are currently being built.