Journal of the Korean Society for Library and Information Science 2023 KCI Impact Factor : 0.84
-
pISSN : 1225-598X / eISSN : 2982-6292
- https://journal.kci.go.kr/kslis
KJCKorea
Journal Central
A Comparative Research on End-to-End Clinical Entity and Relation Extraction using Deep Neural Networks: Pipeline vs. Joint Models
- Journal of the Korean Society for Library and Information Science
- 2023, 57(1), pp.93-114
- DOI : 10.4275/KSLIS.2023.57.1.093
- Publisher : 한국문헌정보학회
- Research Area : Interdisciplinary Studies > Library and Information Science
- Received : January 17, 2023
- Accepted : February 21, 2023
- Published : February 28, 2023
1경기대학교
Excellent Accredited
ABSTRACT
Information extraction can facilitate the intensive analysis of documents by providing semantic triples which consist of named entities and their relations recognized in the texts. However, most of the research so far has been carried out separately for named entity recognition and relation extraction as individual studies, and as a result, the effective performance evaluation of the entire information extraction systems was not performed properly. This paper introduces two models of end-to-end information extraction that can extract various entity names in clinical records and their relationships in the form of semantic triples, namely pipeline and joint models and compares their performances in depth. The pipeline model consists of an entity recognition sub-system based on bidirectional GRU-CRFs and a relation extraction module using multiple encoding scheme, whereas the joint model was implemented with a single bidirectional GRU-CRFs equipped with multi-head labeling method. In the experiments using i2b2/VA 2010, the performance of the pipeline model was 5.5% (F-measure) higher. In addition, through a comparative experiment with existing state-of-the-art systems using large-scale neural language models and manually constructed features, the objective performance level of the end-to-end models implemented in this paper could be identified properly.
Statistics
-
251 Viewed
-
170 Downloaded
Tools
Issue List
Citation status
* References for papers published after 2023 are currently being built.
[other]
Bekoulis, G.
/ 2018
/ Adversarial training for multi-context joint entity and relation extraction
/ ArXiv:1808.06876
[journal]
Bekoulis, G.
/ 2018
/ Joint entity recognition and relation extraction as a multi-head selection problem
/ Expert Systems with Applications
114
: 34~45
[report]
Bhattacharyya, P.
/ 2017
/ End-to-end Relation Extraction using Neural Networks and Markov Logic Networks
/ EACL
[journal]
Chiu, J. P.
/ 2016
/ Named entity recognition with bidirectional LSTM-CNNs
/ Transactions of the Association for Computational Linguistics
4
: 357~370
[other]
Cho, K.
/ 2014
/ On the properties of neural machine translation: encoder-decoder approaches
/ rXiv:1409.1259 [Cs, Stat]
[journal]
Choi, S. -P.
/ 2014
/ An intensive case study on kernel-based relation extraction
/ Multimedia Tools and Applications
71(2)
: 741~767
[other]
Chung, J.
/ 2014
/ Empirical evaluation of gated recurrent neural networks on sequence modeling
/ ArXiv:1412.3555 [Cs]
[journal]
Collobert, R.
/ 2011
/ Natural language processing(almost)from scratch
/ Journal of Machine Learning Research
12
: 2493~2537
[journal]
Dang, T. H.
/ 2018
/ D3NER: Biomedical named entity recognition using CRF-biLSTM improved with fine-tuned embeddings of various linguistic information
/ Bioinformatics (Oxford, England)
34(20)
: 3539~3546
[other]
Dozat, T.
/ 2016
/ Deep biaffine attention for neural dependency parsing
/ ArXiv:1611.01734 [Cs]
[journal]
Du, C.
/ 2018
/ Text classification research with attention-based recurrent neural networks
/ International Journal of Computers Communications & Control
13(1)
: 50~61
[confproc]
Fang, X.
/ 2021
/ Joint extraction of clinical entities and relations using multi-head selection method
/ 2021 International Conference on Asian Language Processing (IALP)
: 99~104
[other]
Giorgi, J.
/ 2019
/ End-to-end named entity recognition and relation extraction using pre-trained language models
/ ArXiv:1912.13415 [Cs]
[confproc]
Gupta, P
/ 2016
/ Table filling multi-task recurrent neural network for joint entity and relation extraction
/ Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers
: 2537~2547
[other]
Huang, Z.
/ 2015
/ Bidirectional LSTM-CRF models for sequence tagging
/ ArXiv:1508.01991 [Cs]
[confproc]
Ji, G.
/ 2017
/ Distant supervision for relation extraction with sentence-level attention and entity descriptions
/ Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence
: 3060~3066
[confproc]
Ji, Z.
/ 2021
/ A discrete joint model for entity and relation extraction from clinical notes
/ AMIA Summits on Translational Science Proceedings
: 315~324
[confproc]
Kate, R. J.
/ 2010
/ Joint entity and relation extraction using card-pyramid parsing
/ Proceedings of the Fourteenth Conference on Computational Natural Language Learning
: 203~212
[journal]
김선우
/ 2017
/
A Comparative Study on Deep Learning Topology for Event Extraction from Biomedical Literature
/ 한국문헌정보학회지
/ 한국문헌정보학회
51(4)
: 77~97
[other]
Lample, G.
/ 2016
/ Neural architectures for named entity recognition
/ ArXiv:1603.01360 [Cs]
[journal]
Li, F.
/ 2017
/ A neural joint model for entity and relation extraction from biomedical text
/ BMC Bioinformatics
18(1)
: 198~
[confproc]
Li, Q.
/ 2014
/ Incremental joint extraction of entity mentions and relations
/ Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics
: 402~412
[journal]
Li, Z.
/ 2019
/ Integrating shortest dependency path and sentence sequence into a deep learning framework for relation extraction in clinical text
/ BMC Medical Informatics and Decision Making
19(1)
: 22~
[report]
Manning, C. D.
/ 2014
/ he Stanford CoreNLP natural language processing toolkit
/ Association for Computational Linguistics (ACL) System Demonstrations
: 55~60
[confproc]
Misawa, S.
/ 2017
/ Character-based Bidirectional LSTM-CRF with words and characters for Japanese Named Entity Recognition
/ Proceedings of the First Workshop on Subword and Character Level Models in NLP
: 97~102
[other]
Miwa, M.
/ 2016
/ End-to-end relation extraction using LSTMs on sequences and tree structures
/ ArXiv Preprint ArXiv:1601.00770
[confproc]
Miwa, M.
/ 2014
/ Modeling joint entity and relation extraction with table representation
/ Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)
: 1858~1869
[confproc]
Mollá, D.
/ 2006
/ Named entity recognition for question answering
/ Proceedings of the 2006 Australasian Language Technology Workshop
: 51~58
[other]
Nguyen, D. Q.
/ 2019
/ End-to-end neural relation extraction using deep biaffine attention
/ ArXiv:1812.11275 [Cs]
: 729~738
[journal]
Nguyen, N. T.
/ 2015
/ Wide-coverage relation extraction from MEDLINE using deep syntax
/ BMC Bioinformatics
16(1)
: 107~
[confproc]
Nguyen, T. H.
/ 2015
/ Relation Extraction: Perspective from Convolutional Neural Networks
/ Proceedings of NAACL Workshop on Vector Space Modeling for NLP
[journal]
Unanue, I. J.
/ 2017
/ Recurrent neural networks with specialized word embeddings for health-domain named-entity recognition
/ Journal of Biomedical Informatics
76
: 102~109
[other]
Wu, Y.
/ 2016
/ Google’s neural machine translation system : bridging the gap between human and machine translation
/ ArXiv:1609.08144 [Cs]
[confproc]
Yang, Z.
/ 2016
/ Hierarchical attention networks for document classification
/ Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
: 1480~1489
[confproc]
Zhang, J.
/ 2017
/ ME-MD: An effective framework for neural machine translation with multiple encoders and decoders
/ Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17
: 3392~3398
[journal]
Meishan Zhang
/ 2017
/ End-to-End Neural Relation Extraction with Global Optimization
/ Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
/ Association for Computational Linguistics
[journal]
Zheng, S.
/ 2017
/ Joint entity and relation extraction based on a hybrid neural network
/ Neurocomputing
257
: 59~66
[other]
Zheng, S.
/ 2017
/ Joint extraction of entities and relations based on a novel tagging scheme
/ ArXiv:1706.05075 [Cs]