An Efficient Deep Learning Ensemble Using a Distribution of Label Embedding (라벨 임베딩 분포를 사용한 효율적인 딥러닝 앙상블 방법)

PARK SAEROM (박새롬)

doi:10.9708/jksci.2021.26.01.027

An Efficient Deep Learning Ensemble Using a Distribution of Label Embedding

Journal of The Korea Society of Computer and Information
Abbr : JKSCI
2021, 26(1), pp.27-35
DOI : 10.9708/jksci.2021.26.01.027
Publisher : The Korean Society Of Computer And Information
Research Area : Engineering > Computer Science
Received : November 4, 2020
Accepted : January 4, 2021
Published : January 29, 2021

PARK SAEROM ¹

¹성신여자대학교

Accredited

ABSTRACT

In this paper, we propose a new stacking ensemble framework for deep learning models which reflects the distribution of label embeddings. Our ensemble framework consists of two phases: training the baseline deep learning classifier, and training the sub-classifiers based on the clustering results of label embeddings. Our framework aims to divide a multi-class classification problem into small sub-problems based on the clustering results. The clustering is conducted on the label embeddings obtained from the weight of the last layer of the baseline classifier. After clustering, sub-classifiers are constructed to classify the sub-classes in each cluster. From the experimental results, we found that the label embeddings well reflect the relationships between classification labels, and our ensemble framework can improve the classification performance on a CIFAR 100 dataset.

KEYWORDS

Deep Ensemble Learning, Clustering, Multi-class classification, Label embedding, Stacking Ensemble Model

Citation status

* References for papers published after 2023 are currently being built.

[journal] Alex Krizhevsky / 2017 / ImageNet classification with deep convolutional neural networks / Communications of the ACM / Association for Computing Machinery (ACM) 60(6) : 84~90

[journal] Sutskever, Ilya / 2014 / Sequence to sequence learning with neural networks / Advances in neural information processing systems 27(1) : 3104~3112

[book] Friedman, Jerome / 2001 / The elements of statistical learning / Springer

[other] Malinin, Andrey / 2019 / Ensemble distribution distillation / arXiv preprint arXiv:1905.00076, 2019

[journal] Hyungjin Ko / 2019 / Loss-Driven Adversarial Ensemble Deep Learning for On-Line Time Series Analysis / Sustainability / MDPI AG 11(12) : 3489~

[journal] M. Paz Sesmero / 2020 / CCE: An ensemble architecture based on coupled ANN for solving multiclass problems / Information Fusion / Elsevier BV 58 : 132~152

[journal] S. Park / 2017 / Inductive ensemble clustering using kernel support matching / Electronics Letters / Institution of Engineering and Technology (IET) 53(25) : 1625~1626

[other] Sun, Xu / 2017 / Label embedding network: Learning label representation for soft training of deep networks / arXiv preprint arXiv:1710.10393

[journal] Sellah, Smail / 2020 / Label Clustering for a Novel Problem Transformation in Multi-label Classification / J. UCS 26(1) : 71~88

[book] Tao, Sean / 2019 / Internatio nal Conference on Machine Learning, Optimization, and Data Science / Springer

[journal] Cheng-Yaw Low / 2020 / Stacking-Based Deep Neural Network: Deep Analytic Network for Pattern Classification / IEEE Transactions on Cybernetics / Institute of Electrical and Electronics Engineers (IEEE) 50(12) : 5021~5034

[confproc] Liu, J. Z. / 2020 / Simple and principled uncertainty estimation with deterministic deep learning via distance awareness / Advances in Neural Information Processing Systems

[other] Abdar, M / 2020 / Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges / arXiv preprint arXiv:2011.06225

This paper was written with support from the National Research Foundation of Korea.

KJCKorea
Journal Central

Journal of The Korea Society of Computer and Information 2023 KCI Impact Factor : 0.65

An Efficient Deep Learning Ensemble Using a Distribution of Label Embedding

ABSTRACT

KEYWORDS

Citation status

* References for papers published after 2023 are currently being built.

Journal of The Korea Society of Computer and Information 2023 KCI Impact Factor : 0.65

An Efficient Deep Learning Ensemble Using a Distribution of Label Embedding

ABSTRACT

KEYWORDS

Statistics

Tools

Issue List

Citation status

KCI Citation Counts (0)

REFERENCES (14) * References for papers published after 2023 are currently being built.

Search PDF

Citation

* References for papers published after 2023 are currently being built.