Time-Frequency Representations for Improving Environmental Sound Classification with Deep Learning Models (딥러닝 모델의 환경음 분류 성능 향상을 위한 시간-주파수 표현)

Moon-Ki Back (백문기); Hyoung-Seop Shim (심형섭)

doi:10.29056/jsav.2024.12.24

Time-Frequency Representations for Improving Environmental Sound Classification with Deep Learning Models

Journal of Software Assessment and Valuation
Abbr : JSAV
2024, 20(4), pp.233~242
DOI : 10.29056/jsav.2024.12.24
Publisher : Korea Software Assessment and Valuation Society
Research Area : Engineering > Computer Science
Received : December 3, 2024
Accepted : December 20, 2024
Published : December 31, 2024

Moon-Ki Back ¹, Hyoung-Seop Shim ¹

¹한국과학기술정보연구원

Accredited

ABSTRACT

Spectrograms are widely utilized in audio signal processing research to effectively analyze the magnitude of frequency components but they are limited in representing time-varying phase information. To overcome this limitation, this paper explores a time-frequency representation that combines Power Spectrogram (PS) and Instantaneous Frequency (IF) features and validates its effectiveness through environmental sound classification tasks using various deep learning architectures. Experiments on the ESC-50 dataset demonstrate that ConvNeXt model, leveraging the vertical integration of PS and IF, achieves a classification accuracy of 87.16%, reflecting a 1.7% improvement over conventional methods. The confusion matrix analysis reveals that misclassifications often occur for water-related sounds and sirens, as they exhibit highly similar time- frequency patterns, making them challenging to distinguish. This study highlights the potential of the proposed approach to enhance the performance of deep learning models in audio-related tasks, particularly for small- to medium-scale datasets and anticipates broad applicability in sound-related applications.

KEYWORDS

Time-Frequency, Environmental Sound, Spectrogram, Instantaneous Frequency, Deep Learning

Citation status

* References for papers published after 2024 are currently being built.

[journal] K. Zaman / 2023 / A Survey of Audio Classification Using Deep Learning / IEEE Access 11 : 106620~106649

[confproc] B. Michael / 2019 / At Your Service : Designing Voice Assistant Personalities to Improve Automotive User Interfaces / Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems : 1~11

[journal] B. J. Zhang / 2023 / Nonverbal Sound in Human-Robot Interaction : A Systematic Review / ACM Transactions on Human-Robot Interaction 12(4) : 1~46

[journal] Y. Yuki / 2020 / Audio-Processing-Based Human Detection at Disaster Sites with Unmanned Aerial Vehicle / IEEE Access 8 : 101398~101405

[journal] G. Ciaburro / 2020 / Improving Smart Cities Safety Using Sound Events Detection Based on Deep Neural Network Algorithms / MDPI Informatics 7(3) : 23~

[confproc] A. Mesaros / 2017 / DCASE 2017 Challenge setup: Tasks, datasets and baseline system / DCASE 2017-workshop on detection and classification of acoustic scenes and events

[confproc] J. F. Gemmeke / 2017 / Audio Set : An ontology and human-labeled dataset for audio events / International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

[confproc] K. J. Piczak / 2015 / ESC : Dataset for Environmental Sound Classification / Proceedings of the 23rd ACM international conference on Multimedia : 1015~1018

[confproc] J. Salamon / 2014 / A Dataset and Taxonomy for Urban Sound Research / Proceedings of the 22nd ACM international conference on Multimedia : 1041~1044

[journal] A. Khamparia / 2019 / Sound Classification Using Convolutional Neural Network and Tensor Deep Stacking Network / IEEE Access 7 : 7717~7727

[journal] F. Demir / 2020 / A new pyramidal concatenated CNN approach for environmental sound classification / Applied Acoustics 170 : 107520~

[journal] W. Mu / 2021 / Environmental sound classification using temporal-frequency attention based convolutional neural network / Scientific Reports 11

[web] A. Dosovitskiy / 2020 / An Image is Worth 16x16 Words : Transformers for Image Recognition at Scale / arXiv / 10.48550/arXiv.2010.11929

[confproc] Y. Zhang / 2022 / Spectrogram Transformers for Audio Classification / 2022 IEEE International Conference on Imaging Systems and Techniques (IST)

[confproc] J. Luo / 2022 / Vision Transformer based Audio Classification using Patch-level Feature Fusion / 2022 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)

[confproc] K. He / 2016 / Deep Residual Learning for Image Recognition / Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) : 770~778

[confproc] Z. Liu / 2022 / A ConvNet for the 2020s / Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) : 11976~11986

[confproc] Z. Liu / 2021 / Swin Transformer : Hierarchical Vision Transformer using Shifted Windows / 2021 IEEE/CVF International Conference on Computer Vision (ICCV) : 10012~10022

[web] M. Oquab / 2023 / DINOv2 : Learning Robust Visual Features without Supervision / arXiv / 10.48550/arXiv.2304.07193

[confproc] J. Deng / 2009 / ImageNet : A large-scale hierarchical image database / 2009 IEEE Conference on Computer Vision and Pattern Recognition : 248~255

[web] I. Loshchilov / 2019 / Decoupled Weight Decay Regularization / arXiv / 10.48550/arXiv.1711.05101

[journal] Z. Zhang / 2019 / Learning Attentive Representations for Environmental Sound Classification / IEEE Access 7 : 130327~130339

KJCKorea
Journal Central

Journal of Software Assessment and Valuation 2024 KCI Impact Factor : 0.32

Time-Frequency Representations for Improving Environmental Sound Classification with Deep Learning Models

ABSTRACT

KEYWORDS

Citation status

* References for papers published after 2024 are currently being built.

Journal of Software Assessment and Valuation 2024 KCI Impact Factor : 0.32

Time-Frequency Representations for Improving Environmental Sound Classification with Deep Learning Models

ABSTRACT

KEYWORDS

Statistics

Tools

Issue List

Citation status

KCI Citation Counts (0)

REFERENCES (22) * References for papers published after 2024 are currently being built.

Search PDF

Citation

* References for papers published after 2024 are currently being built.