Towards the Generation of Language-based Sound Summaries Using Electroencephalogram Measurements (뇌파측정기술을 활용한 언어 기반 사운드 요약의 생성 방안 연구)

Kim, Hyun Hee (김현희); Yong-Ho Kim (김용호)

doi:10.3743/KOSIM.2019.36.3.131

Towards the Generation of Language-based Sound Summaries Using Electroencephalogram Measurements

Journal of the Korean Society for Information Management
Abbr : JKOSIM
2019, 36(3), pp.131~148
DOI : 10.3743/KOSIM.2019.36.3.131
Publisher : 한국정보관리학회
Research Area : Interdisciplinary Studies > Library and Information Science
Received : August 19, 2019
Accepted : September 24, 2019
Published : September 30, 2019

Kim, Hyun Hee ¹, Yong-Ho Kim ²

¹명지대학교
²부경대학교

Accredited

ABSTRACT

This study constructed a cognitive model of information processing to understand the topic of a sound material and its characteristics. It then proposed methods to generate sound summaries, by incorporating anterior-posterior N400/P600 components of event-related potential (ERP) response, into the language representation of the cognitive model of information processing. For this end, research hypotheses were established and verified them through ERP experiments, finding that P600 is crucial in screening topic-relevant shots from topic-irrelevant shots. The results of this study can be applied to the design of classification algorithm, which can then be used to generate the content-based metadata, such as generic or personalized sound summaries and video skims.

KEYWORDS

cognitive model, event-related potential response, anterior-posterior N400, anterior-posterior P600, sound summaries, video skims, artificial neural networks, content-based metadata

Citation status

* References for papers published after 2023 are currently being built.

[journal] Huda Omar Alwehaibi / 2015 / The Impact Of Using YouTube In EFL Classroom On Enhancing EFL Students' Content Learning / Journal of College Teaching & Learning (TLC) / Clute Institute 12(2) : 121~126

[book] Baddeley, A. / 2007 / Working memory, thought, and action / Oxford University Press

[journal] Juliane Burmester / 2014 / Context updating during sentence comprehension: The effect of aboutness topic / Brain and Language / Elsevier BV 137 : 62~76

[journal] Buzzetto-More, N. / 2015 / Student attitudes towards the integration of youtube in online, hybrid, and web-assisted courses: An examination of the impact of course modality on perception / MERLOT Journal of Online Learning and Teaching 11 : 55~73

[journal] Katherine A. DeLong / 2014 / Predictability, plausibility, and two late ERP positivities during written sentence comprehension / Neuropsychologia / Elsevier BV 61 : 150~162

[journal] W. James Evans / 1995 / Olfactory event-related potentials in normal human subjects: effects of age and gender / Electroencephalography and Clinical Neurophysiology / Elsevier BV 95(4) : 293~301

[journal] Geyer, A. / 2006 / Plausibility and sentence comprehension. An ERP study / Cognitive Neuroscience Supplement 1(1)

[book] Hakoda, Y. / 2014 / 인지심리학 / 한국뇌기반교육연구소

[journal] Weiming Hu / 2011 / A Survey on Visual Content-Based Video Indexing and Retrieval / IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) / Institute of Electrical and Electronics Engineers (IEEE) 41(6) : 797~819

[journal] Albert Kim / 2005 / The independence of combinatory semantic processing: Evidence from event-related potentials / Journal of Memory and Language / Elsevier BV 52(2) : 205~225

[journal] Hyun Hee Kim / 2016 / Generic speech summarization of transcribed lecture videos: Using tags and their semantic relations / Journal of the Association for Information Science and Technology / Wiley 67(2) : 366~379

[journal] Hyun Hee Kim / 2019 / Video summarization using event-related potential responses to shot boundaries in real-time video watching / Journal of the Association for Information Science and Technology / Wiley 70(2) : 164~175

[journal] Hyun Hee Kim / 2019 / ERP/MMR algorithm for classifying topic-relevant and topic-irrelevant visual shots of documentary videos / Journal of the Association for Information Science and Technology / Wiley 70(9) : 931~941

[journal] Marta Kutas / 2011 / Thirty Years and Counting: Finding Meaning in the N400 Component of the Event-Related Brain Potential (ERP) / Annual Review of Psychology / Annual Reviews 62(1) : 621~647

[book] Luck, S. J. / 2014 / An introduction to the event-related potential technique / MIT Press

[confproc] Maskey, S. / 2006 / Summarizing speech without text using hidden markov models / Proceedings of the Human Language Technology Conference of the NAACL / Association for Computational Linguistics : 89~92

[book] Martin, D. / 2018 / YouTube: The ultimate 2018 guide to grow your youtube channel, make money fast with proven techniques and foolproof step by step strategies / CreateSpace Independent Publishing Platform

[book] Mayer, R. E. / 2005 / The Cambridge handbook of multimedia learning / Cambridge University Press : 134~146

[journal] Jinyoung Moon / 2019 / Detecting user attention to video segments using interval EEG features / Expert Systems with Applications / Elsevier BV 115 : 578~592

[journal] Hiroko Nakano / 2014 / Electrophysiological response to omitted stimulus in sentence processing / NeuroReport / Ovid Technologies (Wolters Kluwer Health) 25(14) : 1169~1174

[journal] Mante S. Nieuwland / 2012 / If the real world were irrelevant, so to speak: The role of propositional truth-value in counterfactual sentence comprehension / Cognition / Elsevier BV 122(1) : 102~109

[journal] Jos J. A. van Berkum / 1999 / Semantic Integration in Sentences and Discourse: Evidence from the N400 / Journal of Cognitive Neuroscience / MIT Press - Journals 11(6) : 657~671

[journal] Luming Wang / 2013 / New is not always costly: evidence from online processing of topic and contrast in Japanese / Frontiers in Psychology / Frontiers Media SA 4

[journal] Stephen M. Wilson / 2018 / Convergence of spoken and written language processing in the superior temporal sulcus / NeuroImage / Elsevier BV 171 : 62~74

[journal] Justin Jian Zhang / 2012 / Active learning with semi-automatic annotation for extractive speech summarization / ACM Transactions on Speech and Language Processing / Association for Computing Machinery (ACM) 8(4) : 1~25

This paper was written with support from the National Research Foundation of Korea.

KJCKorea
Journal Central

Journal of the Korean Society for Information Management 2023 KCI Impact Factor : 1.31

Towards the Generation of Language-based Sound Summaries Using Electroencephalogram Measurements

ABSTRACT

KEYWORDS

Citation status

* References for papers published after 2023 are currently being built.

Journal of the Korean Society for Information Management 2023 KCI Impact Factor : 1.31

Towards the Generation of Language-based Sound Summaries Using Electroencephalogram Measurements

ABSTRACT

KEYWORDS

Statistics

Tools

Issue List

Citation status

KCI Citation Counts (1)

REFERENCES (25) * References for papers published after 2023 are currently being built.

Search PDF

Citation

* References for papers published after 2023 are currently being built.