뇌파정보를 활용한 영상물 요약 알고리즘 설계와 평가 (Design and Evaluation of Video Summarization Algorithm based on EEG Information)

김현희 (Kim, Hyun Hee); 김용호 (Yong-Ho Kim)

doi:10.4275/KSLIS.2018.52.4.091

Design and Evaluation of Video Summarization Algorithm based on EEG Information

Journal of the Korean Society for Library and Information Science
2018, 52(4), pp.91~110
DOI : 10.4275/KSLIS.2018.52.4.091
Publisher : 한국문헌정보학회
Research Area : Interdisciplinary Studies > Library and Information Science

Kim, Hyun Hee ¹, Yong-Ho Kim ²

¹명지대학교
²부경대학교

Excellent Accredited

ABSTRACT

We proposed a video summarization algorithm based on an ERP (Event Related Potentials)-based topic relevance model, a MMR (Maximal Marginal Relevance), and discriminant analysis to generate a semantically meaningful video skim. We then conducted implicit and explicit evaluations to evaluate our proposed ERP/MMR-based method. The results showed that in the implicit and explicit evaluations, the average scores of the ERP / MMR methods were statistically higher than the average score of the SBD (Shot Boundary Detection) method used as a competitive baseline, respectively. However, there was no statistically significant difference between the average score of ERP/MMR (λ = 0.6) method and that of ERP/MMR (λ = 1.0) method in both assessments.

KEYWORDS

ERP, ERP/MMR Model, Topic Relevance Model, Video Skim, Implicit Evaluation, Explicit Evaluation

Journal Copyright Policy

This license enables reusers to copy and distribute the material in any medium or format in unadapted form only, for noncommercial purposes only, and only if attribution is given to the creator.

Citation status

* References for papers published after 2025 are currently being built.

[journal] 권준수 / 2000 / 인지 기능 연구에서의 사건관련전위의 이용 / 인지과학작업 1(1) : 79~98

[journal] 김용호 / 2016 / Automatic Extraction Techniques of Topic - relevant Visual Shots Using Realtime Brainwave Responses: ERP N400 and P600 Hypotheses Test / 멀티미디어학회논문지 / 한국멀티미디어학회 19(8) : 1260~1274

[book] 정영미 / 2012 / 정보검색연구 / 연세대학교 대학출판문화원

[confproc] Allegretti, M. / 2015 / When Relevance Judgement is Happening?: An EEG-based study / Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval / ACM : 719~722

[journal] Burmester, J. / 2014 / Context Updating during Sentence Comprehension : The Effect of Aboutness Topic / Brain and Language 137 : 62~76

[confproc] Carbonell, J. / 1998 / The Use of MMR, Diversity-based Reranking for Reordering Documents and Producing Summaries / Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval / ACM : 335~336

[journal] Chen, F. / 2011 / An Autonomous Framework to Produce and Distribute Personalized Team-sport Video Summaries : A Basketball Case Study / IEEE Transactions on Multimedia 13(6) : 1381~1394

[journal] Eugster, M. J. / 2016 / Natural Brain-information Interfaces : Recommending Information by Relevance Inferred from Human Brain Signals / Scientific Reports 6(38580) : 1~10

[journal] Evans, W. J. / 1995 / Olfactory Event-related Potentials in Normal Human Subjects : Effects of Age and Gender / Electroencephalography and Clinical Neurophysiology 95(4) : 293~301

[journal] Hu, W. / 2011 / A Survey on Visual Content-based Video Indexing and Retrieval / IEEE Transactions on Systems, Man, and Cybernetics, Part C(Applications and Reviews) 41(6) : 797~819

[web] IBM Research Corp / 2016 / Morgan | IBM Creates First Movie Trailer by AI / https://www.youtube.com/watch?v=gJEzuYynaiw

[journal] Kim, H. H. / 2010 / Toward a Conceptual Framework of Key-frame Extraction and Storyboard Display for Video Summarization / Journal of the Association for Information Science and Technology 61(5) : 927~939

[confproc] Lin, C. / 2004 / ROUGE: A Package for Automatic Evaluation of Summaries / Proceedings of the Workshop on Text Summarization Branches Out (WAS 2004)

[book] Luck, S. J. / 2014 / An Introduction to the Event-related Potential Technique / MIT press

[book] Mayer, R. E. / 2009 / Multimedia Learning / Cambridge University Press

[journal] Mehmood, I. / 2016 / Divide-and-conquer based Summarization Framework for Extracting Affective Video Content / Neurocomputing 174 : 393~403

[confproc] Mishra, R. / 2015 / Real time and Non Real time Video Shot Boundary Detection using Dual Tree Complex Wavelet Transform / 2015 International Conference on Industrial Instrumentation and Control / ICIC : 1495~1500

[confproc] Mostafa, J. / 2016 / Deepening the Role of the User: Neuro-Physiological Evidence as a Basis for Studying and Improving Search / Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval / ACM : 63~70

[journal] Mudrik, L. / 2014 / Synchronous Contextual Irregularities Affect Early Scene Processing : Replication and Extension / Neuropsychologia 56 : 447~458

[journal] Nakano, H. / 2014 / Electrophysiological Response to Omitted Stimulus in Sentence Processing / NeuroReport 25(14) : 1169~1174

[confproc] Over, P. / 2008 / The TRECVid 2008 BBC Rushes Summarization Evaluation / Proceedings of the 2nd ACM TRECVid Video Summarization Workshop / ACM : 1~20

[journal] Schumacher, P. B. / 2012 / Positional Influences on Information Packaging : Insights from Topological Fields in German / Journal of Memory and Language 67(2) : 295~310

[journal] Seoane, L. F. / 2015 / Images from the Mind : BCI Image Evolution based on Rapid Serial Visual Presentation of Polygon Primitives / Brain-Computer Interfaces 2(1) : 40~56

[journal] Sitnikova, T. / 2008 / Two Neurocognitive Mechanisms of Semantic Integration during the Comprehension of Visual Real-world Events / Journal of Cognitive Neuroscience 20(11) : 2037~2057

[journal] Tavassolipour, M. / 2014 / Event Detection and Summarization in Soccer Videos using Bayesian Network and Copula / IEEE Transactions on Circuits and Systems for Video technology 24(2) : 291~304

[journal] Wang, L. / 2013 / New is not always Costly : Evidence from Online Processing of Topic and Contrast in Japanese / Frontiers in Psychology 4 : 363~

[journal] Xu, R. / 2014 / Enhanced low-latency Detection of Motor Intention from EEG for Closedloop Brain-computer Interface Applications / IEEE Transactions on Biomedical Engineering 61(2) : 288~296

[journal] Yang, J. / 2012 / Channel Selection and Classification of Electroencephalogram Signals : An Artificial Neural Network and Genetic Algorithm-based Approach / Artificial Intelligence in Medicine 55(2) : 117~126

[confproc] Zhu, X. / 2007 / A Text-to-picture Synthesis System for Augmenting Communication / Proceedings of the 22nd National Conference on Artificial Intelligence / AAAI Press 2 : 1590~1595

This paper was written with support from the National Research Foundation of Korea.

KJCKorea
Journal Central

Journal of the Korean Society for Library and Information Science 2025 KCI Impact Factor : 1.37