시선추적-뇌파 기반의 비디오 요약 생성 방안 연구 (Video Summarization Using Eye Tracking and Electroencephalogram (EEG) Data)

김현희 (Kim, Hyun Hee); 김용호 (Yong-Ho Kim)

doi:10.4275/KSLIS.2022.56.1.095

Video Summarization Using Eye Tracking and Electroencephalogram (EEG) Data

Journal of the Korean Society for Library and Information Science
2022, 56(1), pp.95-117
DOI : 10.4275/KSLIS.2022.56.1.095
Publisher : 한국문헌정보학회
Research Area : Interdisciplinary Studies > Library and Information Science
Received : January 17, 2022
Accepted : February 22, 2022
Published : February 28, 2022

Kim, Hyun Hee ¹, Yong-Ho Kim ²

¹명지대학교
²부경대학교 신문방송학과

Excellent Accredited

ABSTRACT

This study developed and evaluated audio-visual (AV) semantics-based video summarization methods using eye tracking and electroencephalography (EEG) data. For this study, twenty-seven university students participated in eye tracking and EEG experiments. The evaluation results showed that the average recall rate (0.73) of using both EEG and pupil diameter data for the construction of a video summary was higher than that (0.50) of using EEG data or that (0.68) of using pupil diameter data. In addition, this study reported that the reasons why the average recall (0.57) of the AV semantics-based personalized video summaries was lower than that (0.69) of the AV semantics-based generic video summaries. The differences and characteristics between the AV semantics-based video summarization methods and the text semantics-based video summarization methods were compared and analyzed.

KEYWORDS

EEG, Eye Tracking, Video Summarization, Audio-visual Semantics, Text Semantics

Citation status

* References for papers published after 2023 are currently being built.

[journal] 김현희 / 2016 / Understanding Topical Relevance of Multimedia based on EEG Techniques / 한국문헌정보학회지 / 한국문헌정보학회 50(3) : 361~381

[journal] 김현희 / 2019 / Towards the Generation of Language-based Sound Summaries Using Electroencephalogram Measurements / 정보관리학회지 / 한국정보관리학회 36(3) : 131~148

[thesis] 안형모 / 2013 / 텍스트 기반 학습에서 학습매체의 크기와 종류가 학습 집중도에 미치는 영향 / 석사 / 한국교원대학교 대학원

[journal] 윤정원 / 2021 / How do Formats of Health Related Facebook Posts Effect on Eye Movements and Cognitive Outcomes? / 한국문헌정보학회지 / 한국문헌정보학회 55(3) : 219~237

[book] 정영미 / 2012 / 정보검색연구 / 연세대학교 출판문화원

[journal] Ajanki, A. / 2009 / Can eyes reveal interest? Implicit queries from gaze patterns / User Modeling and User-Adapted Interaction 19(4) : 307~339

[other] Bezugam, S. S. / 2021 / Efficient video summarization framework using EEG and eye-tracking signals / arXiv preprint arXiv: 2101.11249

[other] Bhattacharya, N. / 2018 / Relating eye-tracking measures with changes in knowledge on search tasks / arXiv preprint arXiv: 1805.02399

[confproc] Bhattacharya, N. / 2020 / Relevance prediction from eye-movements using semi-interpretable convolutional neural networks / Proceedings of the 2020 Conference on Human Information Interaction and Retrieval : 223~233

[journal] Evans, W. J. / 1995 / Olfactory event-related potentials in normal human subjects : Effects of age and gender / Electroencephalography and Clinical Neurophysiology 95(4) : 293~301

[book] Foley, J. J. / 2015 / Encyclopedia of Information Science and Technology / IGI Global : 5897~5905

[journal] Golenia, J. E. / 2018 / Implicit relevance feedback from electroencephalography and eye tracking in image search / Journal of Neural Engineering 15(2) : 026002~

[journal] Gwizdka, J. / 2017 / Temporal dynamics of eye-tracking and EEG during reading and relevance decisions / Journal of the Association for Information Science and Technology 68(10) : 2299~2312

[confproc] Gwizdka, J. / 2015 / Differences in eye-tracking measures between visits and revisits to relevant and irrelevant web pages / Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval / ACM : 811~814

[book] Hansen, D. / 2010 / Analyzing social media networks with NodeXL : Insights from a connected world / Morgan Kaufmann

[journal] Hughes, J. R. / 1999 / Conventional and quantitative electroencephalography in psychiatry / The Journal of neuropsychiatry and clinical neurosciences 11(2) : 190~208

[confproc] Katti, H. / 2011 / Affective video summarization and story board generation using pupillary dilation and eye gaze / 2011 IEEE International Symposium on Multimedia / IEEE : 319~326

[journal] Kaur, T. / 2015 / Text extraction from natural scene using PCA / International Journal of Computer Science Engineering & Technology 5(7) : 272~277

[journal] Kim, H. H. / 2019 / Video summarization using event-related potential responses to shot boundaries in real-time video watching / Journal of the Association for Information Science and Technology 70(2) : 164~175

[journal] Kim, H. H. / 2019 / ERP/MMR algorithm for classifying topic-relevant and topic-irrelevant visual shots of documentary videos / Journal of the Association for Information Science and Technology 70(9) : 931~941

[journal] Moon, J. / 2019 / Detecting user attention to video segments using interval EEG features / Expert Systems with Applications 115 : 578~592

[confproc] Oliveira, F. T. / 2009 / Discriminating the relevance of web search results with measures of pupil size / Proceedings of the SIGCHI Conference on Human Factors in Computing Systems / ACM : 2209~2212

[journal] Schumacher, P. B. / 2012 / Positional influences on information packaging : Insights from topological fields in German / Journal of Memory and Language 67(2) : 295~310

[confproc] Shi, Z. F. / 2017 / Attention evaluation with eye tracking glasses for EEG-based emotion recognition / 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER) / IEEE : 86~89

[journal] Syn, S. Y. / 2021 / Investigation on reading behaviors and cognitive outcomes of Facebook health information / Online Information Review 45(6) : 1097~1115

[journal] van Berkum, J. J. / 1999 / Semantic integration in sentences and discourse : Evidence from the N400 / Journal of Cognitive Neuroscience 11(6) : 657~671

[journal] Wang, L. / 2013 / New is not always costly : evidence from online processing of topic and contrast in Japanese / Frontiers in Psychology 4 : 363~

[confproc] Zhao, L. M. / 2018 / Active feedback framework with scan-path clustering for deep affective models / International Conference on Neural Information Processing / Springer : 330~340

This paper was written with support from the National Research Foundation of Korea.

KJCKorea
Journal Central

Journal of the Korean Society for Library and Information Science 2023 KCI Impact Factor : 0.84

Video Summarization Using Eye Tracking and Electroencephalogram (EEG) Data

ABSTRACT

KEYWORDS

Citation status

* References for papers published after 2023 are currently being built.

Journal of the Korean Society for Library and Information Science 2023 KCI Impact Factor : 0.84

Video Summarization Using Eye Tracking and Electroencephalogram (EEG) Data

ABSTRACT

KEYWORDS

Statistics

Tools

Issue List

Citation status

KCI Citation Counts (0)

REFERENCES (28) * References for papers published after 2023 are currently being built.

Search PDF

Citation

* References for papers published after 2023 are currently being built.