본문 바로가기
  • Home

An Accurate Log Object Recognition Technique

  • Journal of The Korea Society of Computer and Information
  • Abbr : JKSCI
  • 2023, 28(2), pp.89-97
  • DOI : 10.9708/jksci.2023.28.02.089
  • Publisher : The Korean Society Of Computer And Information
  • Research Area : Engineering > Computer Science
  • Received : January 16, 2023
  • Accepted : February 6, 2023
  • Published : February 28, 2023

Jiho Ju 1 Byungchul Tak 1

1경북대학교

Accredited

ABSTRACT

In this paper, we propose factors that make log analysis difficult and design technique for detecting various objects embedded in the logs which helps in the subsequent analysis. In today’s IT systems, logs have become a critical source data for many advanced AI analysis techniques. Although logs contain wealth of useful information, it is difficult to directly apply techniques since logs are semi-structured by nature. The factors that interfere with log analysis are various objects such as file path, identifiers, JSON documents, etc. We have designed a BERT-based object pattern recognition algorithm for these objects and performed object identification. Object pattern recognition algorithms are based on object definition, GROK pattern, and regular expression. We find that simple pattern matchings based on known patterns and regular expressions are ineffective. The results show significantly better accuracy than using only the patterns and regular expressions. In addition, in the case of the BERT model, the accuracy of classifying objects reached as high as 99%.

Citation status

* References for papers published after 2022 are currently being built.

This paper was written with support from the National Research Foundation of Korea.