본문 바로가기
  • Home

Resolving Memory Bottlenecks in Hardware Accelerators with Data Prefetch

  • Journal of The Korea Society of Computer and Information
  • Abbr : JKSCI
  • 2024, 29(6), pp.1-12
  • Publisher : The Korean Society Of Computer And Information
  • Research Area : Engineering > Computer Science
  • Received : April 1, 2024
  • Accepted : June 3, 2024
  • Published : June 28, 2024

Hyein Lee 1 Jinoo Joung 1

1상명대학교

Accredited

ABSTRACT

Deep learning with faster and more accurate results requires large amounts of storage space and large computations. Accordingly, many studies are using hardware accelerators for quick and accurate calculations. However, the performance bottleneck is due to data movement between the hardware accelerators and the CPU. In this paper, we propose a data prefetch strategy that can efficiently reduce such operational bottlenecks. The core idea of the data prefetch strategy is to predict the data needed for the next task and upload it to local memory while the hardware accelerator (Matrix Multiplication Unit, MMU) performs a task. This strategy can be enhanced by using a dual buffer to perform read and write operations simultaneously. This reduces latency and execution time of data transfer. Through simulations, we demonstrate a 24% improvement in the performance of hardware accelerators by maximizing parallel processing with dual buffers and bottlenecks between memories with data prefetch.

Citation status

* References for papers published after 2022 are currently being built.

This paper was written with support from the National Research Foundation of Korea.