PGEMMlib : An Efficient Tiled GEMM Library for PIM Devices with Data Transfer Cost Minimization = PGEMMlib: 데이터 전송 비용 최소화를 통한 PIM 장치용 효율적인 GEMM 라이브러리

다국어 초록 (Multilingual Abstract)

To address memory bottlenecks in processor-centric systems, the ProcessingIn-Memory (PIM) paradigm, which promises high levels of parallelism with high memory bandwidth and low memory access latency, has been studied for decades. After decades of research, several vendors have recently begun to commercialize near-bank PIM architectures. General Matrix Multiplication (GEMM), one of the primary linear algebra kernels, serves an important role in deep learning and scientific applications. However, to the best of our knowledge, there’s no publicly available software implementation tailored to these new architectures, but only research for General Matrix Vector Multiplication (GEMV) or hardware approach for GEMM exists.
In this paper, we propose PGEMMlib, an efficient GEMM library for the realworld PIM system. PGEMMlib provides the various tiling techniques and the TileSelector to operate the efficient GEMM in the near-bank PIM. The proposed tiling techniques can reduce the memory transfer overhead and maximize the parallelism to perform GEMM in PIM with the appropriate tiling factors. In the real-world PIM,
the appropriate tiling factors vary significantly depending on the input GEMM shape and the given PIM resources. Therefore, we also provide Tile-Selector, based on an analytical approach, in order to select appropriate input-aware tiling factors from a large search space of tiling variants. With the selected parameters, TileSelector allows efficient execution of input GEMM operations. Also, PGEMMlib provides the parallel method of BatchedGEMM, which has been widely used in deep learning models. Compared to the baseline GEMM from existing GEMV operations, PGEMMlib provides a peak speedup of up to 9.32x and 1.68x geometric mean performance improvement for key matrix multiplications used in large language models. Moreover, PGEMMlib’s GEMM is highly scalable, making it easy to expand to larger PIM systems in the near future.

목차 (Table of Contents)

• CHAPTER I. INTRODUCTION . 1
• CHAPTER II. BACKGROUND & MOTIVATION 4
• SECTION I. NEAR-BANK PIM SYSTEMS . 4
• SECTION II. PROGRAMMING MODEL FOR PIM ARCHITECTURES 5
• SECTION III. MEMORY-BOUND PROBLEM ON PROGRAMMING MODEL 6
• CHAPTER III. PGEMMLIB 8
• SECTION I. OVERVIEW 8
• SECTION II. BASEGEMM 9
• SECTION III. NTILEGEMM & MTILEGEMM 10
• SECTION IV. KTILEGEMM & MIXEDGEMM 11
• SECTION V. BATCHEDGEMM 12
• CHAPTER IV. PGEMMLIB’S TILE-SELECTOR 13
• SECTION I. KERNEL EXECUTION TIME PREDICTION 13
• 4.1.1 Key Factor 1 : 𝒏𝑫𝑶𝑻𝑫𝑷𝑼 13
• 4.1.2 Key Factor 2: K-tilesize 14
• 4.1.3 Approximated Kernel Execution Time. 15
• SECTION II. DATA TRANSFER TIME PREDICTION . 15
• 4.2.1 Data Transfer Size . 15
• 4.2.2 Memory Bandwidth 16
• 4.2.3 Approximated Data Transfer Time . 16
• CHAPTER V. EVALUATION 17
• SECTION I. EVALUATION SETUP 17
• 5.1.1 System Configuration 17
• 5.1.2 Data set 17
• SECTION II. EVALUATION OF NTILEGEMM 18
• 5.2.1 Reduced Memory Transfer Size with N-factor . 18
• 5.2.2 More Considerations for NtileGEMM. 19
• SECTION III. EVALUATION OF KTILEGEMM 19
• 5.3.1 Reduced Memory Transfer Size with K-factor 19
• 5.3.2 More Considerations for KtileGEMM 20
• SECTION III. EVALUATION OF TILE-SELECTOR 22
• 5.4.1 Evaluation of SingleGEMM . 22
• 5.4.1 Evaluation of BatchedGEMM 23
• CHAPTER VI. RELATED WORKS 24
• CHAPTER VII. CONCLUSION 25
• REFERENCES 26
• ABSTRACT IN KOREAN 32