Resolving performance uncertainty of NVMe over TCP in multi-tenant environments

다국어 초록 (Multilingual Abstract)

Non-Volatile Memory Express (NVMe) over TCP is a promising technology for accessing remote storage, but its protocol processing implementations were not designed for multi-tenant containerized environments, leading to severe performance interference and imbalance issues. These problems stem from two root causes: (1) on the host, indirect CPU usage by high-priority kernel workers and network interrupt handling violates container isolation, and (2) on the target, network-level static session distribution leads to CPU load imbalance.
In this paper, we propose a comprehensive, two-part solutions implemented within the Linux kernel to address these challenges on both the host and target sides. For the host, i-NVMe ensures CPU isolation by accurately accounting for the indirect CPU costs of the I/O workers, charges it to containers in proportion to their NVMe over TCP traffic. Also, to limit the high-priority kernel worker’s CPU consumptions, we apply the adaptive traffic throttling. For the target, lb-nvmet performs dynamic session steering, migrating sessions to less-congested cores based on real-time load metrics, while using an adaptive threshold to prevent excessive migration overhead.
Our evaluation on a real testbed with diverse workloads demonstrates that the proposed schemes work in concert to provide strong CPU isolation and significantly reduce performance disparities. Notably, our solution improves performance uniformity by up to 5.7× without degrading total I/O throughput, making NVMe over TCP a viable and predictable technology for modern multi-tenant storage systems.

목차 (Table of Contents)

• Chapter 1. Introduction 1
• Chapter 2. i-NVMe: Isolated NVMe over TCP for a Containerized Environment 4
• 2.1 Background and Motivation 4
• 2.2 Design and Implementation of i-NVMe 11
• 2.3 Performance Evaluation of i-NVMe 24
• 2.4 Related Works 33
• 2.5 Summary 35
• Chapter 3. Resolving NVMe over TCP Uncertainty via Elastic Session Steering 36
• 3.1 Background and Motivation 36
• 3.2 Design and Implementation of lb-nvmet 50
• 3.3 Performance Evaluationof lb-nvme 59
• 3.4 Related Works 72
• 3.5 Summary 73
• Conclusion 74
• References 76
• Korean Abstraction 83