Design Tradeoffs in Backend Organization in Out-Of-Order RISC-V Processors.

Publication
Symposium on Parallelism in Algorithms and Architectures
Date

Abstract

Although not optimal for Large Language Model (LLM) inference, general-purpose processors remain a practical choice due to their ubiquity and accessibility. This work explores strategies to maxi- mize server utilization when incorporating such applications into the workload mix. To do that, we conduct an exhaustive profiling process to analyze hardware-software interactions, leading to two key observations. First, we have validated and quantified the com- mon intuitions regarding LLM execution, with a specific focus on the microarchitecture’s backend. Second, we observe the relatively low contention of LLMs and conventional applications (e.g., SPEC CPU17) when running together. Inspired by these findings, we explore whether combining applications of each type on a common server, if properly balanced, could lead to a better system utiliza- tion. Using both state-of-the-art server configurations and slightly older systems, we demonstrate that executing LLMs on general- purpose processors is feasible with minimal impact on co-located applications and lead to a better overall server performance.