#multi-request inference

Heterogeneous Memory Chiplets Accelerate Multi-Request LLM Inference (NUS)NEW⭐9

NUS researchers propose heterogeneous memory chiplets that combine high-bandwidth, high-capacity, and near-memory compute to accelerate multi-request LLM infere...