#llm inference
Llm Inference: 6 AI articles covering llm inference news, analysis, and research
Articles
Concurrent HBM and Host Memory Access Boosts LLM Inference⭐9
Georgia Tech, Nvidia, and Stanford show that concurrent HBM and host memory access boosts LLM inference throughput by hiding latency and scaling beyond HBM capa...
Heterogeneous Memory Chiplets Accelerate Multi-Request LLM⭐9
NUS researchers propose heterogeneous memory chiplets that combine high-bandwidth, high-capacity, and near-memory compute to accelerate multi-request LLM infere...
DeepSeek-V4.1-Flash Launches with 1M Context, FP4 KV Cache, and⭐9
DeepSeek-V4.1-Flash launches with 1M context, FP4 KV cache, and cross-layer attention reuse—cutting KV memory to 890 bytes per token for input-heavy agent workl...
Workload-Driven HBF Substrate for Capacity-Scalable LLM Inference⭐8
Workload-driven HBF substrate enables capacity-scalable LLM inference with 3.2x bandwidth gains and 40% lower energy.
Accelerating LLM Inference via Vector Index Based Output Embeddings⭐10
Vector index-based output embeddings accelerate LLM decoding by replacing dense projections with HNSW retrieval, boosting throughput up to 82% while preserving ...
Hybrid HBM-HBF Architecture in LLM Inference⭐9
Hybrid HBM-HBF architecture for LLM inference cuts memory bottlenecks. Oxford research combines high-bandwidth memory and flash for scalable AI performance.