#prefill/decode disaggregation

Prefill/Decode Disaggregation: 1 AI articles covering prefill/decode disaggregation news, analysis, and research

Articles

Google Cloud Releases llm-d 1.0: Cache-Aware Routing Across TPU

Google Cloud Releases llm-d 1.0: Cache-Aware Routing Across TPUNEW⭐8

πŸ“… 2ε°ζ—Άε‰πŸ‘ 4 views

Google Cloud releases llm-d 1.0, an open-source Kubernetes-native inference stack with cache-aware routing across TPU and GPU fleets for distributed AI serving.