#kv cache offload

Google Cloud Releases llm-d 1.0: Cache-Aware Routing Across TPUNEW⭐8

Google Cloud releases llm-d 1.0, an open-source Kubernetes-native inference stack with cache-aware routing across TPU and GPU fleets for distributed AI serving.