#inference serving

Tsinghua AI Open-Sources AKV: A Cache Engineering Server for LLMNEW⭐9

Tsinghua researchers open-source AKV, a KV cache engineering server on SGLang that cuts LLM agent inference costs by 27.7% while improving task scores by 1.7 po...