#cpu inference
Cpu Inference: 4 AI articles covering cpu inference news, analysis, and research
Articles
Accelerating LLM Inference via Vector Index Based Output Embeddings⭐10
Vector index-based output embeddings accelerate LLM decoding by replacing dense projections with HNSW retrieval, boosting throughput up to 82% while preserving ...
Speculative Decoding on CPUs: Nearly 4x Faster Token Generation⭐10
Speculative decoding with DFlash accelerates CPU token generation up to 3.92x on Qwen3.5-9B, cutting costs by 74%. Learn how vLLM makes it possible.
Liquid AI Releases LFM2.5-Encoder-230M and LFM2.5-Encoder-350M:⭐7
Liquid AI launches LFM2.5-Encoder-230M and 350M bidirectional models optimized for fast 8K context inference on CPU, enabling efficient real-time edge NLP.
LFM2.5-Encoders for Fast Long-Context Inference on CPU⭐8
LFM2.5-Encoders enable fast long-context inference on CPU with 0.2B parameters. Optimized for edge devices, privacy, and offline use. Hugging Face model with 69...