#cpu inference
Cpu Inference: 7 AI articles covering cpu inference news, analysis, and research
Articles
A Coding Guide to Cactus Whistle: Does a 16.9 MB Speech Model⭐8
Cactus Whistle, a 16.9 MB CPU speech model, tested against Whisper base: word error rate, speed, streaming, timestamps, keyword biasing, and speech-to-tool call...
Mistral AI Releases Timestral: A 209M-Parameter Forecasting⭐10
Mistral AI releases Timestral, a family of four zero-shot time series models up to 209M parameters that runs on CPUs and beats Google's TimesFM-3 on key benchma...
How to Turn a RECIST Line into a 3D Tumor Segmentation Mask⭐10
Learn how Lumina converts a single RECIST line from a CT scan into a full 3D tumor segmentation mask, covering design decisions, data prep, and implementation.
Accelerating LLM Inference via Vector Index Based Output Embeddings⭐10
Vector index-based output embeddings accelerate LLM decoding by replacing dense projections with HNSW retrieval, boosting throughput up to 82% while preserving ...
Speculative Decoding on CPUs: Nearly 4x Faster Token Generation⭐10
Speculative decoding with DFlash accelerates CPU token generation up to 3.92x on Qwen3.5-9B, cutting costs by 74%. Learn how vLLM makes it possible.
Liquid AI Releases LFM2.5-Encoder-230M and LFM2.5-Encoder-350M:⭐7
Liquid AI launches LFM2.5-Encoder-230M and 350M bidirectional models optimized for fast 8K context inference on CPU, enabling efficient real-time edge NLP.
LFM2.5-Encoders for Fast Long-Context Inference on CPU⭐8
LFM2.5-Encoders enable fast long-context inference on CPU with 0.2B parameters. Optimized for edge devices, privacy, and offline use. Hugging Face model with 69...