#local inference
Local Inference: 9 AI articles covering local inference news, analysis, and research
Articles
Nace AI Open-Sources Drex Max v1: A 27B Decision Model That Runsβ9
Nace AI open-sources Drex Max v1, a 27B decision model with 262K context that runs locally on Apple Silicon and NVIDIA GPUs via GGUF and llama.cpp forks.
NVIDIA Releases Nemotron 3.5 Bolt E4B 4GB: An Open Smallβ8
NVIDIA's Nemotron 3.5 Bolt E4B is an open 4GB small language model for local voice agents, game NPCs, and robots, running on RTX, Jetson, and Spark hardware.
Run Fast, Private Agents On-Device with LFM2.5-24B-A2Bβ8
LFM2.5-24B-A2B is LiquidAI's 24B sparse model for fast, private on-device AI agents. Learn its architecture, use cases, and how to run it locally.
ChestPheNoT: Deployable, Auditable Label-Status-Evidenceβ9
ChestPheNoT extracts finding labels, three-class status, and verbatim evidence spans from radiology reports using a locally deployable, auditable 0.5β3B model.
Hugging Face Transformers Now Runs llama.cpp Quants Nativelyβ8
Hugging Face Transformers now natively supports llama.cpp GGUF quantized models, letting you run efficient local LLMs and multimodal models directly via the Tra...
Best Open-Source Agent Harnesses for Local LLMs in 2026β8
Ranking 11 open-source agent harnesses for local LLMs in 2026 by docs, license, maintenance, and safety, plus context window, tool calling, and memory tips.
Time Capsule of Testable Human Knowledge: 41 Years of Jeopardy!β9
A 9GB local AI model answers 67% of 529,939 Jeopardy! clues across 41 seasons, proving portable, free knowledgeβoutperforming IBM's 2011 Watson on post-training...
Perplexity Introduces Portable Computer on NVIDIA DGX Spark:β7
Perplexity's Portable Computer on NVIDIA DGX Spark enables local AI execution, OS-enforced sandboxing, and zero per-token fees for on-device tasks.
Fastino Releases GLiNER2.5: A Boundary-Prediction Architectureβ7
Fastino's GLiNER2.5 removes span enumeration with boundary prediction, boosting efficiency, joint relation extraction, and zero-shot F1 on multilingual benchmar...