Cactus Compute has unveiled Needle 2, an open-source 45M-parameter model designed for tool calling, device use, and structured extraction. The entire model is packaged as a single 14MB binary, enabling a full session to run in approximately 28MB of RAM. Weights are trained and deployed at CQ2-bit precision using Cactus Quants, and the model is sealed within Cactus's proprietary C++ engine, eliminating the need for a runtime installation or inference-time downloads. Reported decode speeds reach 500 tokens per second on a Raspberry Pi 5, 400β1,500 tokens per second on Meta Quest 3S and Apple Vision Pro, and 300β700 tokens per second on sub-$200 smartphones.
Design Philosophy: Narrow and Purpose-Built
The team at Cactus Compute deliberately frames Needle 2's niche: mapping a messy sentence onto a typed function signature requires no world knowledge or open-ended prose generation. This premise is why 45M parameters suffice for the task, and why the model targets hardware lacking GPUs or NPUs. In an AI landscape increasingly focused on massive models, Needle 2 represents a counter-trend toward efficiency and specializationβa timely approach as developers in 2026 seek to deploy AI on commodity and even low-power devices.
Deployment Readiness: Out-of-the-Box Support
Needle 2 is production-ready, shipping as prebuilt binaries and a static library for macOS, Linux (x86-64, ARM64, ARMv7, RISC-V, MIPS32el), Windows, Android, iOS/watchOS/tvOS, and WebAssembly. Cactus reports that Pebble already integrates Needle locally in the Index 01 app for offline voice actions, demonstrating real-world viability.
Target Organizations
- Who benefits most: Teams shipping firmware or apps on constrained hardware, including seed-stage wearable and IoT startups, mid-market consumer electronics OEMs, robotics teams, and large device manufacturers requiring an offline fallback. Cloud-first SaaS teams will find less value, as the model's strengths lie in edge deployment.
- Key industries: Smart home, wearables, low-end mobile, automotive in-cabin control, service robotics, retail kiosks and point-of-sale systems, routers and IP cameras, and regulated sectors like healthcare or finance where audio or data cannot leave the device.
- Typical applications: Offline voice assistants, on-device command interpretation, structured data extraction from user input, and firmware-level automation.
By prioritizing minimal resource usage and seamless integration, Needle 2 offers a pragmatic solution for developers aiming to embed AI capabilities into devices without cloud dependencies.
via MarkTechPost