BotPsy: An On-Device Personal Assistant Powered by a Sub-Billion-Parameter Agentic Language Model
By Milan Gritta ยท Published November 11, 2026
As AI assistants become more deeply embedded in daily life, the question of where they run is increasingly consequential. Cloud-based assistants offer raw capability, but they also demand constant connectivity and raise legitimate privacy concerns around the sensitive personal data they handle. BotPsy takes the opposite approach: it is a personal assistant that runs entirely on-device, driven by an agentic language model with fewer than one billion parameters.
Why a Sub-Billion-Parameter Model?
The case for small, on-device models has only strengthened by 2026. Current flagship smartphones ship with NPUs capable of comfortably running sub-1B-parameter LLMs at interactive speeds, and consumers have grown wary of routing personal data through third-party servers. BotPsy is built around three core design goals:
- Privacy by default. All inference, context, and user data stay on the device. Nothing is transmitted to a remote server.
- Always available. The assistant works offline, with no dependency on network connectivity or cloud uptime.
- Agentic capability on a limited budget. Rather than behaving as a simple chatbot, BotPsy plans, invokes tools, and executes multi-step tasks โ all from a compact model size.
Agentic Behavior in a Compact Package
The term agentic is central to how BotPsy differs from conventional on-device assistants. Instead of mapping a single user utterance to a single response, the model operates as an agent that can:
- Decompose a user request into a sequence of subtasks.
- Select and invoke on-device tools and APIs (calendar, notifications, messaging, local search).
- Reason across intermediate results before producing a final answer or action.
- Maintain context across turns while respecting the strict memory and compute constraints of a mobile device.
Achieving this with a sub-billion-parameter model requires careful alignment between model capacity and the tool interface. BotPsy's architecture and training pipeline are designed to keep the reasoning burden tractable โ narrowing the tool space and structuring prompts so the model can plan reliably without the scale of a frontier LLM.
The Trade-Offs of Running at the Edge
Running an assistant on-device is not without costs, and BotPsy is explicit about its constraints. A sub-billion-parameter model will never match the breadth of a large cloud model, so the system is deliberately scoped to the assistant tasks where a small agent can succeed reliably. Latency, thermal budgets, and battery life on real devices are treated as first-class engineering concerns rather than afterthoughts.
The result is a pragmatic position in the 2026 assistant landscape: not a general-purpose reasoner, but a private, responsive, and dependable personal agent that keeps the user's data where it belongs โ on the user's device.
BotPsy is published under the QVAC project. To read more articles from the team, visit the blog.