Thinking Machines Releases Inkling 2 Mini: A 26B Looped MoE That Runs on a 32GB Mac
Thinking Machines Lab has released Inkling 2 Mini, an open-weight looped transformer with 26.4B total parameters. Rather than adding more weights, the model reuses its middle layers three times per token, keeping the checkpoint near 20GB β small enough to run on a 32GB Mac or a single NVIDIA DGX Spark. For developers, Inkling 2 Mini offers a local option for agentic coding, tool use, and document workflows. It ships under Apache 2.0 on Hugging Face, with hosted inference and fine-tuning available through Tinker.
TL;DR
- Size: 26.4B total parameters, 3.4B active (5.4B effective per token with looping), up to 1M-token context.
- Runs on: Apple silicon with 32GB+ memory, one NVIDIA DGX Spark, or GPUs with 32GB+ VRAM. GGUF builds are available via Unsloth and llama.cpp.
- Performance: 20.1 on the Artificial Analysis Index, tied with Qwen3.8-27B (self-reported).
- Best result: 79.2% on SWE-bench Verified (bash only).
- Worst results: 15.2% on HLE-Diamond and 50.0% on MultiChallenge.
- Bottom line: Delivers strong agentic coding from a 20GB checkpoint, but shows weaker knowledge recall and multi-turn instruction following.
What Is Inkling 2 Mini?
Inkling 2 Mini is a looped transformer β a Mixture-of-Experts (MoE) architecture that reuses its middle layers three times per token instead of scaling parameter count. This design keeps the checkpoint compact (~20GB) while achieving 5.4B effective parameters per token, making it practical for local deployment and resource-constrained environments.
Architecture Highlights
- Looped middle layers: Repeats the same weights three times per token, trading compute for memory efficiency.
- Mixture-of-Experts routing: 26.4B total parameters with only 3.4B active at any time.
- Long context: Supports up to 1M tokens, suitable for document-heavy and agentic workflows.
Deployment and Availability
Inkling 2 Mini is designed for local-first deployment in 2026's increasingly memory-constrained AI landscape:
- Apple silicon: Runs on Macs with 32GB+ unified memory.
- NVIDIA DGX Spark: Fits on a single device.
- GPUs: Compatible with any GPU offering 32GB+ VRAM.
- Quantized builds: GGUF versions available via Unsloth and llama.cpp for broader hardware compatibility.
Licensing and Tooling
- License: Apache 2.0, released on Hugging Face.
- Hosted inference and fine-tuning: Available on Tinker.
Performance Benchmarks
Inkling 2 Mini delivers mixed results across standard evaluations, with strengths in agentic coding and weaknesses in knowledge-intensive and multi-turn tasks.
| Benchmark | Score | Notes |
|---|---|---|
| Artificial Analysis Index | 20.1 | Tied with Qwen3.8-27B (self-reported) |
| SWE-bench Verified | 79.2% | Bash only |
| MultiChallenge | 50.0% | Weaker multi-turn instruction following |
| HLE-Diamond | 15.2% | Limited knowledge recall |
Strengths
- Agentic coding: Strong SWE-bench Verified performance (79.2%) from a compact 20GB checkpoint.
- Tool use and document workflows: Well-suited for local agentic applications.
Limitations
- Knowledge recall: 15.2% on HLE-Diamond indicates weaker factual knowledge retrieval.
- Multi-turn following: 50.0% on MultiChallenge suggests room for improvement in complex conversational instruction following.
Why It Matters in 2026
As organizations push toward local and edge AI deployments, memory efficiency has become a defining constraint. Inkling 2 Mini's looped MoE design demonstrates a viable path to capable agentic models that fit within consumer hardware budgets β particularly relevant as Apple silicon and compact accelerators like the DGX Spark gain traction for private, on-device AI workflows.
Getting Started
- Model weights: Hugging Face β Thinking Machines Inkling 2 Mini
- Hosted inference and fine-tuning: Tinker Playground
- Release announcement: Thinking Machines Lab
via MarkTechPost
