Thinking Machines Lab Releases Inkling-Small: A 276B Total, 12B Active Open Weights Multimodal MoE Model

On August 2, 2026, Thinking Machines Lab unveiled Inkling-Small, an open-weights Mixture-of-Experts (MoE) model with 276B total parameters and 12B active per inference step. This positions it at about a quarter of the scale of the flagship Inkling model, which boasts 975B total and 41B active parameters. Trained on NVIDIA GB300 NVL72 systems, Inkling-Small natively reasons over text, images, and audio. It supports a context window of up to 1M tokens and features adjustable thinking effort. The weights are released under the Apache 2.0 license and are available on Hugging Face.

Deployment Requirements

Inkling-Small is designed for practical deployment, with its quantized checkpoint being key to accessibility. According to the official model card, the BF16 checkpoint requires a minimum of 600 GB of aggregated VRAM, which can be satisfied by setups such as 4x NVIDIA B300 or 8x NVIDIA H200 GPUs. For more resource-constrained environments, the NVFP4 quantized checkpoint reduces the VRAM requirement to 180 GB. This enables W4A4 inference on a single B300 (requiring SM100+ architecture) or W4A16 on two H200 GPUs. Supported runtimes include SGLang, vLLM, TokenSpeed, Unsloth, and Hugging Face.

Implications for Adoption

The single-GPU deployment path is a game-changer, moving a 276B-parameter model out of the realm of frontier labs and into the hands of startups and mid-sized enterprises. Startups can now self-host Inkling-Small on a single rented B300 instance, while enterprises with existing H200 capacity can serve it without additional hardware investments. Regulated industries—such as financial services, healthcare, insurance, telecom, and the public sector—can leverage a private-weights option for sensitive workloads. Use cases include coding agents, terminal automation, document and chart understanding, and, with audio support, call-center analytics, voice interfaces, and meeting summarization.

As of 2026, the release of Inkling-Small underscores a trend toward more efficient, accessible multimodal AI, enabling broader adoption and innovation across sectors.

via MarkTechPost