NVIDIA has released Nemotron-Labs-TwoTower, a diffusion language model built on a pretrained autoregressive backbone. The model is available as open weights under the NVIDIA Nemotron Open Model License.
Nemotron-Labs-TwoTower leverages a frozen Nemotron-3-Nano-30B-A3B backbone, combining autoregressive generation with diffusion-based refinement. This hybrid approach aims to improve output quality, controllability, and inference efficiency compared to purely autoregressive models.
Key Features
- Open-Weight Release: Available under the permissive NVIDIA Nemotron Open Model License, enabling research and commercial use.
- Frozen Backbone: Built on the existing Nemotron-3-Nano-30B-A3B, a compact but powerful autoregressive model, which remains unchanged during training of the diffusion components.
- Diffusion Language Modeling: Applies iterative denoising to generate text, allowing for finer control over output characteristics such as style, tone, and factual consistency.
- 2026 Context: As of mid-2026, NVIDIA continues to advance its open-weight AI ecosystem, with Nemotron-Labs-TwoTower representing a significant step toward more flexible and capable language models.
Implications
This release is notable for combining two major trends in AI: the scalability of autoregressive models and the precision of diffusion-based generation. Developers and researchers can now experiment with hybrid architectures without the need to train large models from scratch, potentially accelerating progress in areas like content creation, dialogue systems, and automated reasoning.
For more details, see the Nemotron-Labs-TwoTower paper and the official NVIDIA announcement.
via MarkTechPost