NVIDIA Releases Nemotron-Labs-Sigma: A Continuous Diffusion

NVIDIA Releases Nemotron-Labs-Sigma: A Continuous Diffusion Language Model Family with 14 Checkpoints at 3B and 8B Scale


NVIDIA has released Nemotron-Labs-Sigma, a collection of 14 open checkpoints at 3B and 8B scale. Nemotron-Labs-Sigma is a continuous diffusion language model family that denoises text in a 16-D embedding space. Most diffusion LLMs, such as LLaDA and Dream, operate on discrete tokens. Sigma works in a smooth latent space instead. The NVIDIA team states that this design makes few-step distillation and trajectory steering practical for text.


TL;DR


  • Size: 3B and 8B parameters across 14 checkpoints. Context is 4,096 tokens (8,192 for Sigma-8B-instruct).
  • Runs on: 1 NVIDIA GPU in bfloat16 (Ampere, Lovelace, Hopper, Blackwell). Roughly 120 to 140 tokens/s on a single H100. No CPU support and no quantized variants listed.
  • Performance: Sigma-8B base averages 71.16 on NVIDIA's 6-task suite, ahead of NLD-8B (70.22).
  • Best result: 91.96 on GSM8K (Sigma-8B-instruct, 256 DDPM steps).

via MarkTechPost

Related