NVIDIA has unveiled NemotronLabs VoiceChat 11B, an open-source 11-billion-parameter end-to-end speech-to-speech model designed for real-time, full-duplex conversations. Unlike traditional cascaded pipelines that chain automatic speech recognition (ASR), a large language model (LLM), and text-to-speech (TTS), this model unifies streaming speech understanding and generation within a single network. By eliminating multi-model orchestration and API handoffs, it significantly reduces end-to-end latency—achieving a measured smooth turn-taking latency of 448 milliseconds on the Full-Duplex-Bench 1.0 benchmark. The model can listen while speaking, allowing users to barge in mid-turn; the agent yields promptly, with a take-over rate of 1.00 at 480 ms. Notably, it is the first open full-duplex model to support live tool calling during conversation, using a dedicated output channel for
Is It Deployable?
Partial—suitable for pilots today, but not yet for production. The model weights and container are publicly available under a permissive license. However, NVIDIA states that the checkpoint is 'ready for research purposes only.' The repository documents several real-world limitations: a two-minute audio context ceiling, potential degradation into non-recoverable gibberish after multiple turns, runaway self-talk at the end of a turn, and dropped words during user transcription. These constraints mean teams should evaluate the model carefully in controlled environments before considering broader deployment.
In the context of 2026, this release signals NVIDIA’s push to make conversational AI more immersive and efficient. Full-duplex speech models like this one promise to transform customer service, voice assistants, and real-time translation by reducing the awkward pauses inherent in turn-based systems. Yet, the noted failure modes highlight the ongoing challenge of balancing low latency with stability in open-ended dialogue. As research progresses, we can expect iterative improvements to address these issues, paving the way for production-ready systems in the near future.
via MarkTechPost