Tencent open-sources AngelSpec: a unified training framework for speculative decoding
Tencent has released AngelSpec, an open-source, torch-native training framework for speculative-decoding draft models. The framework supports both autoregressive multi-token prediction (MTP) and the block-parallel DFlash family of draft architectures. As of mid-2026, the project is hosted on GitHub and ready for community contributions.
Existing speculative-decoding approaches typically train a single drafter to maximize performance on averaged benchmarks. However, real-world serving traffic rarely matches such averaged mixtures. AngelSpec treats workload heterogeneity as a first-class design constraint, specializing the draft model's structure, training data, and verification depth for the specific deployment context.
Why one universal drafter underperforms
Speculative decoding is lossless: a lightweight drafter proposes several future tokens, and the target model verifies them in a single forward pass using rejection sampling. Overall acceleration depends on two key factors: the number of accepted draft tokens and the duration of each complete draft–verify cycle.
In practice, serving workloads exhibit high variability in request patterns, token distributions, and latency requirements. A drafter optimized for average performance across a static benchmark suite may fail to deliver consistent speedups across heterogeneous traffic. AngelSpec addresses this by allowing practitioners to tailor draft models to their specific operational conditions.
AngelSpec: architecture and capabilities
AngelSpec is built on the Hy3 model architecture and provides a unified framework for training two distinct types of draft models:
- Autoregressive multi-token prediction (MTP): learns to predict multiple future tokens sequentially, improving acceptance rates through better draft quality.
- Block-parallel DFlash: generates draft tokens in parallel blocks, reducing latency by exploiting hardware parallelism.
The framework natively integrates with PyTorch, making it accessible to the existing deep learning ecosystem without requiring custom runtime modifications. By open-sourcing AngelSpec, Tencent aims to enable broader adoption of speculative decoding techniques and accelerate research into more efficient inference for large language models.
Implications for 2026 AI infrastructure
As large language models continue to scale in size and deployment footprint, inference efficiency becomes a critical operational concern. Speculative decoding offers a practical path to reduce latency and cost without sacrificing output quality. AngelSpec's focus on workload specialization aligns with the growing trend toward adaptive, context-aware inference systems. In 2026, we expect such frameworks to become standard building blocks in production AI infrastructure.
via MarkTechPost