Polymathic AI and Princeton Release FlashMap: A Flow-Map Drafter for 6.14x Faster LLM Decoding
By Asif Razzaq β October 28, 2026
Researchers from Polymathic AI, Princeton University, USC ISI, NYU, and the Flatiron Institute have released FlashMap, a speculative decoding method that replaces the discrete diffusion drafter with a continuous flow map. FlashMap drafts an entire 16-token block in a single forward pass, with all tokens in the block kept mutually consistent. On Qwen3-8B, the team reports up to 6.14x lossless speedup on GSM8K. Drafter weights for Qwen3-8B and Qwen3-4B are available on Hugging Face, alongside vLLM and SGLang plugins.
TL;DR
- Size: 0.5B-parameter drafter (500M for Qwen3-8B, 475M for Qwen3-4B); drafts blocks of K = 16 tokens.
- Runs on: the same GPU as the target model. Paper speedups use 1 NVIDIA H200 NVL at concurrency 1, in bf16.
- Performance: fastest drafter tested on Qwen3-8B, about 7% above a retrained DFlash on average.
- Best case: 6.14x speedup on GSM8K at temperature 0 (acceptance length 7.69 tokens).
- Worst case: 2.28x speedup on Alpaca at temperature 1. On Qwen3-4B (temperature 0), it trails DFlash on 4 of 9 tasks.
Why It Matters in 2026
As LLM inference costs continue to dominate enterprise AI budgets, speculative decoding has become a standard lever for latency reduction. Most production drafters still rely on discrete diffusion or autoregressive drafting, which trade off block consistency against throughput. FlashMap's continuous flow-map formulation sidesteps that trade-off, delivering multi-token drafts in one pass while keeping the block internally coherentβan approach that fits naturally with the vLLM and SGLang serving stacks now widely deployed for Qwen3-class models.
Resources
- Paper: arXiv:2610.24365
- Weights: Hugging Face β FlashMap collection
- Code: GitHub β vLLM and SGLang plugins
via MarkTechPost