#tree-based inference

TreeGraft: Adaptive Multi-Drafter Grafting for Tree-Based Speculative DecodingNEW⭐10

TreeGraft enables adaptive multi-drafter speculative decoding, boosting LLM inference speed by 15.1% across benchmarks.