analiza aiqbench
Tencent's AngelSpec addresses the critical challenge of speculative decoding efficiency by decoupling drafter architectures to match specific workload characteristics, such as separating high-entropy conversation from structured code reasoning. By integrating adaptive verification budgeting and hybrid target conditioning, this framework offers a practical solution for accelerating inference on large-scale models like Hy3 without compromising accuracy. This advancement represents significant progress in optimizing the computational overhead of next-token prediction strategies essential for deploying AI at scale.
Tencent Open-Sources AngelSpec: A Unified Training Framework for MTP and Block-Parallel Speculative Decoding on Hy3 Models
Tencent has released AngelSpec, an open-source torch-native framework for training speculative-decoding draft models across six architectures. It introduces DFly, a block-diffusion drafter with hybrid target conditioning and a hidden-correction autoregressive head, and integrates D-cut for runtime-adaptive verification budgeting. On HY3-295B-A21B with TP=8, DFly-8 delivers a 1.98–2.40× speedup over autoregressive decoding across concurrency 4 to 64. The post Tencent Open-Sources AngelSpec: A Unified Training Framework for MTP and Block-Parallel Speculative Decoding on Hy3 Models appeared first on MarkTechPost.
Tencent has released AngelSpec, an open-source, torch-native training framework for speculative-decoding draft models. The release covers both autoregressive multi-token prediction (MTP) and the block-parallel DFlash family. Most speculative-decoding work searches for one drafter that scores well on an averaged benchmark mixture. Real serving traffic does not look like that mixture. AngelSpec treats workload heterogeneity as a first-class design constraint, and specializes structure, training data, and verification depth around it. Speculative decoding is lossless. A lightweight drafter proposes several future tokens, and the target model verifies them together in one forward pass using rejection sampling. Acceleration then depends on two things: how many draft tokens get accepted, and how long the complete draft–verify round takes. Those two quantities move in opposite directions…
Extras afișat conform termenilor RSS ai publicației · 121 cuvinte · textul integral și figurile sunt pe site-ul publicației
