aiqbench takeAI Products

Nunchux AI's VC-Attention addresses critical bottlenecks in video Diffusion Transformers by combining value rotation and mean subtraction to enable efficient low-bit quantization without retraining. This approach significantly reduces the computational overhead of the softmax stage and mitigates quantization errors, offering a practical path toward faster generation on current hardware like the RTX 5090 and B200.

Source coveringMarkTechPost

What this isA model's reading of the article and related coverage, written for aiqbench. Not the publisher's words.

Nunchux AI Introduces VC-Attention: A Training-Free Low-Bit Attention Kernel That Speeds Up Video Diffusion Transformers

MarkTechPostSeptember 17, 20264 min read

Nunchux AI has released VC-Attention, a training-free low-bit attention kernel built for video Diffusion Transformers (DiTs). It targets 2 problems at once: value quantization error and a slow softmax stage. Why Attention is the Video Bottleneck Video DiTs flatten a clip into 1 sequence of spatiotemporal tokens and run full self-attention at every layer. A […] The post Nunchux AI Introduces VC-Attention: A Training-Free Low-Bit Attention Kernel That Speeds Up Video Diffusion Transformers appeared first on MarkTechPost.

Nunchux AI has released VC-Attention, a training-free low-bit attention kernel built for video Diffusion Transformers (DiTs). It targets two problems at once: value quantization error and a slow softmax stage. Video DiTs flatten a clip into one sequence of spatiotemporal tokens and run full self-attention at every layer. A 5-second 720p Wan2.2-14B clip spans about 70K tokens. On the RTX 5090, attention takes more than 64% of generation time. The research team states that attention is about two thirds of…

Excerpt shown under the publisher's RSS terms · 80 words · full text and figures are on the publisher's site

Read at MarkTechPost ↗Ask about this article