Understanding Attention Algorithms and Their Backends for Image and Video Generation

ComfyUI has become a widely adopted tool for running modern generative AI workloads, from text-to-image and video generation to more advanced pipelines like WAN2.2. Its node-based workflow makes it accessible without requiring deep coding knowledge, but under the hood it still relies heavily on one of the most important building blocks in modern AI models: attention.

July 20, 2026 by Aleksa Arsic, Filip Jankovic, Liang Shen, Hosang Yoon.

17 min read. | 4126 total words.

ComfyUI has become a widely adopted tool for running modern generative AI workloads, from text-to-image and video generation to more advanced pipelines like WAN2.2. Its node-based workflow makes it accessible without requiring deep coding knowledge, but under the hood it still relies heavily on one of the most important building blocks in modern AI models: attention.

For users running on AMD GPUs with ROCm™, choosing the right attention implementation can have a significant impact on performance, memory usage, and stability. However, this choice is far from straightforward. ComfyUI exposes multiple attention flags, each mapping to different algorithms and backend implementations (AOTriton, CK Tile, rocBLAS/hipBLASLt, Triton), often with non-obvious priority rules and interactions.

This blog breaks down these layers and offers a practical guide to help you make informed decisions when running ComfyUI for image and video generation on Linux-based systems with AMD RDNA™ architecture GPUs.

For background and setup across AMD platforms, refer to the earlier ROCm blogs Running ComfyUI on AMD Instinct GPU, Getting Started with ComfyUI on AMD Radeon™ RX 9000 Series GPUs, and Running ComfyUI in Windows with ROCm on WSL.

Safe, well-integrated, automatically selects optimized AOTriton backend. Note: CK backend is experimentally enabled for the purposes of this blog. Officially, it is not available for RDNA architectures (see –use-pytorch-cross-attention chapter).

In our benchmarks, typically delivered the highest throughput on the tested configurations.

Keep disabled or explicitly override.

xformers has higher priority and may silently override other choices. Note: Not stable (see --disable-xformers chapter)

Reduces memory footprint by avoiding full attention computation.

Most stable but slowest path.

Flags that explicitly disable xformers. For example, xformers will override the --use-flash-attention flag.

PyTorch SDPA is the safest default on AMD due to AOTriton integration.

Flash Attention (CK Tile) typically gives optimal raw performance.

xformers can silently override your choice, so be explicit when needed.

Split/quad attention trades performance for memory safety.

These choices can significantly impact generation throughput, memory usage, and overall stability. In some cases, selecting the right attention backend can significantly impact performance on AMD GPUs.

Note: PyTorch SDPA (CK) and the modified PyTorch/AOTriton builds used in this blog are experimental and not officially supported on RDNA at the time of writing.

At a high level, selecting an attention implementation in ComfyUI is not determined solely by the CLI flag. The final implementation depends on ComfyUI priority rules, installed packages, AMD/ROCm availability checks, and backend dispatch.

Available Out-of-the-box?

Requires sageattention package. Not officially supported on AMD. On gfx12xx, additional patching may be required due to Triton compilation issues. In the measured workloads, Sage Attention was generally slower than PyTorch SDPA on Navi31 and Strix-Halo, and had stability issues on Navi48.

High-priority path. If installed, xformers may be selected automatically and can silently override other choices. It should be handled carefully because it was observed to be unstable in some tested scenarios.

Requires DAO-AILab/flash-attention package. In the benchmarks, CK-backed Flash Attention delivered the highest throughput in most cases. Make sure xformers is disabled or bypassed when you want this path to be selected.

AOTriton by default; CK experimentally possible with TORCH_ROCM_FA_PREFER_CK=1 or torch.backends.cuda.preferred_rocm_fa_library("ck")

Safest and most integrated option on AMD. In ComfyUI, --use-pytorch-cross-attention also bypasses xformers for cross attention. CK-backed PyTorch SDPA was experimentally enabled for this analysis but is not officially supported for RDNA architectures.

Computes attention in slices to reduce memory pressure. This trades performance for memory safety and also suppresses AMD auto-detection of PyTorch/AOTriton attention.

Lowest-priority path. Useful when other optimized attention implementations are unavailable or unsuitable, but generally not the performance-oriented choice.

Attention flag selects a high-level algorithm (e.g. PyTorch SDPA, Flash Attention).

The algorithm maps to a backend implementation (AOTriton, CK Tile, Triton, rocBLAS/hipBLASLt).

The backend determines performance characteristics such as throughput and memory usage.

Understanding this mapping is important because the CLI flag ultimately determines which attention algorithm and which ROCm backend is used, directly impacting performance.

These options are mutually exclusive, and their behavior depends on installed libraries and internal priority rules. For example, --use-sage-attention requires sageattention package to be installed, while --use-flash-attention needs corresponding DAO-AILab/flash-attention. Also, Linux supports all of the listed backends, while some of them might have limited support on Windows.

python ComfyUI/main.py --use-\*-attention

Apart from that, there are settings for controlling attention precision and up casting. These won’t be covered in this blog:

ComfyUI chooses the global attention implementation based on the priority order in ComfyUI/comfy/ldm/modules/attention.py at master · Comfy-Org/ComfyUI:

--use-pytorch-cross-attention (overrides xformers) OR auto-detected: supported arch + PyTorch ≥ 2.7 + AOTriton present + no --use-split/quad flag

The VAE decoder follows a separate attention selection path: ComfyUI/comfy/ldm/modules/diffusionmodules/model.py at master · Comfy-Org/ComfyUI. On AMD GPUs, PyTorch SDPA is intentionally disabled for VAE due to stability issues at high resolutions. As a result, VAE attention typically falls back to xformers (if available) or split attention.

ROCm xformers installed and if not disabled via --disable-xformers. Falls back to slice_attention on NotImplementedError: ComfyUI/comfy/ldm/modules/diffusionmodules/model.py at master · Comfy-Org/ComfyUI.

Fallback when PyTorch attention or xformers is disabled. Uses Batched matrix multiplication implementation (falls back to rocBLAS/hipBLASLt)

ComfyUI will automatically enable PyTorch SDPA attention with compatible PyTorch/ROCm versions: ComfyUI/comfy/model_management.py at master · Comfy-Org/ComfyUI.

FP8 operations are enabled for newer AMD architectures: ComfyUI/comfy/model_management.py at master · Comfy-Org/ComfyUI

TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 is set by default within ComfyUI: ComfyUI/main.py at master · Comfy-Org/ComfyUI, which enables experimental memory efficient attention (backed by AOTriton on AMD). In practice we need to pass –use-pytorch-cross-attention to get this behavior.

--use-split-cross-attention and --use-quad-cross-attention suppress AMD auto-detection of PyTorch/AOTriton attention.

Without xformers and without a supported architecture, the default path is sub-quadratic attention.

Flash attention (AOTriton, CK) (AOTriton by default, or CK if TORCH_ROCM_FA_PREFER_CK=1 or torch.backends.cuda.preferred_rocm_fa_library("ck") is set)

Memory efficient attention (AOTriton, CK) (AOTriton by default, or CK if TORCH_ROCM_FA_PREFER_CK=1 or torch.backends.cuda.preferred_rocm_fa_library("ck") is set)

Pure PyTorch implementation (e.g. batched matrix multiplication - falls back to rocBLAS/hipBLASLt)

PyTorch SDPA Flash attention and Memory efficient attention share the same backends for AMD/ROCm, essentially making them the same.

With Attention priority and Attention backends we have covered all ComfyUI attention algorithms from a high-level perspective. Therefore, the following paragraphs deep dive into the attention algorithms behind different CLI flags by following the Attention priority discussed previously. Also, ComfyUI AMD/ROCm specifics will be covered.

Note: sageattention works out-of-the-box for gfx1100 and gfx1151. For gfx12xx the patch needs to be applied as there are triton compilation errors. These are known bugs:

Sage attention is a quantization technique that boosts the transformer attention performance by using INT8 quantization and matrix smoothing.

Currently, sageattention is not an officially supported package from AMD, it is limited to CUDA stack and should be used for experimental purposes. There are community and partial AMD efforts to enable the Triton route through Sage attention.

Passing low_precision_attention=False, will bypass sage-attention and fallback to attention_pytorch and if sage-attention raises any exception, it logs an error and falls back to attention_pytorch.

Note: For mGPU T2V WAN2.2 workload, xformers produces Memory access faults on Navi48 VAE Decoder phase.

If xformers library is detected, it will be used automatically if not disabled by this flag. Therefore, no special activation flag is needed to use xformers as based on the priority chain, ComfyUI selects xformers second (after sage attention).

Attention_xformers calls memory-efficient attention and lets the xformers library determine which ROCm library or backend is used. (ComfyUI/comfy/ldm/modules/attention.py at master · Comfy-Org/ComfyUI). Also, –use-pytorch-cross-attention explicitly sets XFORMERS_IS_AVAILABLE = False (ComfyUI/comfy/model_management.py at master · Comfy-Org/ComfyUI) and it is the flag that actively disables xformers.

xformers is not available out-of-the-box. Therefore, to install it, follow the install from source instructions from the xformers repository: ROCm/xformers: Hackable and optimized Transformers building blocks, supporting a composable construction.

This flag will utilize Flash attention 2 algorithm described in the following paper: FlashAttention-2: Faster Attention with Better Parllelism and Work Partitioning. Usually, this implementation produces strong results on AMD GPUs.

To build DAO-AILab/flash-attention, follow the Readme.md from the official DAO-AILab/flash-attention repository: Dao-AILab/flash-attention.

Building DAO-AILab/flash-attention without FLASH_ATTENTION_TRITON_AMD_ENABLE provides the option to toggle between CK Tile and Triton FA with FLASH_ATTENTION_TRITON_AMD_ENABLE=TRUE.

For peak throughput, enable FLASH_ATTENTION_TRITON_AMD_AUTOTUNE="TRUE" to search for optimal settings, which incurs a one-time warmup cost. Alternatively, if not autotuning, FLASH_ATTENTION_FWD_TRITON_AMD_CONFIG_JSON may be used to set a single triton config overriding the hardcoded defaults for attn_fwd.

Both Flash Attention and Memory Efficient Attention dispatch to the same ROCm kernel library, which is AOTriton by default or CK if TORCH_ROCM_FA_PREFER_CK=1 or torch.backends.cuda.preferred_rocm_fa_library("ck") is set. Math, or pure C++ implementation, is a PyTorch SDPA that does not use kernel fusion, tiling, or memory optimizations. Currently, CK PyTorch cross attention is not supported for RDNA architectures. Flash Attention is not to be confused with the DAO-AILab/flash-attention package. Even though they share a similar name, Flash Attention in this context refers to PyTorch SDPA Flash Attention 2 implementation.

In this analysis, we used an internal experimental build of PyTorch 2.10 with CK-enabled SDPA on Navi31, Navi48, and Strix-Halo. This configuration is not officially supported on RDNA architectures and is used here solely for experimental comparison.

Also, for this blog, we evaluated a newer Triton version within AOTriton (0.11.1b) and compared it against the official AOTriton configuration. Specifically, we used the Triton version from AOTriton 0.12 (ROCm/triton at aotriton/0.12), as our experiments showed that upgrading the Triton dependency can provide measurable performance benefits.

This is ComfyUI’s pure PyTorch attention implementation that helps avoid out‑of‑memory (OOM) errors by computing attention in slices rather than all at once: ComfyUI/comfy/ldm/modules/attention.py at master · Comfy-Org/ComfyUI. It is one of the last resorts in the priority chain and should be used when there is a need for reduced memory pressure. Split cross-attention is available out-of-the-box without the need to install any additional packages.

On AMD/ROCm, PyTorch dispatches the underlying batched matrix multiplications (e.g., einsum) down through its own ROCm backend to rocBLAS/hipBLASLt. Also, for AMD, passing this flag suppresses the auto-enabling of PyTorch attention (ENABLE_PYTORCH_ATTENTION) on supported arches: ComfyUI/comfy/model_management.py at master · Comfy-Org/ComfyUI.

--use-quad-cross-attention activates sub-quadratic attention, which is an implementation of the algorithm from the paper “Self-attention Does Not Need O(n²) Memory”. Sub-quadratic attention is the default fallback when none of sage/xformers/flash/pytorch/split attention are active or available and should be used when split cross-attention produces out-of-memory issues. Quad cross-attention is available out-of-the-box without need to install any additional packages.

For AMD, passing this flag suppresses the auto-enabling of PyTorch attention (ENABLE_PYTORCH_ATTENTION) on supported arches: ComfyUI/comfy/model_management.py at master · Comfy-Org/ComfyUI.

As with split-cross-attention, the quad-cross-attention will be dispatched to rocBLAS/hipBLASLt as it is using PyTorch batched matrix multiplication functions.

DISCLAIMER: Performance results are based on AMD testing using the WAN2.2 workflows, hardware platforms, software versions, and configurations described in this article. Actual results will vary depending on model, workload, software stack, system configuration, backend selection, and optimization settings.

In our measurements, Flash attention with CK delivered the highest throughput on the tested workloads.

Differences are more pronounced on larger models and unified-memory systems (e.g., Strix Halo)

In these tests, RDNA4 (Navi48) delivered roughly ~60% higher throughput than RDNA3 (Navi31) under the same workload and backend.

comfyui_txt2video_Wan2.2-Lightning-T2V-A14B-4steps-lora-HIGH-fp16

comfyui_image2video_wan2.2-i2v-scale-fp8-lightx-workflow This workflow needs to be modified for the ComfyUI to be able to change the FA algorithm. The default value is hardcoded to SDPA, change it to “comfy” to control the FA with --use-flash-attention, change it to “sageattn” to use it with --use-sage-attention.

comfyui_image2video_wan2.2-i2v-scale-fp8-lightx-workflow

This workflow needs to be modified for the ComfyUI to be able to change the FA algorithm. The default value is hardcoded to SDPA, change it to “comfy” to control the FA with --use-flash-attention, change it to “sageattn” to use it with --use-sage-attention.

comfyui_txt2video_Wan2.2-Lightning-T2V-A14B-4steps-lora-HIGH-fp16#

The figures below show the performance of different Flash Attention backends on Navi 31, Navi 48, and Strix Halo (128 GB), including throughput and percentage throughput differences relative to PyTorch AOTriton SDPA for the WAN 2.2 T2V workload:

Throughput percentage difference (vs. AOTriton SDPA) [%]

Sage Attention (1.0.6)

comfyui_image2video_wan2.2-i2v-scale-fp8-lightx-workflow#

The figures below show the performance of different Flash Attention backends on Navi 31, Navi 48, and Strix Halo (128 GB), including throughput and percentage throughput differences relative to PyTorch AOTriton SDPA for the WAN 2.2 I2V workload:

Throughput percentage difference (vs. AOTriton SDPA) [%]

Sage Attention (1.0.6)

Note: Entries in the tables in parentheses “(XY)” are the numbers from cold start runs, while the numbers without parentheses “XY” are the numbers from the second run with the warm cache.

At a high level, performance is driven by three factors: hardware generation, backend selection, and workload constraints. Therefore, these results provide a useful snapshot of how both hardware evolution and software maturity impact attention-heavy diffusion workloads on AMD platforms.

Hardware scaling (from RDNA3 to RDNA4). Keeping the workload and backend constant, RDNA4 (Navi48) delivers roughly +60% higher throughput compared to RDNA3 (Navi31). This is the clearest signal in the data: newer GPU architectures translate directly into meaningful gains for attention-dominated workloads, independent of software improvements.

Software stack maturity. Even on specific hardware, moving across different attention backends and software versions results in significant performance differences. Depending on the configuration, throughput improves by:

This highlights that the ROCm attention stack is still evolving rapidly, with backend selection playing a major role in overall performance.

Composable Kernels (CK) as the winner. Across nearly all tested configurations, CK-backed implementations (either via PyTorch SDPA-CK or DAO-AILab Flash Attention with CK) were typically the fastest in our measurements. This consistency confirms that hand-tuned kernel implementations remain the dominant factor for peak performance.

Strix Halo: trading throughput for capacity. While Strix Halo shows approximately 3× lower raw throughput compared to Navi48, which is expected due to its APU design, it enables workloads that would otherwise not fit on discrete GPUs, thanks to its 128GB unified memory. In this context, the key metric shifts from how fast it runs to whether it runs at all. Importantly, software improvements still scale well here, with up to +50% gains when comparing AOTriton with CK, indicating that the same optimization trends apply even in memory-bound scenarios.

Current performance insights and optimization opportunities. The results highlight several opportunities for continued optimization and refinement:

Cold-start latency (3–11 minutes on Navi48), primarily driven by MIOpen kernel database tuning rather than steady-state execution performance. Thankfully, warm database runs eliminate most of this overhead, indicating strong runtime efficiency once the system is initialized.

Sage Attention shows room for further tuning in this context, suggesting opportunities to improve its performance for these workloads or explore alternative optimizations.

The observed performance of the bumped AOTriton on Strix-Halo highlights an interesting area for deeper analysis, which may unlock additional performance gains with further investigation.

In this blog, you learned how ComfyUI attention algorithms map to ROCm backends and how those backend choices influence performance, memory usage, and stability on AMD GPUs. You also learned how to select optimal attention configuration for your workload using practical recommendations, implementation comparisons, and benchmark-driven performance analysis. Across all measured configurations, the dominant driver of throughput is the choice of backend (CK > AOTriton-bumped > AOTriton); silicon generation (RDNA4 over RDNA3) compounds those gains.

While multiple attention paths are available, their behavior is shaped by a combination of ComfyUI priority rules, installed libraries, and the underlying ROCm backend. In practice, this means that seemingly small configuration choices, such as leaving xformers enabled, can significantly affect which attention implementation is actually used.

PyTorch SDPA (AOTriton) provides the most reliable and easiest-to-use default, with good performance.

PyTorch SDPA (CK) provides mixed results. In the case of I2V workload the results are similar or even the same as DAO-AILab/flash-attention (CK backend), while in the case of T2V it even performs slightly better than DAO-AILab/flash-attention (CK backend) on Navi48. Nevertheless, it is not yet officially supported.

DAO-AILab/flash-attention (CK backend) delivers the highest throughput, with improvements ranging from ~20% up to 40–50% depending on the model and hardware.

xformers must be handled carefully, as it is not stable and can silently override other attention choices.

Split and sub-quadratic attention remain important fallbacks for memory-constrained scenarios, trading performance for stability.

Sage attention produces significantly worse results than PyTorch SDPA (AOTriton) for Navi31 and Strix-Halo and therefore is not recommended. Additionally, on gfx12xx there are known issues with the sageattention package and thu-ml/SageAttention#365 needs to be included. Nevertheless, Sage attention, in certain cases, outperforms only PyTorch SDPA (AOTriton).

These findings highlight an important takeaway: the CLI flag alone does not determine performance - the full stack, from flag to backend, must be considered.

Use --use-pytorch-cross-attention for a stable, well-integrated default.

Use --use-flash-attention (with xformers disabled) when maximizing throughput.

Use split or sub-quadratic attention when memory pressure becomes a limiting factor.

Ultimately, optimal configuration still depends on the specific workload, model size, and GPU architecture. For optimal results, benchmarking on your target setup remains essential.

Note: CK-based PyTorch SDPA and modified AOTriton configurations used here are experimental and not officially supported on RDNA architectures at the time of writing.

Performance results are based on AMD testing using the WAN2.2 workflows, hardware platforms, software versions, and configurations described in this article. Actual results will vary depending on model, workload, software stack, system configuration, backend selection, and optimization settings. The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions, and typographical errors. The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. Any computer system has risks of security vulnerabilities that cannot be completely prevented or mitigated. AMD assumes no obligation to update or otherwise correct or revise this information. However, AMD reserves the right to revise this information and to make changes from time to time to the content hereof without obligation of AMD to notify any person of such revisions or changes. THIS INFORMATION IS PROVIDED ‘AS IS.” AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS, OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION. AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE LIABLE TO ANY PERSON FOR ANY RELIANCE, DIRECT, INDIRECT, SPECIAL, OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. AMD, the AMD Arrow logo, AMD Radeon™, Radeon™, RDNA™, AMD Instinct™, Radeon Instinct™, CDNA, ROCm™ and combinations thereof are trademarks of Advanced Micro Devices, Inc. Other product names used in this publication are for identification purposes only and may be trademarks of their respective companies. Linux® is the registered trademark of Linus Torvalds in the U.S. and other countries. Python is a trademark of the Python Software Foundation. PyTorch, the PyTorch logo and any related marks are trademarks of The Linux Foundation. © 2026 Advanced Micro Devices, Inc. All rights reserved

Third-party content is licensed to you directly by the third party that owns the content and is not licensed to you by AMD. ALL LINKED THIRD-PARTY CONTENT IS PROVIDED “AS IS” WITHOUT A WARRANTY OF ANY KIND. USE OF SUCH THIRD-PARTY CONTENT IS DONE AT YOUR SOLE DISCRETION AND UNDER NO CIRCUMSTANCES WILL AMD BE LIABLE TO YOU FOR ANY THIRD-PARTY CONTENT. YOU ASSUME ALL RISK AND ARE SOLELY RESPONSIBLE FOR ANY DAMAGES THAT MAY ARISE FROM YOUR USE OF THIRD-PARTY CONTENT.

Text integral sub licența deschisă a publicației · 4177 cuvinte

analiza aiqbench

Understanding Attention Algorithms and Their Backends for Image and Video Generation

Subiecte conexe