Zyphra ZYFA: FP8 FlashAttention for AMD MI355X Delivers Up to 1.76x Speedup
By Michal Sutter | November 6, 2026
Zyphra has released ZYFA (Zyphra FlashAttention on AMD), a set of FP8 and BF16 prefill attention kernels for AMD Instinct MI355X GPUs. ZYFA is Zyphra's FP8 attention engine for long-context prefill, and the code is open source under Apache 2.0. On real model activations, its FP8 kernels run 1.16–1.76x faster than AITER, AMD's default kernel library. At head dimension 128, ZYFA's FP8 error is 62–78% lower than AITER's.
TL;DR
- Framework: A kernel library for causal forward (prefill) attention in FP8 e4m3 and BF16, supporting head dim 128 and the MLA shape (192/128).
- Runs on: AMD Instinct MI355X only (CDNA4, gfx950), ROCm 7.2.3, PyTorch 2.12. Drop-in
flashattnfuncinterface. - Performance: 1.16–1.76x AITER's FP8 throughput on real activations; BF16 is roughly on par (1.01–1.02x).
- Best case: 2,672 TFLOP/s FP8 at 32K tokens, or 53% of the FP8 datasheet peak (AITER: 39%).
- Worst case: Only 1.16x on Mistral-7B activations; FP8 clock drops to 1,628 MHz versus AITER's 2,108 MHz.
What Is ZYFA?
ZYFA is shipped in the form of a technical report accompanying the open-source release. In the context of the 2026 accelerator landscape, the MI355X's FP8 capabilities have become a focal point for long-context inference and training workloads, and ZYFA arrives as one of the more mature open kernel libraries targeting that hardware directly.
(Content truncated in source; remainder of the article was not provided.)
via MarkTechPost