feat: add per-module automatic flash attention selection (#2132)

This commit is contained in:
leejet authored and GitHub committed 2026-10-13 22:58:45 +08:00
1 parent 60423ab5f6
commit d763dae3b8
53 files changed
+805 -125

No files matched your search

+1 -2
View File
@@ -40,8 +40,7 @@ the intended output scale:
- `--linear-scale` scales Linear inputs before matrix multiplication and rescales
the result.
- `--attn-scale` scales attention keys and values (K/V). It takes effect only in
the Flash Attention path, where `--fa` or `--diffusion-fa` is enabled and the
backend supports it.
the Flash Attention path, when selected and supported by the backend.
The two values can be set independently and apply across model components. The
default `0` preserves each model's built-in settings; `1` explicitly disables the