perf: accelerate INT8 diffusion inference with CUDA fusions (#2235)

This commit is contained in:
leejet authored and GitHub committed 2026-10-28 22:02:46 +08:00
1 parent 8cbe905fdd
commit 77b72a2efe
12 files changed
+113 -128

No files matched your search

+28 -4
View File
@@ -2,18 +2,23 @@
## Get the Code
```
```shell
git clone --recursive https://github.com/leejet/stable-diffusion.cpp
cd stable-diffusion.cpp
```
- If you have already cloned the repository, you can use the following command to update the repository to the latest code.
If you already cloned the repository without submodules, initialize them from the repository root:
```shell
git submodule update --init --recursive
```
To update an existing checkout and its submodules:
```shell
cd stable-diffusion.cpp
git pull origin master
git submodule init
git submodule update
git submodule update --init --recursive
```
## Selecting a GGML source tree
@@ -102,6 +107,25 @@ cmake .. -DSD_CUDA=ON
cmake --build . --config Release
```
CUDA builds with the bundled patched GGML use the CUTLASS submodule by default
for eligible INT8 GEMM and CuTe attention operations. If its headers are missing,
CMake stops with an initialization hint; it does not download them automatically.
Run `git submodule update --init --recursive` from the repository root before
configuring.
To build with the existing CUDA implementations without CUTLASS, configure from
the repository root with:
```shell
cmake -S . -B build -DSD_CUDA=ON -DGGML_CUDA_CUTLASS_PATH:PATH=
cmake --build build --config Release
```
Alternatively, set `GGML_CUDA_CUTLASS_PATH` to another local CUTLASS source
directory (tested with 3.9.2). Existing CMake caches retain their configured path;
pass `-UGGML_CUDA_CUTLASS_PATH` when reconfiguring to restore the bundled default.
Non-CUDA builds do not require CUTLASS.
## Build with HipBLAS
This provides GPU acceleration using AMD GPU. Make sure to have the ROCm toolkit installed.