mirror of
https://github.com/leejet/stable-diffusion.cpp.git
synced 2026-11-10 19:06:33 +00:00
perf: accelerate INT8 diffusion inference with CUDA fusions (#2235)
This commit is contained in:
1 parent
8cbe905fdd
commit
77b72a2efe
12 files changed
+113
-128
No files matched your search
+28
-4
@@ -2,18 +2,23 @@
|
||||
|
||||
## Get the Code
|
||||
|
||||
```
|
||||
```shell
|
||||
git clone --recursive https://github.com/leejet/stable-diffusion.cpp
|
||||
cd stable-diffusion.cpp
|
||||
```
|
||||
|
||||
- If you have already cloned the repository, you can use the following command to update the repository to the latest code.
|
||||
If you already cloned the repository without submodules, initialize them from the repository root:
|
||||
|
||||
```shell
|
||||
git submodule update --init --recursive
|
||||
```
|
||||
|
||||
To update an existing checkout and its submodules:
|
||||
|
||||
```shell
|
||||
cd stable-diffusion.cpp
|
||||
git pull origin master
|
||||
git submodule init
|
||||
git submodule update
|
||||
git submodule update --init --recursive
|
||||
```
|
||||
|
||||
## Selecting a GGML source tree
|
||||
@@ -102,6 +107,25 @@ cmake .. -DSD_CUDA=ON
|
||||
cmake --build . --config Release
|
||||
```
|
||||
|
||||
CUDA builds with the bundled patched GGML use the CUTLASS submodule by default
|
||||
for eligible INT8 GEMM and CuTe attention operations. If its headers are missing,
|
||||
CMake stops with an initialization hint; it does not download them automatically.
|
||||
Run `git submodule update --init --recursive` from the repository root before
|
||||
configuring.
|
||||
|
||||
To build with the existing CUDA implementations without CUTLASS, configure from
|
||||
the repository root with:
|
||||
|
||||
```shell
|
||||
cmake -S . -B build -DSD_CUDA=ON -DGGML_CUDA_CUTLASS_PATH:PATH=
|
||||
cmake --build build --config Release
|
||||
```
|
||||
|
||||
Alternatively, set `GGML_CUDA_CUTLASS_PATH` to another local CUTLASS source
|
||||
directory (tested with 3.9.2). Existing CMake caches retain their configured path;
|
||||
pass `-UGGML_CUDA_CUTLASS_PATH` when reconfiguring to restore the bundled default.
|
||||
Non-CUDA builds do not require CUTLASS.
|
||||
|
||||
## Build with HipBLAS
|
||||
|
||||
This provides GPU acceleration using AMD GPU. Make sure to have the ROCm toolkit installed.
|
||||
|
||||
Reference in new issue
Block a user