mirror of
https://github.com/leejet/stable-diffusion.cpp.git
synced 2026-11-11 11:26:34 +00:00
feat: support disk params backend (#1651)
This commit is contained in:
1 parent
276025e054
commit
bdb431ad95
27 files changed
+134
-65
No files matched your search
+33
-1
@@ -21,6 +21,38 @@ and the compute buffer shrink in the debug log:
|
||||
|
||||
Using `--offload-to-cpu` allows you to offload weights to the CPU, saving VRAM without reducing generation speed.
|
||||
|
||||
## Use params backend to reduce VRAM or RAM usage.
|
||||
|
||||
`--params-backend` controls where model parameters are kept. If it is not set, parameters use the same backend as `--backend`, so a GPU runtime backend also keeps parameters in VRAM.
|
||||
|
||||
Use CPU params to reduce VRAM usage:
|
||||
|
||||
```shell
|
||||
--backend cuda0 --params-backend cpu
|
||||
```
|
||||
|
||||
This keeps model weights in system RAM and moves them to the runtime backend when needed. `--offload-to-cpu` is a compatibility shortcut that prepends `*=cpu` to `--params-backend`, so explicit module assignments can still override it:
|
||||
|
||||
```shell
|
||||
--offload-to-cpu --params-backend te=disk
|
||||
```
|
||||
|
||||
Use disk params to reduce both VRAM and RAM usage:
|
||||
|
||||
```shell
|
||||
--backend cuda0 --params-backend disk
|
||||
```
|
||||
|
||||
This reloads parameters from the model file on demand and releases them after use. It has the lowest memory residency, but can be slower because weights must be read again. `disk` is never selected implicitly; set it explicitly when RAM usage matters more than reload cost.
|
||||
|
||||
Per-module assignments can target only the largest modules:
|
||||
|
||||
```shell
|
||||
--backend cuda0 --params-backend diffusion=disk,te=cpu,vae=cpu
|
||||
```
|
||||
|
||||
See [backend selection](./backend.md) for full syntax.
|
||||
|
||||
## Use quantization to reduce memory usage.
|
||||
|
||||
[quantization](./quantization_and_gguf.md)
|
||||
[quantization](./quantization_and_gguf.md)
|
||||
Reference in new issue
Block a user