mirror of
https://github.com/leejet/stable-diffusion.cpp.git
synced 2026-11-10 19:06:33 +00:00
feat: support packed INT4 convrot models (#2276)
Co-authored-by: noctrex <noctrex@gmail.com>
This commit is contained in:
1 parent
58bb3c1b1c
commit
7952693004
48 files changed
+2003
-102
No files matched your search
@@ -104,6 +104,16 @@ and backend `graph_plan_*` interfaces do not dispatch these extensions.
|
||||
|
||||
## Backend services
|
||||
|
||||
Packed ComfyUI INT4 linear layers use `sd.linear_w4_convrot.v1`, built by
|
||||
`ggml_mul_mat_w4_convrot`. The operator consumes byte-packed I8 weights and the
|
||||
explicit output of `sd.quantize_i8_convrot.v1`; F32 callers get a quantization
|
||||
node during graph construction. Logical output dimensions are stored in the
|
||||
operator attributes, so backend copies of the packed activation do not need to
|
||||
retain the producer's op marker. W4A4 uses row scales; W4A8 additionally exposes
|
||||
the codebook, channel scales, and FP8 relative scales as tensor dependencies.
|
||||
CPU, CUDA/HIP/MUSA, and Vulkan register kernels through the same extension API
|
||||
as INT8. No ggml submodule changes are required.
|
||||
|
||||
Services are discovered through `ggml_backend_reg_get_proc_address`:
|
||||
|
||||
| Service | Purpose |
|
||||
|
||||
Reference in new issue
Block a user