mirror of
https://github.com/leejet/stable-diffusion.cpp.git
synced 2026-11-10 19:06:33 +00:00
feat: add Fibo 2.0 Dev and Edit Turbo support (#2180)
This commit is contained in:
1 parent
939895f00e
commit
2e2ba3faec
22 files changed
+931
-69
No files matched your search
+11
-1
@@ -19,6 +19,7 @@ Depending on the architecture, different models handle reference images differen
|
||||
| **Krea2 (Community Edit LoRAs)** | `krea2_ostris_edit` |
|
||||
| [**Mage-Flow-Edit**](./mage_flow.md#image-editing) | `mage_flow` |
|
||||
| [**LLaDA-Image**](./llada_image.md#image-editing) | `llada_image` |
|
||||
| [**Fibo 2.0 Dev / Edit Turbo**](./fibo2.md#examples) | `fibo2` |
|
||||
| **Anima (Community Edit LoRAs)** | `cosmos_reference` |
|
||||
|
||||
Stable-diffusion.spp also supports basic Unet-based editing models like instruct-pix2pix or CosXL-Edit. This document is not about those.
|
||||
@@ -50,6 +51,7 @@ The `--ref-image-args` argument accepts a comma-separated list of key-value pair
|
||||
| `flux_kontext` | FLUX.1 Kontext |
|
||||
| `longcat` | LongCat Image Edit |
|
||||
| `flux2` | FLUX.2 models |
|
||||
| `fibo2` | Fibo 2.0 Dev / Edit Turbo |
|
||||
| `qwen` | Qwen Image Edit |
|
||||
| `qwen_layered` | Qwen Image Layered |
|
||||
| `z_image_omni` | Boogu, Z-Image Omni |
|
||||
@@ -73,7 +75,12 @@ If presets are insufficient, you can manually configure the following parameters
|
||||
| `ref_index_mode` | string | Behavior of the RoPE index. | `fixed`, `increase`, `decrease` |
|
||||
| `force_ref_timestep_zero` | bool | Forces timestep=0 for reference tokens. | `true`, `false` (Krea2 only) |
|
||||
| `resize_before_vae` | bool | Whether reference images are resized before VAE encoding. | `true`, `false` |
|
||||
| `vae_input_max_pixels` | int | Maximum pixel area for VAE reference inputs. | Integer |
|
||||
| `resize_vae_to_target` | bool | Use output dimensions for every reference, overriding area-based sizing. | `true`, `false` |
|
||||
| `resize_single_ref_to_target` | bool | Use output dimensions when exactly one reference is supplied. | `true`, `false` |
|
||||
| `vae_size_buckets` | bool | Choose aligned dimensions within ±10% of the requested area, minimizing aspect-ratio cropping. Ignored when using output dimensions. | `true`, `false` |
|
||||
| `vae_center_crop` | bool | Scale to cover the selected dimensions with Lanczos, then center-crop. Otherwise resize directly. | `true`, `false` |
|
||||
| `vae_input_max_pixels` | int | VAE area budget, also capped by output area in ordinary resizing. With size buckets, this is the target area and output area does not cap it. | Integer |
|
||||
| `vlm_use_vae_input` | bool | Reuse the resized/cropped VAE input for the VLM, before the VLM's own resize. Has no effect when `resize_before_vae=false`. | `true`, `false` |
|
||||
| `vlm_resize_mode` | string | How to resize VLM reference inputs. | `longest_side`, `area`, `none` |
|
||||
| `vlm_max_size` | int | Maximum VLM input size; interpreted according to `vlm_resize_mode`. | Integer |
|
||||
| `vlm_min_size` | int | Minimum VLM input size; interpreted according to `vlm_resize_mode`. | Integer |
|
||||
@@ -88,6 +95,7 @@ For a technical overview of how each preset is configured, see the table below.
|
||||
| `flux_kontext` | No | `fixed` | `none` | |
|
||||
| `longcat` | Yes | `fixed` | `area` | |
|
||||
| `flux2` | No | `increase` | `none` | |
|
||||
| `fibo2` | Yes | `increase` | `longest` | Shared reference crops; clean reference timesteps; see [Fibo 2.0](./fibo2.md) |
|
||||
| `qwen` | Yes | `increase` | `area` | |
|
||||
| `qwen_layered` | Yes | `decrease` | `area` | |
|
||||
| `mage_flow` | Yes | `increase` | `longest` | `vlm_max_size = 384`, VAE input resized to target |
|
||||
@@ -99,3 +107,5 @@ For a technical overview of how each preset is configured, see the table below.
|
||||
**Additional Default Notes:**
|
||||
- **VLM Input Sizes:** For most presets, `vlm_max_size` and `vlm_min_size` are set to `-1`, meaning the values are model-dependent and handled automatically. In `area` mode they represent pixel area; in `longest_side` mode they represent a side length in pixels.
|
||||
- **VAE Input Size:** `vae_input_max_pixels` defaults to $1024 \times 1024$ pixels (`1048576`).
|
||||
- **VAE Geometry:** `resize_vae_to_target` defaults to `true` for Mage-Flow and LLaDA-Image. Fibo enables `resize_single_ref_to_target`, `vae_size_buckets`, and `vae_center_crop`. These options are otherwise `false`. Bucket dimensions use the model's alignment (normally 16, Qwen Image 32), with a minimum side of four alignment units; if no bucket fits, output dimensions are used.
|
||||
- **Shared Encoder Input:** `vlm_use_vae_input` defaults to `true` for Fibo and Qwen Image 2.1, otherwise `false`. All these options can be overridden individually after selecting a preset.
|
||||
@@ -0,0 +1,53 @@
|
||||
# How to Use
|
||||
|
||||
Fibo 2.0 uses a Fibo diffusion transformer, the standard FLUX.2 VAE, and Qwen3-VL 4B as the LLM text encoder. The Qwen tokenizer is built in.
|
||||
|
||||
## Download weights
|
||||
|
||||
- Download Fibo 2.0 Dev
|
||||
- safetensors: https://huggingface.co/Comfy-Org/Fibo-2.0/tree/main/diffusion_models
|
||||
- Download Fibo 2.0 Edit Turbo
|
||||
- safetensors: https://huggingface.co/Comfy-Org/Fibo-2.0/tree/main/diffusion_models
|
||||
- Download vae
|
||||
- safetensors: https://huggingface.co/black-forest-labs/FLUX.2-dev/tree/main
|
||||
- Download Qwen3-VL 4B
|
||||
- safetensors: https://huggingface.co/Comfy-Org/Fibo-2.0/tree/main/text_encoders
|
||||
- gguf: https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct-GGUF/tree/main
|
||||
|
||||
The original Diffusers weights from [briaai/Fibo-2.0-dev](https://huggingface.co/briaai/Fibo-2.0-dev) and [briaai/Fibo-2.0-edit-turbo](https://huggingface.co/briaai/Fibo-2.0-edit-turbo) are also supported. Pass the transformer's `diffusion_pytorch_model.safetensors.index.json` to `--diffusion-model` and keep its shards alongside it.
|
||||
|
||||
INT8 convrot weights load directly; see [INT8 convrot](./int8_convrot.md) for backend requirements.
|
||||
|
||||
## Examples
|
||||
|
||||
### Fibo 2.0 Dev
|
||||
|
||||
Dev supports both text-to-image generation and reference-image editing.
|
||||
|
||||
```
|
||||
.\bin\Release\sd-cli.exe --diffusion-model ..\models\diffusion_models\fibo2_dev_int8_convrot.safetensors --llm ..\models\text_encoders\Qwen3-VL-4B-Instruct-Q4_K_M.gguf --llm_vision ..\models\text_encoders\Qwen3-VL-4B-Instruct-mmproj-BF16.gguf --vae ..\models\vae\flux2_ae.safetensors -p "a lovely cat holding a sign says 'fibo2.cpp'" --cfg-scale 5.0 --sampling-method euler -v --offload-to-cpu
|
||||
```
|
||||
|
||||
<img src="../assets/fibo2/t2i.png" />
|
||||
|
||||
```
|
||||
.\bin\Release\sd-cli.exe --diffusion-model ..\models\diffusion_models\fibo2_dev_int8_convrot.safetensors --llm ..\models\text_encoders\Qwen3-VL-4B-Instruct-Q4_K_M.gguf --llm_vision ..\models\text_encoders\Qwen3-VL-4B-Instruct-mmproj-BF16.gguf --vae ..\models\vae\flux2_ae.safetensors -r ..\assets\flux\flux1-dev-q8_0.png -p "change 'flux.cpp' to 'fibo2.cpp'" --cfg-scale 5.0 --sampling-method euler -v --offload-to-cpu
|
||||
```
|
||||
|
||||
<img src="../assets/fibo2/dev_edit.png" />
|
||||
|
||||
### Fibo 2.0 Edit Turbo
|
||||
|
||||
Edit Turbo is distilled for reference-image editing in 4 steps with CFG 1.
|
||||
|
||||
```
|
||||
.\bin\Release\sd-cli.exe --diffusion-model ..\models\diffusion_models\fibo2_edit_turbo_int8_convrot.safetensors --llm ..\models\text_encoders\Qwen3-VL-4B-Instruct-Q4_K_M.gguf --llm_vision ..\models\text_encoders\Qwen3-VL-4B-Instruct-mmproj-BF16.gguf --vae ..\models\vae\flux2_ae.safetensors -r ..\assets\flux\flux1-dev-q8_0.png -p "change 'flux.cpp' to 'fibo2.cpp'" --cfg-scale 1 --steps 4 --sampling-method euler -v --offload-to-cpu
|
||||
```
|
||||
|
||||
The BF16 files accept the same options. Use dimensions divisible by 16. Repeat `-r` to use multiple reference images; instructions can refer to `<image_1>`, `<image_2>`, etc. See [Image Editing](./edit.md) for reference-image preprocessing options. Separate GGUF text encoders need their vision projector passed with `--llm_vision` for editing.
|
||||
|
||||
The automatic `fibo2` reference preset combines `resize_single_ref_to_target=true`, `vae_size_buckets=true`, `vae_center_crop=true`, and `vlm_use_vae_input=true`. A single reference is scaled to cover the output dimensions and center-cropped. Multiple references use dimensions divisible by 16, chosen within ±10% of one megapixel to minimize cropping for each image's aspect ratio; `vae_input_max_pixels` overrides that target area. The VAE and vision encoder receive the same crops, with smaller copies used by the vision encoder. These are generic reference-image options and can be overridden individually: for example, `--ref-image-args vae_center_crop=false` uses direct resizing, while `resize_before_vae=false` skips this automatic crop and resize step entirely.
|
||||
|
||||
The vision encoder defaults to `vlm_resize_mode=longest_side,vlm_max_size=768`. Override these with `--ref-image-args`, for example `vlm_max_size=512`, or use `vlm_resize_mode=none` to skip size-based scaling. Vision inputs are still aligned to the encoder's patch grid.
|
||||
|
||||
Plain prompts and structured JSON captions are accepted. This integration uses the caption-only text-rendering path (`render_text_images=False`); it does not generate auxiliary glyph images or expand prompts with Bria VGL. Native Fibo mask preprocessing is not connected to `--mask`.
|
||||
Reference in new issue
Block a user