feat: unify text encoder selection across H3 variants (#2340)

docs/minimax_h3.md

@@ -12,12 +12,31 @@ Pass the four MiniMax-H3 components separately:
 - `--diffusion-model`: MiniMax-H3 diffusion transformer
 - `--vae`: MiniMax-H3 video VAE
 - `--audio-vae`: MiniMax-H3 audio VAE
-- `--llm`: the MiniMax-H3 Qwen3-VL-32B text encoder checkpoint
+- `--llm`: the MiniMax-H3 Qwen3-VL-32B text encoder, or Qwen3-VL-4B with the P-H3 text connector
 
-The text encoder must be the MiniMax-H3 variant: Qwen3-VL-32B truncated to 50
+For the 32B option, use the MiniMax-H3 variant: Qwen3-VL-32B truncated to 50
 language layers and exported without the final language-model normalization.
-Its Qwen3-VL vision tower, including the three DeepStack mergers, must also be
-present. If the vision tower is stored separately, pass it with `--llm_vision`.
+
+For the smaller 4B option, pass the P-H3-Image text encoder, which bundles the
+connector, with `--llm`. Alternatively, pass Qwen3-VL-4B with `--llm` and the
+[separate P-H3 text connector](https://huggingface.co/PrunaAI/P-H3-Image/blob/main/text_connector/diffusion_pytorch_model.safetensors)
+with `--embeddings-connectors`. For example, replace the `--llm` argument in the
+commands below with either:
+
+```sh
+--llm ..\models\text_encoders\p-h3-image_text_encoder_int8_convrot.safetensors
+```
+
+or:
+
+```sh
+--llm ..\models\text_encoders\qwen3vl_4b.gguf --embeddings-connectors ..\models\text_encoders\p-h3-image_text_connector.safetensors
+```
+
+Reference-image and reference-video conditioning needs the matching encoder's
+Qwen3-VL vision tower, including the three DeepStack mergers. If stored
+separately, pass it with `--llm_vision`. Keep using the MiniMax-H3 video and audio
+VAEs with either text encoder.
 
 Both the original time-embedder DiT and the smaller AdaLN curve-table variant
 are detected from their weights.
@@ -30,6 +49,8 @@ are detected from their weights.
 - Download qwen3vl_32b_minimax_h3
     - safetensors: https://huggingface.co/Comfy-Org/MiniMax-H3/tree/main/text_encoders
     - gguf: https://huggingface.co/leejet/MiniMax-H3-GGUF/tree/main
+- Or download the P-H3-Image text encoder (Qwen3-VL-4B with the text connector)
+    - safetensors: https://huggingface.co/PrunaAI/P-H3-Image-Comfy/tree/main/text_encoders
 - Download vae
     - safetensors: https://huggingface.co/Comfy-Org/MiniMax-H3/tree/main/vae
 - Download audio vae

docs/p_h3_image.md

@@ -1,6 +1,6 @@
 # How to Use
 
-P-H3-Image supports text-to-image and reference-image editing. It uses a MiniMax-H3 diffusion transformer, a single-frame VAE, and Qwen3-VL-4B with a text connector. Prompt enhancement is not included.
+P-H3-Image supports text-to-image and reference-image editing. It uses a MiniMax-H3 diffusion transformer, a single-frame VAE, and either Qwen3-VL-4B with a text connector or the MiniMax-H3 Qwen3-VL-32B text encoder. Prompt enhancement is not included.
 
 ## Download weights
 
@@ -10,9 +10,22 @@ P-H3-Image supports text-to-image and reference-image editing. It uses a MiniMax
     - safetensors: https://huggingface.co/PrunaAI/P-H3-Image-Comfy/tree/main/vae
 - Download Qwen3-VL-4B with the text connector
     - https://huggingface.co/PrunaAI/P-H3-Image-Comfy/tree/main/text_encoders
+- Or download the MiniMax-H3 Qwen3-VL-32B text encoder
+    - safetensors: https://huggingface.co/Comfy-Org/MiniMax-H3/tree/main/text_encoders
+    - gguf: https://huggingface.co/leejet/MiniMax-H3-GGUF/tree/main
 
 Alternatively, reuse Qwen3-VL-4B, including its vision tower, with the [separate text connector](https://huggingface.co/PrunaAI/P-H3-Image/blob/main/text_connector/diffusion_pytorch_model.safetensors). Save the connector as `p-h3-image_text_connector.safetensors` and pass it with `--embeddings-connectors`.
 
+The MiniMax-H3 Qwen3-VL-32B encoder does not need a text connector. To use it in
+either example below, omit `--embeddings-connectors` and replace `--llm` with:
+
+```sh
+--llm ..\models\text_encoders\qwen3vl_32b_minimax_h3-Q4_K_M.gguf
+```
+
+Reference-image editing needs the matching encoder's vision tower. If stored
+separately, pass it with `--llm_vision`.
+
 Use the P-H3 image VAE; the MiniMax-H3 video VAE is not compatible. No audio VAE is needed.
 
 ## Examples

examples/server/recipes/hyperflow-h3-flash-fl2va.json

@@ -3,7 +3,7 @@
   "id": "hyperflow-h3-flash-fl2va",
   "name": "HyperFlow H3 Flash FL2VA",
   "size_multiple": 32,
-  "description": "Generate video and audio from text, optionally with start/end images. Use the HyperFlow H3 Flash FL2VA transformer. Reuses MiniMax-H3 components with the HyperFlow 6-step schedule. Audio VAE is optional for silent video.",
+  "description": "Generate video and audio from text, optionally with start/end images. Use the HyperFlow H3 Flash FL2VA transformer. Reuses MiniMax-H3 components with either the MiniMax-H3 Qwen3-VL-32B text encoder or Qwen3-VL-4B with the P-H3 text connector and the HyperFlow 6-step schedule. Audio VAE is optional for silent video.",
   "model_url": "https://huggingface.co/VideoRebirthAI/hyperflow-h3-flash",
   "slots": [
     {
@@ -14,7 +14,17 @@
     {
       "id": "llm",
       "required": true,
-      "label": "MiniMax-H3 Qwen 3 VL 32B Text Encoder"
+      "label": "MiniMax-H3 Qwen 3 VL 32B or P-H3 Qwen 3 VL 4B Text Encoder"
+    },
+    {
+      "id": "llm_vision",
+      "required": false,
+      "label": "Vision tower (when not included in the text encoder)"
+    },
+    {
+      "id": "embeddings_connectors",
+      "required": false,
+      "label": "P-H3 Text Connector (for Qwen 3 VL 4B, when not included in the text encoder)"
     },
     {
       "id": "vae",

examples/server/recipes/hyperflow-h3-flash-ref2va.json

@@ -3,7 +3,7 @@
   "id": "hyperflow-h3-flash-ref2va",
   "name": "HyperFlow H3 Flash Ref2VA",
   "size_multiple": 32,
-  "description": "Generate video and audio with reference images. Use the HyperFlow H3 Flash Ref2VA transformer; video/audio references are not validated for these weights. Reuses MiniMax-H3 components with the HyperFlow 6-step schedule. Audio VAE is optional for silent video.",
+  "description": "Generate video and audio with reference images. Use the HyperFlow H3 Flash Ref2VA transformer; video/audio references are not validated for these weights. Reuses MiniMax-H3 components with either the MiniMax-H3 Qwen3-VL-32B text encoder or Qwen3-VL-4B with the P-H3 text connector and the HyperFlow 6-step schedule. Reference images need the matching vision tower. Audio VAE is optional for silent video.",
   "model_url": "https://huggingface.co/VideoRebirthAI/hyperflow-h3-flash",
   "slots": [
     {
@@ -14,7 +14,17 @@
     {
       "id": "llm",
       "required": true,
-      "label": "MiniMax-H3 Qwen 3 VL 32B Text Encoder"
+      "label": "MiniMax-H3 Qwen 3 VL 32B or P-H3 Qwen 3 VL 4B Text Encoder"
+    },
+    {
+      "id": "llm_vision",
+      "required": false,
+      "label": "Vision tower (required for reference images when not included in the text encoder)"
+    },
+    {
+      "id": "embeddings_connectors",
+      "required": false,
+      "label": "P-H3 Text Connector (for Qwen 3 VL 4B, when not included in the text encoder)"
     },
     {
       "id": "vae",

examples/server/recipes/minimax-h3-fl2va.json

@@ -3,7 +3,7 @@
   "id": "minimax-h3-fl2va",
   "name": "MiniMax H3 FL2VA",
   "size_multiple": 32,
-  "description": "Generate video and audio, optionally with start/end images. Supports full and pruned FL2VA weights. Audio VAE is optional for silent video.",
+  "description": "Generate video and audio, optionally with start/end images. Supports full and pruned FL2VA weights with either the MiniMax-H3 Qwen3-VL-32B text encoder or Qwen3-VL-4B with the P-H3 text connector. Audio VAE is optional for silent video.",
   "model_url": "https://huggingface.co/MiniMaxAI/MiniMax-H3",
   "slots": [
     {
@@ -13,7 +13,17 @@
     {
       "id": "llm",
       "required": true,
-      "label": "MiniMax-H3 Qwen 3 VL 32B Text Encoder"
+      "label": "MiniMax-H3 Qwen 3 VL 32B or P-H3 Qwen 3 VL 4B Text Encoder"
+    },
+    {
+      "id": "llm_vision",
+      "required": false,
+      "label": "Vision tower (when not included in the text encoder)"
+    },
+    {
+      "id": "embeddings_connectors",
+      "required": false,
+      "label": "P-H3 Text Connector (for Qwen 3 VL 4B, when not included in the text encoder)"
     },
     {
       "id": "vae",

examples/server/recipes/minimax-h3-ref2va.json

@@ -3,7 +3,7 @@
   "id": "minimax-h3-ref2va",
   "name": "MiniMax H3 Ref2VA",
   "size_multiple": 32,
-  "description": "Supply image, video or audio references with compatible Ref2VA weights. Audio VAE is optional for silent video.",
+  "description": "Supply image, video or audio references with compatible Ref2VA weights. Use either the MiniMax-H3 Qwen3-VL-32B text encoder or Qwen3-VL-4B with the P-H3 text connector. Image/video references need the matching vision tower. Audio VAE is optional for silent video.",
   "model_url": "https://huggingface.co/MiniMaxAI/MiniMax-H3",
   "slots": [
     {
@@ -13,7 +13,17 @@
     {
       "id": "llm",
       "required": true,
-      "label": "MiniMax-H3 Qwen 3 VL 32B Text Encoder"
+      "label": "MiniMax-H3 Qwen 3 VL 32B or P-H3 Qwen 3 VL 4B Text Encoder"
+    },
+    {
+      "id": "llm_vision",
+      "required": false,
+      "label": "Vision tower (required for image/video references when not included in the text encoder)"
+    },
+    {
+      "id": "embeddings_connectors",
+      "required": false,
+      "label": "P-H3 Text Connector (for Qwen 3 VL 4B, when not included in the text encoder)"
     },
     {
       "id": "vae",

examples/server/recipes/p-h3-image.json

@@ -3,7 +3,7 @@
   "id": "p-h3-image",
   "name": "P-H3-Image",
   "size_multiple": 32,
-  "description": "Supports text-to-image and reference-image editing with the fixed 8-step schedule. Use the P-H3 single-frame VAE and Qwen3-VL-4B with its vision tower. Supply the P-H3 text connector separately unless it is included in the text encoder.",
+  "description": "Supports text-to-image and reference-image editing with the fixed 8-step schedule. Use the P-H3 single-frame VAE and either Qwen3-VL-4B with the P-H3 text connector or the MiniMax-H3 Qwen3-VL-32B text encoder. Reference images need the matching vision tower.",
   "model_url": "https://huggingface.co/PrunaAI/P-H3-Image",
   "slots": [
     {
@@ -13,7 +13,7 @@
     {
       "id": "llm",
       "required": true,
-      "label": "Qwen 3 VL 4B Text Encoder"
+      "label": "MiniMax-H3 Qwen 3 VL 32B or P-H3 Qwen 3 VL 4B Text Encoder"
     },
     {
       "id": "llm_vision",
@@ -23,7 +23,7 @@
     {
       "id": "embeddings_connectors",
       "required": false,
-      "label": "P-H3 Text Connector (required when not included in the text encoder)"
+      "label": "P-H3 Text Connector (for Qwen 3 VL 4B, when not included in the text encoder)"
     },
     {
       "id": "vae",

src/conditioning/conditioner.hpp

@@ -2030,20 +2030,23 @@ struct LLMEmbedder : public Conditioner {
                                                "text_encoders.llm",
                                                enable_vision,
                                                weight_manager);
-        // MiniMax-H3-Image also accepts the P-H3-Image text encoder in place of the MiniMax-H3 Qwen3-VL-32B encoder.
-        const bool has_ph3_connector = tensor_storage_map.find(std::string(PH3ImageTextConnectorRunner::prefix) + ".proj.weight") !=
-                                       tensor_storage_map.end();
-        if (version == VERSION_P_H3_IMAGE || (version == VERSION_MINIMAX_H3_IMAGE && has_ph3_connector)) {
-            if (llm->config.num_layers != 36 || llm->config.hidden_size != 2560) {
-                throw std::runtime_error(version == VERSION_P_H3_IMAGE
-                                             ? "P-H3-Image requires the Qwen3-VL-4B text encoder"
-                                             : "The P-H3-Image text connector requires the Qwen3-VL-4B text encoder");
+        if (sd_version_is_minimax_h3(version)) {
+            const bool has_ph3_connector = tensor_storage_map.count(std::string(PH3ImageTextConnectorRunner::prefix) + ".proj.weight") != 0;
+            if (has_ph3_connector) {
+                if (llm->config.num_layers != 36 || llm->config.hidden_size != 2560) {
+                    throw std::runtime_error("The P-H3-Image text connector requires the Qwen3-VL-4B text encoder");
+                }
+                ph3_connector = std::make_shared<PH3ImageTextConnectorRunner>(backend, tensor_storage_map, weight_manager);
+            } else if (llm->config.hidden_size == 2560) {
+                throw std::runtime_error(
+                    sd_format("%s with Qwen3-VL-4B requires the P-H3-Image text connector; "
+                              "pass it with --embeddings-connectors or use the MiniMax-H3 Qwen3-VL-32B text encoder",
+                              sd_version_name(version)));
+            } else if (llm->config.hidden_size != 5120 || (llm->config.num_layers != 50 && llm->config.num_layers != 64)) {
+                throw std::runtime_error(
+                    sd_format("%s requires the MiniMax-H3 Qwen3-VL-32B text encoder or Qwen3-VL-4B with the P-H3-Image text connector",
+                              sd_version_name(version)));
             }
-            ph3_connector = std::make_shared<PH3ImageTextConnectorRunner>(backend, tensor_storage_map, weight_manager);
-        } else if (version == VERSION_MINIMAX_H3_IMAGE && llm->config.hidden_size == 2560) {
-            throw std::runtime_error(
-                "MiniMax-H3-Image with Qwen3-VL-4B requires the P-H3-Image text connector; "
-                "pass it with --embeddings-connectors or use the MiniMax-H3 Qwen3-VL-32B text encoder");
         }
         int pad_id = 151643;
         if (arch == LLM::LLMArch::MISTRAL_SMALL_3_2 || arch == LLM::LLMArch::MINISTRAL_3_3B) {

src/model/diffusion/fibo2.hpp

@@ -131,8 +131,8 @@ namespace Fibo2 {
                     v = reshape(project("to_v", kv));
                 }
             }
-            q = std::dynamic_pointer_cast<RMSNorm>(blocks["norm_q"])->forward(ctx, q);
-            k = std::dynamic_pointer_cast<RMSNorm>(blocks["norm_k"])->forward(ctx, k);
+            q            = std::dynamic_pointer_cast<RMSNorm>(blocks["norm_q"])->forward(ctx, q);
+            k            = std::dynamic_pointer_cast<RMSNorm>(blocks["norm_k"])->forward(ctx, k);
             auto prepare = [&](ggml_tensor* h) {
                 if (pe) {
                     return Rope::apply_rope(ctx, h, pe);

src/model/diffusion/iris.hpp

@@ -91,10 +91,10 @@ namespace Iris {
         }
 
         ggml_tensor* forward(GGMLRunnerContext* ctx, ggml_tensor* x, ggml_tensor* pe = nullptr, ggml_tensor* mask = nullptr) {
-            auto c        = ctx->ggml_ctx;
-            auto head_dim = dim / heads;
-            auto qkv      = std::dynamic_pointer_cast<Linear>(blocks["qkv"])->forward(ctx, x);
-            auto qkv_dim  = qkv->ne[0] / 3;
+            auto c         = ctx->ggml_ctx;
+            auto head_dim  = dim / heads;
+            auto qkv       = std::dynamic_pointer_cast<Linear>(blocks["qkv"])->forward(ctx, x);
+            auto qkv_dim   = qkv->ne[0] / 3;
             auto split_qkv = [&](int i) {
                 return ggml_view_4d(c, qkv, head_dim, qkv_dim / head_dim, qkv->ne[1], qkv->ne[2] * qkv->ne[3],
                                     head_dim * qkv->nb[0], qkv->nb[1], qkv->nb[2], i * qkv_dim * qkv->nb[0]);

src/model/diffusion/mmdit.hpp

@@ -373,7 +373,7 @@ public:
                          ggml_tensor* x) {
         auto qkv = pre_attention(ctx, x);
         x        = ggml_ext_attention_ext(ctx, qkv[0], qkv[1], qkv[2], num_heads, nullptr, true, ctx->flash_attn_enabled);  // [N, n_token, dim]
-        x        = post_attention(ctx, x);                                                                                   // [N, n_token, dim]
+        x        = post_attention(ctx, x);                                                                                  // [N, n_token, dim]
         return x;
     }
 };

src/model/diffusion/seedvr2.hpp

@@ -194,17 +194,17 @@ namespace SeedVR2 {
                     return x;
                 return std::dynamic_pointer_cast<AdaSingle>(blocks["ada." + branch(text)])->forward(ctx, x, emb, layer, gate);
             };
-            auto vqkv = linear("attn.proj_qkv", false, ada(ggml_rms_norm(c, vid, 1e-5f), false, "attn"));
-            auto tqkv = linear("attn.proj_qkv", true, ada(ggml_rms_norm(c, txt, 1e-5f), true, "attn"));
-            vqkv      = ggml_get_rows(c, vqkv, inputs.input(ctx, plan.order));
-            auto vpe  = rotation(ctx, inputs.input(ctx, plan.positions));
-            auto tpe  = rotation(ctx, inputs.input(ctx, plan.text_positions));
+            auto vqkv      = linear("attn.proj_qkv", false, ada(ggml_rms_norm(c, vid, 1e-5f), false, "attn"));
+            auto tqkv      = linear("attn.proj_qkv", true, ada(ggml_rms_norm(c, txt, 1e-5f), true, "attn"));
+            vqkv           = ggml_get_rows(c, vqkv, inputs.input(ctx, plan.order));
+            auto vpe       = rotation(ctx, inputs.input(ctx, plan.positions));
+            auto tpe       = rotation(ctx, inputs.input(ctx, plan.text_positions));
             auto split_qkv = [&](ggml_tensor* x, int i) {
                 int64_t qkv_dim = x->ne[0] / 3;
                 return ggml_view_4d(c, x, 128, qkv_dim / 128, x->ne[1], x->ne[2] * x->ne[3],
                                     128 * x->nb[0], x->nb[1], x->nb[2], i * qkv_dim * x->nb[0]);
             };
-            auto qkv  = [&](ggml_tensor* x, bool text, ggml_tensor* pe) {
+            auto qkv = [&](ggml_tensor* x, bool text, ggml_tensor* pe) {
                 std::vector<ggml_tensor*> parts{nullptr, nullptr, nullptr};
                 for (int i = 0; i < 2; ++i) {
                     auto h   = std::dynamic_pointer_cast<RMSNorm>(blocks[std::string(i == 0 ? "attn.norm_q." : "attn.norm_k.") + branch(text)])->forward(ctx, split_qkv(x, i));
@@ -212,8 +212,8 @@ namespace SeedVR2 {
                     parts[i] = VSR::apply_rope(ctx, h, pe, heads);
                 }
                 // v must stay 3-D [dim, L, N]: the window loop slices dim 1 as tokens.
-                auto v      = split_qkv(x, 2);
-                parts[2]    = ggml_view_3d(c, v, dim, x->ne[1], x->ne[2], v->nb[2], v->nb[3], 0);
+                auto v   = split_qkv(x, 2);
+                parts[2] = ggml_view_3d(c, v, dim, x->ne[1], x->ne[2], v->nb[2], v->nb[3], 0);
                 return parts;
             };
             auto vp = qkv(vqkv, false, vpe), tp = qkv(tqkv, true, tpe);

src/model_loader.cpp

@@ -581,7 +581,7 @@ SDVersion ModelLoader::get_sd_version() const {
             if (tensor_storage_map.count("model.diffusion_model.audio_patch_proj.weight") == 0) {
                 return VERSION_MINIMAX_H3_IMAGE;
             }
-            // P-H3 uses the same DiT architecture as H3; its VAE or connector identifies the image pipeline.
+            // The connector is shared with H3 video; the single-frame VAE identifies P-H3.
             auto decoder = tensor_storage_map.find("first_stage_model.decoder.proj_out.weight");
             if (decoder == tensor_storage_map.end()) {
                 decoder = tensor_storage_map.find("vae.decoder.proj_out.weight");
@@ -589,17 +589,6 @@ SDVersion ModelLoader::get_sd_version() const {
             if (decoder != tensor_storage_map.end() && decoder->second.ne[1] == 768) {
                 return VERSION_P_H3_IMAGE;
             }
-            for (const std::string prefix : {"", "embeddings_connectors."}) {
-                if (tensor_storage_map.count(prefix + "text_encoders.ph3_image_connector.proj.weight") ||
-                    tensor_storage_map.count(prefix + "text_encoders.llm.text_connector.proj.weight")) {
-                    return VERSION_P_H3_IMAGE;
-                }
-            }
-            auto connector = tensor_storage_map.find("embeddings_connectors.proj.weight");
-            if (connector != tensor_storage_map.end() && connector->second.logical_ne0() == 12800 &&
-                connector->second.ne[1] == 5120 && tensor_storage_map.count("embeddings_connectors.sink_embedding")) {
-                return VERSION_P_H3_IMAGE;
-            }
             return VERSION_MINIMAX_H3;
         }
         if (name == "model.diffusion_model.y_embedder.layer_pool.weight" &&

src/name_conversion.cpp

@@ -1741,7 +1741,7 @@ std::string convert_tensor_name(std::string name, SDVersion version) {
 
     const std::vector<std::pair<std::string, std::string>> connector_prefix_map = {
         {"embeddings_connectors.text_encoders.", "text_encoders."},
-        {"embeddings_connectors.", sd_version_is_minimax_h3_image(version) ? "text_encoders.ph3_image_connector." : ""},
+        {"embeddings_connectors.", sd_version_is_minimax_h3(version) ? "text_encoders.ph3_image_connector." : ""},
     };
     replace_with_prefix_map(name, connector_prefix_map);