feat: unify text encoder selection across H3 variants (#2340)

docs/minimax_h3.md

@@ -12,12 +12,31 @@ Pass the four MiniMax-H3 components separately:
 - `--diffusion-model`: MiniMax-H3 diffusion transformer
 - `--vae`: MiniMax-H3 video VAE
 - `--audio-vae`: MiniMax-H3 audio VAE
-- `--llm`: the MiniMax-H3 Qwen3-VL-32B text encoder checkpoint
+- `--llm`: the MiniMax-H3 Qwen3-VL-32B text encoder, or Qwen3-VL-4B with the P-H3 text connector
 
-The text encoder must be the MiniMax-H3 variant: Qwen3-VL-32B truncated to 50
+For the 32B option, use the MiniMax-H3 variant: Qwen3-VL-32B truncated to 50
 language layers and exported without the final language-model normalization.
-Its Qwen3-VL vision tower, including the three DeepStack mergers, must also be
-present. If the vision tower is stored separately, pass it with `--llm_vision`.
+
+For the smaller 4B option, pass the P-H3-Image text encoder, which bundles the
+connector, with `--llm`. Alternatively, pass Qwen3-VL-4B with `--llm` and the
+[separate P-H3 text connector](https://huggingface.co/PrunaAI/P-H3-Image/blob/main/text_connector/diffusion_pytorch_model.safetensors)
+with `--embeddings-connectors`. For example, replace the `--llm` argument in the
+commands below with either:
+
+```sh
+--llm ..\models\text_encoders\p-h3-image_text_encoder_int8_convrot.safetensors
+```
+
+or:
+
+```sh
+--llm ..\models\text_encoders\qwen3vl_4b.gguf --embeddings-connectors ..\models\text_encoders\p-h3-image_text_connector.safetensors
+```
+
+Reference-image and reference-video conditioning needs the matching encoder's
+Qwen3-VL vision tower, including the three DeepStack mergers. If stored
+separately, pass it with `--llm_vision`. Keep using the MiniMax-H3 video and audio
+VAEs with either text encoder.
 
 Both the original time-embedder DiT and the smaller AdaLN curve-table variant
 are detected from their weights.
@@ -30,6 +49,8 @@ are detected from their weights.
 - Download qwen3vl_32b_minimax_h3
     - safetensors: https://huggingface.co/Comfy-Org/MiniMax-H3/tree/main/text_encoders
     - gguf: https://huggingface.co/leejet/MiniMax-H3-GGUF/tree/main
+- Or download the P-H3-Image text encoder (Qwen3-VL-4B with the text connector)
+    - safetensors: https://huggingface.co/PrunaAI/P-H3-Image-Comfy/tree/main/text_encoders
 - Download vae
     - safetensors: https://huggingface.co/Comfy-Org/MiniMax-H3/tree/main/vae
 - Download audio vae

docs/p_h3_image.md

@@ -1,6 +1,6 @@
 # How to Use
 
-P-H3-Image supports text-to-image and reference-image editing. It uses a MiniMax-H3 diffusion transformer, a single-frame VAE, and Qwen3-VL-4B with a text connector. Prompt enhancement is not included.
+P-H3-Image supports text-to-image and reference-image editing. It uses a MiniMax-H3 diffusion transformer, a single-frame VAE, and either Qwen3-VL-4B with a text connector or the MiniMax-H3 Qwen3-VL-32B text encoder. Prompt enhancement is not included.
 
 ## Download weights
 
@@ -10,9 +10,22 @@ P-H3-Image supports text-to-image and reference-image editing. It uses a MiniMax
     - safetensors: https://huggingface.co/PrunaAI/P-H3-Image-Comfy/tree/main/vae
 - Download Qwen3-VL-4B with the text connector
     - https://huggingface.co/PrunaAI/P-H3-Image-Comfy/tree/main/text_encoders
+- Or download the MiniMax-H3 Qwen3-VL-32B text encoder
+    - safetensors: https://huggingface.co/Comfy-Org/MiniMax-H3/tree/main/text_encoders
+    - gguf: https://huggingface.co/leejet/MiniMax-H3-GGUF/tree/main
 
 Alternatively, reuse Qwen3-VL-4B, including its vision tower, with the [separate text connector](https://huggingface.co/PrunaAI/P-H3-Image/blob/main/text_connector/diffusion_pytorch_model.safetensors). Save the connector as `p-h3-image_text_connector.safetensors` and pass it with `--embeddings-connectors`.
 
+The MiniMax-H3 Qwen3-VL-32B encoder does not need a text connector. To use it in
+either example below, omit `--embeddings-connectors` and replace `--llm` with:
+
+```sh
+--llm ..\models\text_encoders\qwen3vl_32b_minimax_h3-Q4_K_M.gguf
+```
+
+Reference-image editing needs the matching encoder's vision tower. If stored
+separately, pass it with `--llm_vision`.
+
 Use the P-H3 image VAE; the MiniMax-H3 video VAE is not compatible. No audio VAE is needed.
 
 ## Examples

examples/server/recipes/hyperflow-h3-flash-fl2va.json

@@ -3,7 +3,7 @@
   "id": "hyperflow-h3-flash-fl2va",
   "name": "HyperFlow H3 Flash FL2VA",
   "size_multiple": 32,
-  "description": "Generate video and audio from text, optionally with start/end images. Use the HyperFlow H3 Flash FL2VA transformer. Reuses MiniMax-H3 components with the HyperFlow 6-step schedule. Audio VAE is optional for silent video.",
+  "description": "Generate video and audio from text, optionally with start/end images. Use the HyperFlow H3 Flash FL2VA transformer. Reuses MiniMax-H3 components with either the MiniMax-H3 Qwen3-VL-32B text encoder or Qwen3-VL-4B with the P-H3 text connector and the HyperFlow 6-step schedule. Audio VAE is optional for silent video.",
   "model_url": "https://huggingface.co/VideoRebirthAI/hyperflow-h3-flash",
   "slots": [
     {
@@ -14,7 +14,17 @@
     {
       "id": "llm",
       "required": true,
-      "label": "MiniMax-H3 Qwen 3 VL 32B Text Encoder"
+      "label": "MiniMax-H3 Qwen 3 VL 32B or P-H3 Qwen 3 VL 4B Text Encoder"
+    },
+    {
+      "id": "llm_vision",
+      "required": false,
+      "label": "Vision tower (when not included in the text encoder)"
+    },
+    {
+      "id": "embeddings_connectors",
+      "required": false,
+      "label": "P-H3 Text Connector (for Qwen 3 VL 4B, when not included in the text encoder)"
     },
     {
       "id": "vae",

examples/server/recipes/hyperflow-h3-flash-ref2va.json

@@ -3,7 +3,7 @@
   "id": "hyperflow-h3-flash-ref2va",
   "name": "HyperFlow H3 Flash Ref2VA",
   "size_multiple": 32,
-  "description": "Generate video and audio with reference images. Use the HyperFlow H3 Flash Ref2VA transformer; video/audio references are not validated for these weights. Reuses MiniMax-H3 components with the HyperFlow 6-step schedule. Audio VAE is optional for silent video.",
+  "description": "Generate video and audio with reference images. Use the HyperFlow H3 Flash Ref2VA transformer; video/audio references are not validated for these weights. Reuses MiniMax-H3 components with either the MiniMax-H3 Qwen3-VL-32B text encoder or Qwen3-VL-4B with the P-H3 text connector and the HyperFlow 6-step schedule. Reference images need the matching vision tower. Audio VAE is optional for silent video.",
   "model_url": "https://huggingface.co/VideoRebirthAI/hyperflow-h3-flash",
   "slots": [
     {
@@ -14,7 +14,17 @@
     {
       "id": "llm",
       "required": true,
-      "label": "MiniMax-H3 Qwen 3 VL 32B Text Encoder"
+      "label": "MiniMax-H3 Qwen 3 VL 32B or P-H3 Qwen 3 VL 4B Text Encoder"
+    },
+    {
+      "id": "llm_vision",
+      "required": false,
+      "label": "Vision tower (required for reference images when not included in the text encoder)"
+    },
+    {
+      "id": "embeddings_connectors",
+      "required": false,
+      "label": "P-H3 Text Connector (for Qwen 3 VL 4B, when not included in the text encoder)"
     },
     {
       "id": "vae",

examples/server/recipes/minimax-h3-fl2va.json

@@ -3,7 +3,7 @@
   "id": "minimax-h3-fl2va",
   "name": "MiniMax H3 FL2VA",
   "size_multiple": 32,
-  "description": "Generate video and audio, optionally with start/end images. Supports full and pruned FL2VA weights. Audio VAE is optional for silent video.",
+  "description": "Generate video and audio, optionally with start/end images. Supports full and pruned FL2VA weights with either the MiniMax-H3 Qwen3-VL-32B text encoder or Qwen3-VL-4B with the P-H3 text connector. Audio VAE is optional for silent video.",
   "model_url": "https://huggingface.co/MiniMaxAI/MiniMax-H3",
   "slots": [
     {
@@ -13,7 +13,17 @@
     {
       "id": "llm",
       "required": true,
-      "label": "MiniMax-H3 Qwen 3 VL 32B Text Encoder"
+      "label": "MiniMax-H3 Qwen 3 VL 32B or P-H3 Qwen 3 VL 4B Text Encoder"
+    },
+    {
+      "id": "llm_vision",
+      "required": false,
+      "label": "Vision tower (when not included in the text encoder)"
+    },
+    {
+      "id": "embeddings_connectors",
+      "required": false,
+      "label": "P-H3 Text Connector (for Qwen 3 VL 4B, when not included in the text encoder)"
     },
     {
       "id": "vae",

examples/server/recipes/minimax-h3-ref2va.json

@@ -3,7 +3,7 @@
   "id": "minimax-h3-ref2va",
   "name": "MiniMax H3 Ref2VA",
   "size_multiple": 32,
-  "description": "Supply image, video or audio references with compatible Ref2VA weights. Audio VAE is optional for silent video.",
+  "description": "Supply image, video or audio references with compatible Ref2VA weights. Use either the MiniMax-H3 Qwen3-VL-32B text encoder or Qwen3-VL-4B with the P-H3 text connector. Image/video references need the matching vision tower. Audio VAE is optional for silent video.",
   "model_url": "https://huggingface.co/MiniMaxAI/MiniMax-H3",
   "slots": [
     {
@@ -13,7 +13,17 @@
     {
       "id": "llm",
       "required": true,
-      "label": "MiniMax-H3 Qwen 3 VL 32B Text Encoder"
+      "label": "MiniMax-H3 Qwen 3 VL 32B or P-H3 Qwen 3 VL 4B Text Encoder"
+    },
+    {
+      "id": "llm_vision",
+      "required": false,
+      "label": "Vision tower (required for image/video references when not included in the text encoder)"
+    },
+    {
+      "id": "embeddings_connectors",
+      "required": false,
+      "label": "P-H3 Text Connector (for Qwen 3 VL 4B, when not included in the text encoder)"
     },
     {
       "id": "vae",

examples/server/recipes/p-h3-image.json

@@ -3,7 +3,7 @@
   "id": "p-h3-image",
   "name": "P-H3-Image",
   "size_multiple": 32,
-  "description": "Supports text-to-image and reference-image editing with the fixed 8-step schedule. Use the P-H3 single-frame VAE and Qwen3-VL-4B with its vision tower. Supply the P-H3 text connector separately unless it is included in the text encoder.",
+  "description": "Supports text-to-image and reference-image editing with the fixed 8-step schedule. Use the P-H3 single-frame VAE and either Qwen3-VL-4B with the P-H3 text connector or the MiniMax-H3 Qwen3-VL-32B text encoder. Reference images need the matching vision tower.",
   "model_url": "https://huggingface.co/PrunaAI/P-H3-Image",
   "slots": [
     {
@@ -13,7 +13,7 @@
     {
       "id": "llm",
       "required": true,
-      "label": "Qwen 3 VL 4B Text Encoder"
+      "label": "MiniMax-H3 Qwen 3 VL 32B or P-H3 Qwen 3 VL 4B Text Encoder"
     },
     {
       "id": "llm_vision",
@@ -23,7 +23,7 @@
     {
       "id": "embeddings_connectors",
       "required": false,
-      "label": "P-H3 Text Connector (required when not included in the text encoder)"
+      "label": "P-H3 Text Connector (for Qwen 3 VL 4B, when not included in the text encoder)"
     },
     {
       "id": "vae",

src/conditioning/conditioner.hpp

@@ -2030,20 +2030,23 @@ struct LLMEmbedder : public Conditioner {
                                                "text_encoders.llm",
                                                enable_vision,
                                                weight_manager);
-        // MiniMax-H3-Image also accepts the P-H3-Image text encoder in place of the MiniMax-H3 Qwen3-VL-32B encoder.
-        const bool has_ph3_connector = tensor_storage_map.find(std::string(PH3ImageTextConnectorRunner::prefix) + ".proj.weight") !=
-                                       tensor_storage_map.end();
-        if (version == VERSION_P_H3_IMAGE || (version == VERSION_MINIMAX_H3_IMAGE && has_ph3_connector)) {
+        if (sd_version_is_minimax_h3(version)) {
+            const bool has_ph3_connector = tensor_storage_map.count(std::string(PH3ImageTextConnectorRunner::prefix) + ".proj.weight") != 0;
+            if (has_ph3_connector) {
                 if (llm->config.num_layers != 36 || llm->config.hidden_size != 2560) {
-                throw std::runtime_error(version == VERSION_P_H3_IMAGE
-                                             ? "P-H3-Image requires the Qwen3-VL-4B text encoder"
-                                             : "The P-H3-Image text connector requires the Qwen3-VL-4B text encoder");
+                    throw std::runtime_error("The P-H3-Image text connector requires the Qwen3-VL-4B text encoder");
                 }
                 ph3_connector = std::make_shared<PH3ImageTextConnectorRunner>(backend, tensor_storage_map, weight_manager);
-        } else if (version == VERSION_MINIMAX_H3_IMAGE && llm->config.hidden_size == 2560) {
+            } else if (llm->config.hidden_size == 2560) {
                 throw std::runtime_error(
-                "MiniMax-H3-Image with Qwen3-VL-4B requires the P-H3-Image text connector; "
-                "pass it with --embeddings-connectors or use the MiniMax-H3 Qwen3-VL-32B text encoder");
+                    sd_format("%s with Qwen3-VL-4B requires the P-H3-Image text connector; "
+                              "pass it with --embeddings-connectors or use the MiniMax-H3 Qwen3-VL-32B text encoder",
+                              sd_version_name(version)));
+            } else if (llm->config.hidden_size != 5120 || (llm->config.num_layers != 50 && llm->config.num_layers != 64)) {
+                throw std::runtime_error(
+                    sd_format("%s requires the MiniMax-H3 Qwen3-VL-32B text encoder or Qwen3-VL-4B with the P-H3-Image text connector",
+                              sd_version_name(version)));
+            }
         }
         int pad_id = 151643;
         if (arch == LLM::LLMArch::MISTRAL_SMALL_3_2 || arch == LLM::LLMArch::MINISTRAL_3_3B) {

src/model_loader.cpp

@@ -581,7 +581,7 @@ SDVersion ModelLoader::get_sd_version() const {
             if (tensor_storage_map.count("model.diffusion_model.audio_patch_proj.weight") == 0) {
                 return VERSION_MINIMAX_H3_IMAGE;
             }
-            // P-H3 uses the same DiT architecture as H3; its VAE or connector identifies the image pipeline.
+            // The connector is shared with H3 video; the single-frame VAE identifies P-H3.
             auto decoder = tensor_storage_map.find("first_stage_model.decoder.proj_out.weight");
             if (decoder == tensor_storage_map.end()) {
                 decoder = tensor_storage_map.find("vae.decoder.proj_out.weight");
@@ -589,17 +589,6 @@ SDVersion ModelLoader::get_sd_version() const {
             if (decoder != tensor_storage_map.end() && decoder->second.ne[1] == 768) {
                 return VERSION_P_H3_IMAGE;
             }
-            for (const std::string prefix : {"", "embeddings_connectors."}) {
-                if (tensor_storage_map.count(prefix + "text_encoders.ph3_image_connector.proj.weight") ||
-                    tensor_storage_map.count(prefix + "text_encoders.llm.text_connector.proj.weight")) {
-                    return VERSION_P_H3_IMAGE;
-                }
-            }
-            auto connector = tensor_storage_map.find("embeddings_connectors.proj.weight");
-            if (connector != tensor_storage_map.end() && connector->second.logical_ne0() == 12800 &&
-                connector->second.ne[1] == 5120 && tensor_storage_map.count("embeddings_connectors.sink_embedding")) {
-                return VERSION_P_H3_IMAGE;
-            }
             return VERSION_MINIMAX_H3;
         }
         if (name == "model.diffusion_model.y_embedder.layer_pool.weight" &&

src/name_conversion.cpp

@@ -1741,7 +1741,7 @@ std::string convert_tensor_name(std::string name, SDVersion version) {
 
     const std::vector<std::pair<std::string, std::string>> connector_prefix_map = {
         {"embeddings_connectors.text_encoders.", "text_encoders."},
-        {"embeddings_connectors.", sd_version_is_minimax_h3_image(version) ? "text_encoders.ph3_image_connector." : ""},
+        {"embeddings_connectors.", sd_version_is_minimax_h3(version) ? "text_encoders.ph3_image_connector." : ""},
     };
     replace_with_prefix_map(name, connector_prefix_map);