CMakeLists.txt

@@ -294,8 +294,6 @@ endif()
 
 if(MSVC)
     target_compile_options(${SD_LIB} PRIVATE $<$<COMPILE_LANGUAGE:CXX>:/bigobj>)
-    # ggml backends can throw C++ exceptions through their C API.
-    target_compile_options(${SD_LIB} PRIVATE $<$<AND:$<COMPILE_LANGUAGE:CXX>,$<CXX_COMPILER_ID:MSVC>>:/EHsc->)
 endif()
 
 if(APPLE)

docs/backend.md

@@ -5,8 +5,7 @@
 - `--backend` selects the runtime backend used to execute model graphs.
 - `--params-backend` selects where model parameters are kept.
 
-If `--params-backend` is not set, auto-fit chooses parameter placement. With
+If `--params-backend` is not set, parameters use the same backend as their module runtime backend.
-`--auto-fit off`, parameters use the same backend as their module runtime backend.
 
 ## Syntax
 
@@ -130,21 +129,17 @@ warning.
 ## Automatic placement (`--auto-fit on|off`)
 
 `--auto-fit` requires `on` or `off` and defaults to `on` when omitted.
-Explicit `--params-backend` assignments disable auto-fit,
+Explicit `--backend` or `--params-backend` assignments disable auto-fit,
 regardless of argument order, even with `--auto-fit on`.
 
-Auto-fit preserves explicit `--backend` assignments, including per-module
+When enabled, auto-fit uses one GPU for `diffusion` / `te` / `vae` computation. It chooses
-assignments and device lists. For modules without a runtime assignment, it chooses
+the GPU with the largest available memory budget (the first device on a tie),
-the GPU with the largest available memory budget (the first device on a tie).
+then derives parameter placements from the model metadata and the remaining
-It then derives parameter placements from the model metadata, each module's
+memory budgets. The chosen backend specifications are printed.
-compute devices, and the remaining memory budgets. The chosen backend
-specifications are printed.
 
 ```shell
 sd-cli -m model.safetensors -p "a cat" --auto-fit on
 sd-cli -m model.safetensors -p "a cat" --auto-fit on --max-vram cuda0=8,cuda1=14
-sd-cli -m model.safetensors -p "a cat" --backend cuda0
-sd-cli -m model.safetensors -p "a cat" --backend diffusion=cuda0,te=cpu,vae=cuda1
 sd-cli -m model.safetensors -p "a cat" --auto-fit off
 ```
 
@@ -154,16 +149,11 @@ GiB", and with no budget set each device's free memory minus a 512 MiB margin
 is used. These resolved GPU budgets, including the safety margin, also drive
 the runner's graph-cut capacity checks.
 
-Runtime capacity checks also leave 512 MiB of currently free device memory for
-backend scratch buffers and pipelines, including with explicit backend assignments.
-They cap stale free-memory reports by the device's total memory minus tracked
-resident allocations and reject reports that exceed the device's total memory.
-
 Components are considered in `diffusion`, `te`, `vae` order so that repeatedly
 used diffusion weights have priority. Each component's weights use the first
 storage location with enough remaining budget:
 
-1. The component's compute GPU, leaving estimated space for computation and weight staging.
+1. The main GPU, leaving estimated space for computation and weight staging.
 2. CPU RAM, reserving the larger of 2 GiB or 10% of available RAM for other work.
 3. Another GPU, choosing the one with the largest remaining budget that fits.
 4. Disk, reloading weights on demand.
@@ -180,17 +170,10 @@ weight to be copied again at every step.
 RAM and GPU budgets are shared across components. Each component uses a single
 parameter backend; several other GPUs' capacities are not combined to store
 one component. If available RAM cannot be queried, RAM residency is skipped.
-Weights stored on another GPU are copied to the component's compute devices for
+Other GPUs store weights only: weights are copied to the main GPU for execution.
-execution. CPU modules use RAM or disk. Compute reserves and cache priority are
+Auto-fit does not select multi-GPU layer/row computation, so `--split-mode` does
-accounted for separately on each device, so a CPU module does not reserve GPU
+not change its placements. Use explicit backend assignments for multi-GPU
-space. Storage on another module's GPU also leaves room for that module's work.
+computation.
-
-Auto-fit does not select multi-GPU layer/row computation itself. Explicit device
-lists and `--split-mode` still control that computation. Before the runners have
-built their split plans, auto-fit conservatively counts the full component size
-on each listed GPU when checking residency and cache space. This can offload
-parameters even when a split layout would fit; use `--auto-fit off` to keep the
-default split-device parameter placement.
 
 For example, a diffusion model whose full weights exceed the main GPU's budget
 can use `--backend diffusion=cuda0 --params-backend diffusion=cpu` when RAM is
@@ -309,7 +292,6 @@ The example CLI/server still accepts these older CPU placement flags as compatib
 Because this default is inserted first, later explicit `--params-backend` entries can still override it, for example `--offload-to-cpu --params-backend te=disk` keeps non-TE parameters on CPU and reloads TE parameters from disk.
 
 Library callers should set `backend` and `params_backend` directly. `sd_ctx_params_init()`
-enables `auto_fit` by default; a nonempty `params_backend` assignment disables it.
+enables `auto_fit` by default; nonempty `backend` or `params_backend` assignments disable it.
-The `backend` assignment constrains auto-fit's compute placement.
 The old CPU/offload fields are no longer part of the C API. Explicit `--backend` and
 `--params-backend` assignments are preferred for new commands.

docs/performance.md

@@ -27,7 +27,7 @@ Using `--offload-to-cpu` allows you to offload weights to the CPU, saving VRAM w
 
 ## Use params backend to reduce VRAM or RAM usage.
 
-`--params-backend` controls where model parameters are kept. If it is not set, auto-fit chooses parameter placement while preserving `--backend`. With `--auto-fit off`, parameters use the same backend as `--backend`, so a GPU runtime backend also keeps parameters in VRAM.
+`--params-backend` controls where model parameters are kept. If it is not set, parameters use the same backend as `--backend`, so a GPU runtime backend also keeps parameters in VRAM.
 
 Use CPU params to reduce VRAM usage:
 

examples/common/common.cpp

@@ -720,9 +720,9 @@ ArgOptions SDContextParams::get_options() {
          }},
         {"",
          "--auto-fit",
-         "on|off (default: on). Preserve --backend (otherwise select one GPU) and place weights on the compute GPU, "
+         "on|off (default: on). Use one GPU for diffusion/te/vae computation and place weights on that GPU, "
          "RAM, another GPU, or disk in that order, according to available memory (--max-vram limits GPU budgets). "
-         "Disabled by explicit --params-backend; uses automatic graph segmentation when needed",
+         "Disabled by explicit --backend or --params-backend; uses automatic graph segmentation when needed",
          on_auto_fit_arg},
         {"",
          "--type",

src/conditioning/conditioner.hpp

@@ -1254,10 +1254,7 @@ struct FluxCLIPEmbedder : public Conditioner {
                                              true,
                                              clip_skip,
                                              false);
-                    if (pooled.empty()) {
+                    GGML_ASSERT(!pooled.empty());
-                        LOG_ERROR("Flux CLIP-L encoding failed");
-                        return {};
-                    }
                 } else {
                     pooled = sd::Tensor<float>::zeros({768});
                 }
@@ -1276,10 +1273,7 @@ struct FluxCLIPEmbedder : public Conditioner {
                                                   input_ids,
                                                   sd::Tensor<float>(),
                                                   false);
-                if (chunk_hidden_states.empty()) {
+                GGML_ASSERT(!chunk_hidden_states.empty());
-                    LOG_ERROR("Flux T5 encoding failed at chunk %d/%zu", chunk_idx + 1, chunk_count);
-                    return {};
-                }
                 chunk_hidden_states = ::apply_token_weights(std::move(chunk_hidden_states), chunk_weights);
                 if (zero_out_masked) {
                     chunk_hidden_states.fill_(0.0f);

src/core/backend_fit.cpp

@@ -58,15 +58,9 @@ namespace sd::backend_fit {
         size_t params_device           = SIZE_MAX;
     };
 
-    struct Runtime {
-        std::string name;
-        std::vector<size_t> devices;
-    };
-
     struct Plan {
         bool valid         = false;
         size_t main_device = SIZE_MAX;
-        std::vector<Runtime> runtimes;
         std::vector<Decision> decisions;
     };
 
@@ -127,14 +121,11 @@ namespace sd::backend_fit {
         return name;
     }
 
-    static std::vector<Device> enumerate_gpu_devices(const sd::ggml_graph_cut::MaxVramAssignment& budgets,
+    static std::vector<Device> enumerate_gpu_devices(const sd::ggml_graph_cut::MaxVramAssignment& budgets) {
-                                                     bool include_other_devices) {
         std::vector<Device> out;
         for (size_t i = 0; i < ggml_backend_dev_count(); ++i) {
             ggml_backend_dev_t dev = ggml_backend_dev_get(i);
-            const auto type        = ggml_backend_dev_type(dev);
+            if (ggml_backend_dev_type(dev) != GGML_BACKEND_DEVICE_TYPE_GPU) {
-            if (type != GGML_BACKEND_DEVICE_TYPE_GPU &&
-                (!include_other_devices || type == GGML_BACKEND_DEVICE_TYPE_CPU)) {
                 continue;
             }
             Device device;
@@ -192,29 +183,18 @@ namespace sd::backend_fit {
         return -1;
     }
 
-    static size_t select_main_device(const std::vector<Device>& devices) {
-        size_t main_device = SIZE_MAX;
-        for (size_t di = 0; di < devices.size(); ++di) {
-            if (devices[di].budget_bytes > 0 &&
-                (main_device == SIZE_MAX || devices[di].budget_bytes > devices[main_device].budget_bytes)) {
-                main_device = di;
-            }
-        }
-        return main_device;
-    }
-
     static Plan compute_plan(const std::vector<Component>& components,
                              const std::vector<Device>& devices,
-                             int64_t ram_budget_bytes,
+                             int64_t ram_budget_bytes) {
-                             const std::vector<Runtime>& runtimes = {}) {
         Plan plan;
-        plan.main_device = select_main_device(devices);
+        for (size_t di = 0; di < devices.size(); ++di) {
-        plan.runtimes    = runtimes;
+            if (devices[di].budget_bytes > 0 &&
-        if (plan.runtimes.empty()) {
+                (plan.main_device == SIZE_MAX || devices[di].budget_bytes > devices[plan.main_device].budget_bytes)) {
-            if (plan.main_device == SIZE_MAX) {
+                plan.main_device = di;
-                return plan;
             }
-            plan.runtimes.resize(components.size(), {devices[plan.main_device].name, {plan.main_device}});
+        }
+        if (plan.main_device == SIZE_MAX) {
+            return plan;
         }
 
         std::vector<size_t> order(components.size());
@@ -232,27 +212,6 @@ namespace sd::backend_fit {
         ram_budget_bytes = std::max<int64_t>(ram_budget_bytes, 0);
         plan.decisions.resize(components.size());
 
-        auto uses_device = [&](size_t ci, size_t di) {
-            const auto& runtime_devices = plan.runtimes[ci].devices;
-            return std::find(runtime_devices.begin(), runtime_devices.end(), di) != runtime_devices.end();
-        };
-        auto headroom_for = [&](size_t ci, size_t di) {
-            // Higher-priority offloaded weights need cache space on their compute devices.
-            int64_t headroom = 0;
-            for (size_t other = 0; other < components.size(); ++other) {
-                if (components[other].params_bytes == 0 || !uses_device(other, di)) {
-                    continue;
-                }
-                const bool resident          = other == ci || plan.decisions[other].params_location == ParamsLocation::MAIN_GPU;
-                const int64_t cached_weights = components[other].kind < components[ci].kind
-                                                   ? components[other].params_bytes
-                                                   : components[other].staging_bytes;
-                headroom                     = std::max(headroom, components[other].reserve_bytes +
-                                                                      (resident ? 0 : cached_weights));
-            }
-            return headroom;
-        };
-
         for (size_t ci : order) {
             const Component& comp = components[ci];
             Decision& decision    = plan.decisions[ci];
@@ -260,19 +219,24 @@ namespace sd::backend_fit {
                 continue;
             }
 
-            const auto& runtime_devices = plan.runtimes[ci].devices;
+            // Higher-priority offloaded weights need GPU cache space across graph runs.
-            const bool fits_runtime     = !runtime_devices.empty() &&
+            int64_t headroom = 0;
-                                      std::all_of(runtime_devices.begin(), runtime_devices.end(), [&](size_t di) {
+            for (size_t other = 0; other < components.size(); ++other) {
-                                          const int64_t headroom = headroom_for(ci, di);
+                if (components[other].params_bytes == 0) {
-                                          return headroom <= remaining[di] && comp.params_bytes <= remaining[di] - headroom;
+                    continue;
-                                      });
-            if (fits_runtime) {
-                decision.params_location = ParamsLocation::MAIN_GPU;
-                decision.params_device   = runtime_devices.front();
-                // Exact split allocations are unavailable until the runners build their plans.
-                for (size_t di : runtime_devices) {
-                    remaining[di] -= comp.params_bytes;
                 }
+                const bool resident          = other == ci || plan.decisions[other].params_location == ParamsLocation::MAIN_GPU;
+                const int64_t cached_weights = components[other].kind < comp.kind
+                                                   ? components[other].params_bytes
+                                                   : components[other].staging_bytes;
+                headroom                     = std::max(headroom, components[other].reserve_bytes +
+                                                                      (resident ? 0 : cached_weights));
+            }
+            int64_t& main_remaining = remaining[plan.main_device];
+            if (headroom <= main_remaining && comp.params_bytes <= main_remaining - headroom) {
+                decision.params_location = ParamsLocation::MAIN_GPU;
+                decision.params_device   = plan.main_device;
+                main_remaining -= comp.params_bytes;
                 continue;
             }
             if (comp.params_bytes <= ram_budget_bytes) {
@@ -280,14 +244,10 @@ namespace sd::backend_fit {
                 ram_budget_bytes -= comp.params_bytes;
                 continue;
             }
-            if (runtime_devices.empty()) {
-                continue;
-            }
 
             size_t best = SIZE_MAX;
             for (size_t di = 0; di < devices.size(); ++di) {
-                const int64_t headroom = headroom_for(ci, di);
+                if (di != plan.main_device && comp.params_bytes <= remaining[di] &&
-                if (!uses_device(ci, di) && headroom <= remaining[di] && comp.params_bytes <= remaining[di] - headroom &&
                     (best == SIZE_MAX || remaining[di] > remaining[best])) {
                     best = di;
                 }
@@ -320,7 +280,7 @@ namespace sd::backend_fit {
                            const std::vector<Device>& devices,
                            int64_t free_ram,
                            int64_t ram_budget) {
-        LOG_INFO("auto-fit plan:");
+        LOG_INFO("auto-fit plan (single-GPU compute on %s):", devices[plan.main_device].name.c_str());
         LOG_INFO("  devices:");
         for (const Device& device : devices) {
             LOG_INFO("    %-12s %-32s free %6lld MiB, budget %6lld MiB",
@@ -333,19 +293,17 @@ namespace sd::backend_fit {
             LOG_INFO("    RAM          free %6lld MiB, params budget %6lld MiB",
                      (long long)(free_ram / MiB), (long long)(ram_budget / MiB));
         }
-        LOG_INFO("  compute-device weight cache priority: diffusion > te > vae");
+        LOG_INFO("  main-GPU weight cache priority: diffusion > te > vae");
-        LOG_INFO("  components (params: compute device -> RAM -> other GPU -> disk):");
+        LOG_INFO("  components (params: main GPU -> RAM -> other GPU -> disk):");
         for (size_t ci = 0; ci < components.size(); ++ci) {
             const Component& comp = components[ci];
             if (comp.params_bytes == 0) {
                 continue;
             }
-            const std::string params = plan.decisions[ci].params_location == ParamsLocation::MAIN_GPU
+            const std::string params = params_backend_name(plan.decisions[ci], devices);
-                                           ? plan.runtimes[ci].name
-                                           : params_backend_name(plan.decisions[ci], devices);
             LOG_INFO("    %-12s params %6lld MiB, compute reserve %5lld MiB -> compute %s, params %s",
                      comp.name, (long long)(comp.params_bytes / MiB), (long long)(comp.reserve_bytes / MiB),
-                     plan.runtimes[ci].name.c_str(), params.c_str());
+                     devices[plan.main_device].name.c_str(), params.c_str());
         }
     }
 
@@ -370,51 +328,6 @@ namespace sd::backend_fit {
         return "";
     }
 
-    static bool resolve_runtimes(const std::vector<Component>& components,
-                                 const std::vector<Device>& devices,
-                                 std::string& runtime_spec,
-                                 std::vector<Runtime>& runtimes,
-                                 std::string& error) {
-        SDBackendAssignment assignment;
-        if (!sd_parse_backend_assignment(runtime_spec, &assignment, &error)) {
-            return false;
-        }
-        const size_t main_device        = select_main_device(devices);
-        const SDBackendModule modules[] = {SDBackendModule::DIFFUSION, SDBackendModule::TE, SDBackendModule::VAE};
-        for (const Component& comp : components) {
-            std::string name = assignment.get(modules[int(comp.kind)]);
-            if (name.empty()) {
-                name = main_device == SIZE_MAX ? "cpu" : devices[main_device].name;
-                if (comp.params_bytes > 0) {
-                    append_assignment(runtime_spec, module_key(comp.kind), name);
-                }
-            }
-            Runtime runtime;
-            for (const std::string& part : split_string(name, '&')) {
-                if (trim(part).empty()) {
-                    continue;
-                }
-                const std::string resolved = sd_backend_resolve_name(part);
-                if (resolved.empty()) {
-                    error = "backend '" + part + "' was not found";
-                    return false;
-                }
-                if (!runtime.name.empty()) {
-                    runtime.name += "&";
-                }
-                runtime.name += resolved;
-                for (size_t di = 0; di < devices.size(); ++di) {
-                    if (devices[di].name == resolved &&
-                        std::find(runtime.devices.begin(), runtime.devices.end(), di) == runtime.devices.end()) {
-                        runtime.devices.push_back(di);
-                    }
-                }
-            }
-            runtimes.push_back(std::move(runtime));
-        }
-        return true;
-    }
-
     bool derive_backend_specs(ModelLoader& loader,
                               ggml_type override_wtype,
                               sd::ggml_graph_cut::MaxVramAssignment& budgets,
@@ -426,18 +339,12 @@ namespace sd::backend_fit {
             return false;
         }
 
-        // Resolve once to ensure dynamic backends are loaded before enumerating devices.
+        const auto components    = estimate_components(loader, override_wtype);
-        sd_backend_resolve_name("");
+        const auto devices       = enumerate_gpu_devices(budgets);
-        const auto components = estimate_components(loader, override_wtype);
-        const auto devices    = enumerate_gpu_devices(budgets, !runtime_spec.empty());
-        std::vector<Runtime> runtimes;
-        if (!runtime_spec.empty() && !resolve_runtimes(components, devices, runtime_spec, runtimes, error)) {
-            LOG_ERROR("%s", error.c_str());
-            return false;
-        }
         const int64_t free_ram   = available_ram_bytes();
         const int64_t ram_budget = std::max<int64_t>(free_ram - std::max<int64_t>(2048 * MiB, free_ram / 10), 0);
-        const auto plan          = compute_plan(components, devices, ram_budget, runtimes);
+        const auto plan          = compute_plan(components, devices, ram_budget);
+        runtime_spec.clear();
         params_spec.clear();
         if (!plan.valid) {
             if (devices.empty()) {
@@ -455,9 +362,7 @@ namespace sd::backend_fit {
                 continue;
             }
             const char* key = module_key(components[ci].kind);
-            if (runtimes.empty()) {
+            append_assignment(runtime_spec, key, devices[plan.main_device].name);
-                append_assignment(runtime_spec, key, plan.runtimes[ci].name);
-            }
             if (plan.decisions[ci].params_location != ParamsLocation::MAIN_GPU) {
                 append_assignment(params_spec, key, params_backend_name(plan.decisions[ci], devices));
             }

src/core/compute_workspace.cpp

@@ -2,14 +2,12 @@
 
 #include <algorithm>
 #include <cstring>
-#include <exception>
 #include <map>
 #include <unordered_map>
 #include <unordered_set>
 
 #include "core/ggml_extend_backend.h"
 #include "core/ggml_graph_cut.h"
-#include "core/util.h"
 #include "ggml-cpu.h"
 #include "ggml/src/ggml-impl.h"
 
@@ -230,23 +228,11 @@ namespace sd {
         }
     }
 
-    bool ComputeWorkspace::segment_end() noexcept {
+    void ComputeWorkspace::segment_end() {
-        if (!active_) {
+        if (active_) {
-            return true;
-        }
-        // Outer cleanup guards must not retry a failed backend submission.
-        active_ = false;
-        try {
             synchronize();
-            return true;
+            active_ = false;
-        } catch (const std::exception& error) {
-            LOG_ERROR("%s workspace synchronization failed during segment cleanup: %s",
-                      ggml_backend_name(backend_), error.what());
-        } catch (...) {
-            LOG_ERROR("%s workspace synchronization failed during segment cleanup: unknown exception",
-                      ggml_backend_name(backend_));
         }
-        return false;
     }
 
     bool ComputeWorkspace::release() {

src/core/compute_workspace.h

@@ -51,7 +51,7 @@ namespace sd {
             const std::function<ggml_backend_t(const ggml_tensor*)>& external_backend,
             const AssignNodes& assign_nodes);
         void synchronize() const;
-        bool segment_end() noexcept;
+        void segment_end();
         bool release();
         bool active() const { return active_; }
         ggml_backend_sched_t scheduler() const { return scheduler_; }

src/core/ggml_extend_backend.cpp

@@ -593,7 +593,7 @@ static ggml_backend_t sd_get_default_backend() {
     return backend;
 }
 
-bool sd_parse_backend_assignment(const std::string& spec, SDBackendAssignment* assignment, std::string* error) {
+static bool sd_parse_backend_assignment(const std::string& spec, SDBackendAssignment* assignment, std::string* error) {
     if (assignment == nullptr) {
         return false;
     }
@@ -660,13 +660,7 @@ void SDBackendAssignment::set_module(SDBackendModule module, const std::string&
 }
 
 void SDBackendHandleDeleter::operator()(ggml_backend_t backend) const {
-    try {
+    ggml_backend_free(backend);
-        ggml_backend_free(backend);
-    } catch (const std::exception& error) {
-        LOG_ERROR("backend cleanup failed: %s", error.what());
-    } catch (...) {
-        LOG_ERROR("backend cleanup failed: unknown exception");
-    }
 }
 
 SDBackendManager::~SDBackendManager() {

src/core/ggml_extend_backend.h

@@ -93,7 +93,6 @@ ggml_status sd_backend_graph_compute_with_eval_callback(ggml_backend_t backend,
                                                         sd_graph_eval_callback_t callback_eval,
                                                         void* callback_eval_user_data);
 std::string sd_backend_resolve_name(const std::string& name);
-bool sd_parse_backend_assignment(const std::string& spec, SDBackendAssignment* assignment, std::string* error);
 const char* sd_backend_module_name(SDBackendModule module);
 void ggml_ext_im_set_f32_1d(const struct ggml_tensor* tensor, int i, float value);
 bool add_rpc_devices(const std::string& servers);

src/core/ggml_runner.cpp

@@ -1,5 +1,4 @@
 #include <algorithm>
-#include <exception>
 #include <map>
 #include <utility>
 
@@ -643,15 +642,8 @@ std::optional<sd::Tensor<float>> GGMLRunner::compute(get_graph_cb_t get_graph,
                 params_tensor_set_.insert(parameter);
         }
     }
-    std::optional<sd::Tensor<float>> output;
+    auto output = execute_graph(graph, n_threads, no_return, read_outputs);
-    try {
+    success     = output.has_value();
-        output = execute_graph(graph, n_threads, no_return, read_outputs);
-    } catch (const std::exception& error) {
-        LOG_ERROR("%s graph execution failed on %s: %s", get_desc().c_str(),
-                  ggml_backend_name(runtime_backend), error.what());
-        return std::nullopt;
-    }
-    success = output.has_value();
     if (success) {
         cache_.graph_end(true);
     }
@@ -964,9 +956,6 @@ std::optional<Tensor<float>> GGMLRunner::execute_graph(ggml_cgraph* graph, int n
                 }
             }
         }
-        if (!workspace_.segment_end()) {
-            return fail_segment("workspace synchronization");
-        }
         // Final outputs and their callbacks may still be views of consumed cuts.
         cut_cache_.prune(segment.future_cut_names);
     }

src/model_manager.cpp

@@ -1584,35 +1584,18 @@ ModelManager::CapacityCheck ModelManager::check_capacity(
     if (request.compute_backend == nullptr || sd_backend_is_cpu(request.compute_backend)) {
         return result;
     }
-    auto add             = [](size_t a, size_t b) { return b > SIZE_MAX - a ? SIZE_MAX : a + b; };
+    auto add                     = [](size_t a, size_t b) { return b > SIZE_MAX - a ? SIZE_MAX : a + b; };
-    const size_t missing = compute_backend_alloc_size(states, true);
+    const size_t missing         = compute_backend_alloc_size(states, true);
-    // Backend scratch buffers and pipelines are not included in graph measurements.
+    result.required_device_bytes = add(request.pending_allocation_bytes, missing);
-    constexpr size_t safety_margin = 512ULL * 1024ULL * 1024ULL;
+    result.required_budget_bytes = add(request.runtime_peak_bytes(), missing);
-    result.required_device_bytes   = add(add(request.pending_allocation_bytes, missing), safety_margin);
+    auto device                  = ggml_backend_get_device(request.compute_backend);
-    result.required_budget_bytes   = add(request.runtime_peak_bytes(), missing);
+    if (device != nullptr) {
-    auto available_device_bytes    = [&](ggml_backend_t backend) {
-        auto device = ggml_backend_get_device(backend);
-        if (device == nullptr) {
-            return SIZE_MAX;
-        }
         size_t free_bytes = 0, total_bytes = 0;
         ggml_backend_dev_memory(device, &free_bytes, &total_bytes);
-        if (free_bytes == 0 && total_bytes == 0) {
+        if (free_bytes != 0 || total_bytes != 0) {
-            return SIZE_MAX;
+            result.available_device_bytes = free_bytes;
         }
-        // Vulkan's heap budget subtraction can underflow when usage exceeds the budget.
+    }
-        if (total_bytes > 0 && free_bytes > total_bytes) {
-            return size_t{0};
-        }
-        const size_t resident = add(compute_backend_resident_bytes(backend),
-                                       add(other_runtime_resident_bytes(request.owner_id, backend),
-                                           request.runtime_resident_bytes));
-        if (total_bytes > 0) {
-            free_bytes = std::min(free_bytes, resident < total_bytes ? total_bytes - resident : 0);
-        }
-        return free_bytes;
-    };
-    result.available_device_bytes = available_device_bytes(request.compute_backend);
     if (request.max_backend_bytes > 0) {
         const size_t resident         = add(compute_backend_resident_bytes(request.compute_backend),
                                             other_runtime_resident_bytes(request.owner_id, request.compute_backend));
@@ -1636,7 +1619,11 @@ ModelManager::CapacityCheck ModelManager::check_capacity(
     // GGML exposes only a split buffer's total size, not per-device allocations.
     // Charge that upper bound on every participant instead of undercounting a shard.
     for (const auto& entry : split_devices) {
-        result.available_device_bytes = std::min(result.available_device_bytes, available_device_bytes(entry.first));
+        size_t free_bytes = 0, total_bytes = 0;
+        ggml_backend_dev_memory(ggml_backend_get_device(entry.first), &free_bytes, &total_bytes);
+        if (free_bytes != 0 || total_bytes != 0) {
+            result.available_device_bytes = std::min(result.available_device_bytes, free_bytes);
+        }
         if (entry.second > 0) {
             const size_t resident         = add(compute_backend_resident_bytes(entry.first),
                                                 other_runtime_resident_bytes(request.owner_id, entry.first));
@@ -1752,18 +1739,12 @@ bool ModelManager::ensure_compute_backend_capacity(
         }
     }
 
-    const auto capacity                = check_capacity(request, required_states);
+    const auto capacity = check_capacity(request, required_states);
-    const std::string available_device = capacity.available_device_bytes == SIZE_MAX
+    LOG_WARN("model manager cannot make enough memory available on %s: need %.2f MB device / %.2f MB budget, available %.2f MB device / %.2f MB budget",
-                                             ? "unknown"
-                                             : sd_format("%.2f MB", capacity.available_device_bytes / (1024.0 * 1024.0));
-    const std::string available_budget = capacity.available_budget_bytes == SIZE_MAX
-                                             ? "unlimited"
-                                             : sd_format("%.2f MB", capacity.available_budget_bytes / (1024.0 * 1024.0));
-    LOG_WARN("model manager cannot make enough memory available on %s: need %.2f MB device / %.2f MB budget, available %s device / %s budget",
              ggml_backend_name(compute_backend),
              capacity.required_device_bytes / (1024.0 * 1024.0),
              capacity.required_budget_bytes / (1024.0 * 1024.0),
-             available_device.c_str(),
+             capacity.available_device_bytes / (1024.0 * 1024.0),
-             available_budget.c_str());
+             capacity.available_budget_bytes / (1024.0 * 1024.0));
     return false;
 }

src/model_manager.h

@@ -31,7 +31,7 @@ public:
     };
 
 private:
-    static constexpr size_t MAX_RESIDENCY_BLOCK_BYTES = 1024ULL * 1024ULL * 1024ULL;
+    static constexpr size_t MAX_RESIDENCY_BLOCK_BYTES = 64ULL * 1024ULL * 1024ULL;
 
     struct TensorState {
         std::string name;

src/pipeline/diffusion_engine.cpp

@@ -862,7 +862,7 @@ bool StableDiffusionGGML::init(const sd_ctx_params_t* sd_ctx_params) {
     backend_spec              = SAFE_STR(sd_ctx_params->backend);
     params_backend_spec       = SAFE_STR(sd_ctx_params->params_backend);
     split_mode_spec           = SAFE_STR(sd_ctx_params->split_mode);
-    auto_fit_enabled          = sd_ctx_params->auto_fit && params_backend_spec.empty();
+    auto_fit_enabled          = sd_ctx_params->auto_fit && backend_spec.empty() && params_backend_spec.empty();
     max_vram_assignment.reset(0.f);
     {
         std::string error;

src/pipeline/image.cpp

@@ -438,10 +438,6 @@ namespace sd::pipeline {
         condition_params.zero_out_masked = false;
         auto cond                        = sd->cond_stage_model->get_learned_condition(sd->n_threads,
                                                                                        condition_params);
-        if (cond.empty()) {
-            LOG_ERROR("failed to encode prompt");
-            return std::nullopt;
-        }
         if (cond.c_concat.empty() && ref_image_params.pass_to_dit) {
             cond.c_concat = latents->concat_latent;  // TODO: optimize
         }
@@ -473,10 +469,6 @@ namespace sd::pipeline {
                 condition_params.zero_out_masked = zero_out_masked;
                 uncond                           = sd->cond_stage_model->get_learned_condition(sd->n_threads,
                                                                                                condition_params);
-                if (uncond.empty()) {
-                    LOG_ERROR("failed to encode negative prompt");
-                    return std::nullopt;
-                }
             }
             if (uncond.c_concat.empty() && ref_image_params.pass_to_dit) {
                 uncond.c_concat = latents->concat_latent;  // TODO: optimize
@@ -502,10 +494,6 @@ namespace sd::pipeline {
                 }
                 img_uncond = sd->cond_stage_model->get_learned_condition(sd->n_threads,
                                                                          condition_params);
-                if (img_uncond.empty()) {
-                    LOG_ERROR("failed to encode image guidance prompt");
-                    return std::nullopt;
-                }
                 if (img_uncond.c_concat.empty() && ref_image_params.pass_to_dit) {
                     img_uncond.c_concat = latents->img_uncond_concat_latent;  // TODO: optimize
                 }