feat: add sdcpp api support (#1407)

examples/server/api.md

@@ -0,0 +1,794 @@
+# stable-diffusion.cpp Server APIs
+
+This document describes the server-facing APIs exposed by `examples/server`.
+
+The server currently exposes three API families:
+
+- `OpenAI API` under `/v1/...`
+- `Stable Diffusion WebUI API` under `/sdapi/v1/...`
+- `sdcpp API` under `/sdcpp/v1/...`
+
+The `sdcpp API` is the native API surface.
+Its request schema is also the canonical schema for `sd_cpp_extra_args`.
+
+Global LoRA rule:
+
+- Server APIs do not parse LoRA tags embedded inside `prompt`.
+- `<lora:...>` prompt syntax is intentionally unsupported in `OpenAI API`, `sdapi`, and `sdcpp API`.
+- LoRA must be passed through structured API fields when the API supports it.
+
+## Overview
+
+### OpenAI API
+
+Compatibility API shaped like OpenAI image endpoints.
+
+Current generation-related endpoints include:
+
+- `POST /v1/images/generations`
+- `POST /v1/images/edits`
+- `GET /v1/models`
+
+### Stable Diffusion WebUI API
+
+Compatibility API shaped like the AUTOMATIC1111 / WebUI endpoints.
+
+Current generation-related endpoints include:
+
+- `POST /sdapi/v1/txt2img`
+- `POST /sdapi/v1/img2img`
+- `GET /sdapi/v1/loras`
+- `GET /sdapi/v1/samplers`
+- `GET /sdapi/v1/schedulers`
+- `GET /sdapi/v1/sd-models`
+- `GET /sdapi/v1/options`
+
+### sdcpp API
+
+Native async API for `stable-diffusion.cpp`.
+
+Current endpoints include:
+
+- `GET /sdcpp/v1/capabilities`
+- `POST /sdcpp/v1/img_gen`
+- `GET /sdcpp/v1/jobs/{id}`
+- `POST /sdcpp/v1/jobs/{id}/cancel`
+- `POST /sdcpp/v1/vid_gen`
+
+`POST /sdcpp/v1/vid_gen` is currently exposed but returns `501 Not Implemented`.
+
+## `sd_cpp_extra_args`
+
+`sd_cpp_extra_args` is an extension mechanism for the compatibility APIs.
+
+Rules:
+
+- Its JSON schema is the same schema used by the native `sdcpp API`.
+- `OpenAI API` and `sdapi` can embed it inside `prompt`.
+- `sdcpp API` does not need it, because the request body already uses the native schema directly.
+
+Embedding format:
+
+```text
+normal prompt text <sd_cpp_extra_args>{"sample_params":{"sample_steps":28}}</sd_cpp_extra_args>
+```
+
+Behavior:
+
+- The server extracts the JSON block.
+- The JSON block is parsed using the same field rules as the `sdcpp API`.
+- The block is removed from the final prompt before generation.
+
+Intended use:
+
+- extend `OpenAI API` requests with native `stable-diffusion.cpp` controls
+- extend `sdapi` requests with native `stable-diffusion.cpp` controls
+
+Not intended use:
+
+- do not use `sd_cpp_extra_args` with `/sdcpp/v1/*`
+
+## OpenAI API
+
+### Purpose
+
+This family exists for client compatibility.
+
+Use it when you want OpenAI-style request and response shapes.
+
+### Native Extension
+
+`OpenAI API` supports `sd_cpp_extra_args` embedded inside `prompt`.
+
+The embedded JSON follows the `sdcpp API` request schema.
+
+### Supported Fields
+
+#### `POST /v1/images/generations`
+
+Currently supported top-level request fields:
+
+| Field | Type | Notes |
+| --- | --- | --- |
+| `prompt` | `string` | Required |
+| `n` | `integer` | Number of images |
+| `size` | `string` | Format `WIDTHxHEIGHT` |
+| `output_format` | `string` | `png`, `jpeg`, or `webp` |
+| `output_compression` | `integer` | Range is clamped to `0..100` |
+
+Native extension fields:
+
+- any `sdcpp API` fields embedded through `sd_cpp_extra_args` inside `prompt`
+
+Response fields:
+
+| Field | Type | Notes |
+| --- | --- | --- |
+| `created` | `integer` | Unix timestamp |
+| `output_format` | `string` | Final encoded image format |
+| `data` | `array<object>` | Generated image list |
+| `data[].b64_json` | `string` | Base64-encoded image bytes |
+
+#### `POST /v1/images/edits`
+
+Currently supported multipart form fields:
+
+| Field | Type | Notes |
+| --- | --- | --- |
+| `prompt` | `string` | Required |
+| `image[]` | `file[]` | Preferred image upload field |
+| `image` | `file` | Legacy single-image upload field |
+| `mask` | `file` | Optional mask image |
+| `n` | `integer` | Number of images |
+| `size` | `string` | Format `WIDTHxHEIGHT` |
+| `output_format` | `string` | `png` or `jpeg` |
+| `output_compression` | `integer` | Range is clamped to `0..100` |
+
+Native extension fields:
+
+- any `sdcpp API` fields embedded through `sd_cpp_extra_args` inside `prompt`
+
+Response fields:
+
+| Field | Type | Notes |
+| --- | --- | --- |
+| `created` | `integer` | Unix timestamp |
+| `output_format` | `string` | Final encoded image format |
+| `data` | `array<object>` | Generated image list |
+| `data[].b64_json` | `string` | Base64-encoded image bytes |
+
+#### `GET /v1/models`
+
+Response fields:
+
+| Field | Type | Notes |
+| --- | --- | --- |
+| `data` | `array<object>` | Available local models |
+| `data[].id` | `string` | Currently fixed to `sd-cpp-local` |
+| `data[].object` | `string` | Currently fixed to `model` |
+| `data[].owned_by` | `string` | Currently fixed to `local` |
+
+### Output Options
+
+`OpenAI API` supports response serialization controls such as:
+
+- `output_format`
+- `output_compression`
+
+### Notes
+
+- `OpenAI API` is synchronous from the HTTP client's perspective.
+- Native async job polling is not exposed through this family.
+- Prompt-embedded `<lora:...>` tags are intentionally unsupported.
+
+## Stable Diffusion WebUI API
+
+### Purpose
+
+This family exists for client compatibility with WebUI-style tools.
+
+Use it when you want `txt2img` / `img2img`-style endpoints and response shapes.
+
+### Native Extension
+
+`sdapi` supports `sd_cpp_extra_args` embedded inside `prompt`.
+
+The embedded JSON follows the `sdcpp API` request schema.
+
+This allows `sdapi` clients to use native `stable-diffusion.cpp` controls without changing the outer request format.
+
+### Supported Fields
+
+#### `POST /sdapi/v1/txt2img`
+
+Currently supported request fields:
+
+| Field | Type | Notes |
+| --- | --- | --- |
+| `prompt` | `string` | Required |
+| `negative_prompt` | `string` | Optional |
+| `width` | `integer` | Positive image width |
+| `height` | `integer` | Positive image height |
+| `steps` | `integer` | Sampling steps |
+| `cfg_scale` | `number` | Text CFG scale |
+| `seed` | `integer` | `-1` means random |
+| `batch_size` | `integer` | Number of images |
+| `clip_skip` | `integer` | Optional |
+| `sampler_name` | `string` | WebUI sampler name |
+| `scheduler` | `string` | Scheduler name |
+| `lora` | `array<object>` | Structured LoRA list |
+| `extra_images` | `array<string>` | Base64 or data URL images |
+
+Native extension fields:
+
+- any `sdcpp API` fields embedded through `sd_cpp_extra_args` inside `prompt`
+
+Response fields:
+
+| Field | Type | Notes |
+| --- | --- | --- |
+| `images` | `array<string>` | Base64-encoded PNG images |
+| `parameters` | `object` | Echo of the parsed outer request body |
+| `info` | `string` | Currently empty string |
+
+#### `POST /sdapi/v1/img2img`
+
+Currently supported request fields:
+
+| Field | Type | Notes |
+| --- | --- | --- |
+| all currently supported `txt2img` fields | same as above | Reused |
+| `init_images` | `array<string>` | Base64 or data URL images |
+| `mask` | `string` | Base64 or data URL image |
+| `inpainting_mask_invert` | `integer` or `boolean` | Treated as invert flag |
+| `denoising_strength` | `number` | Clamped to `0.0..1.0` |
+
+Native extension fields:
+
+- any `sdcpp API` fields embedded through `sd_cpp_extra_args` inside `prompt`
+
+Response fields:
+
+| Field | Type | Notes |
+| --- | --- | --- |
+| `images` | `array<string>` | Base64-encoded PNG images |
+| `parameters` | `object` | Echo of the parsed outer request body |
+| `info` | `string` | Currently empty string |
+
+#### Discovery / Compatibility Endpoints
+
+Currently exposed:
+
+- `GET /sdapi/v1/loras`
+- `GET /sdapi/v1/samplers`
+- `GET /sdapi/v1/schedulers`
+- `GET /sdapi/v1/sd-models`
+- `GET /sdapi/v1/options`
+
+Response fields:
+
+`GET /sdapi/v1/loras`
+
+| Field | Type | Notes |
+| --- | --- | --- |
+| `[].name` | `string` | Display name derived from file stem |
+| `[].path` | `string` | Relative path under the configured LoRA directory |
+
+`GET /sdapi/v1/samplers`
+
+| Field | Type | Notes |
+| --- | --- | --- |
+| `[].name` | `string` | Sampler name |
+| `[].aliases` | `array<string>` | Currently contains the same single sampler name |
+| `[].options` | `object` | Currently empty object |
+
+`GET /sdapi/v1/schedulers`
+
+| Field | Type | Notes |
+| --- | --- | --- |
+| `[].name` | `string` | Scheduler name |
+| `[].label` | `string` | Same value as `name` |
+
+`GET /sdapi/v1/sd-models`
+
+| Field | Type | Notes |
+| --- | --- | --- |
+| `[].title` | `string` | Model stem |
+| `[].model_name` | `string` | Same value as `title` |
+| `[].filename` | `string` | Model filename |
+| `[].hash` | `string` | Placeholder compatibility value |
+| `[].sha256` | `string` | Placeholder compatibility value |
+| `[].config` | `null` | Currently always null |
+
+`GET /sdapi/v1/options`
+
+| Field | Type | Notes |
+| --- | --- | --- |
+| `samples_format` | `string` | Currently fixed to `png` |
+| `sd_model_checkpoint` | `string` | Model stem |
+
+### Notes
+
+- `sdapi` is synchronous from the HTTP client's perspective.
+- Prompt-embedded `<lora:...>` tags are intentionally unsupported.
+
+## sdcpp API
+
+### Purpose
+
+This is the native `stable-diffusion.cpp` API.
+
+Use it when you want:
+
+- async job submission
+- explicit native parameter control
+- frontend-oriented capability discovery
+
+### Job Model
+
+All async generation requests create a job.
+
+Job states:
+
+- `queued`
+- `generating`
+- `completed`
+- `failed`
+- `cancelled`
+
+Common job shape:
+
+```json
+{
+  "id": "job_01HTXYZABC",
+  "kind": "img_gen",
+  "status": "queued",
+  "created": 1775401200,
+  "started": null,
+  "completed": null,
+  "queue_position": 2,
+  "result": null,
+  "error": null
+}
+```
+
+Field types:
+
+| Field | Type |
+| --- | --- |
+| `id` | `string` |
+| `kind` | `string` |
+| `status` | `string` |
+| `created` | `integer` |
+| `started` | `integer \| null` |
+| `completed` | `integer \| null` |
+| `queue_position` | `integer` |
+| `result` | `object \| null` |
+| `error` | `object \| null` |
+
+### Endpoints
+
+#### `GET /sdcpp/v1/capabilities`
+
+Returns frontend-friendly capability metadata.
+
+Typical contents:
+
+| Field | Type |
+| --- | --- |
+| `model` | `object` |
+| `defaults` | `object` |
+| `loras` | `array<object>` |
+| `samplers` | `array<string>` |
+| `schedulers` | `array<string>` |
+| `output_formats` | `array<string>` |
+| `limits` | `object` |
+| `features` | `object` |
+
+Nested fields currently returned:
+
+`model`
+
+| Field | Type |
+| --- | --- |
+| `model.name` | `string` |
+| `model.stem` | `string` |
+| `model.path` | `string` |
+
+`defaults`
+
+| Field | Type |
+| --- | --- |
+| `defaults.prompt` | `string` |
+| `defaults.negative_prompt` | `string` |
+| `defaults.clip_skip` | `integer` |
+| `defaults.width` | `integer` |
+| `defaults.height` | `integer` |
+| `defaults.strength` | `number` |
+| `defaults.seed` | `integer` |
+| `defaults.batch_count` | `integer` |
+| `defaults.auto_resize_ref_image` | `boolean` |
+| `defaults.increase_ref_index` | `boolean` |
+| `defaults.control_strength` | `number` |
+| `defaults.sample_params` | `object` |
+| `defaults.sample_params.scheduler` | `string` |
+| `defaults.sample_params.sample_method` | `string` |
+| `defaults.sample_params.sample_steps` | `integer` |
+| `defaults.sample_params.eta` | `number \| null` |
+| `defaults.sample_params.shifted_timestep` | `integer` |
+| `defaults.sample_params.flow_shift` | `number \| null` |
+| `defaults.sample_params.guidance` | `object` |
+| `defaults.sample_params.guidance.txt_cfg` | `number` |
+| `defaults.sample_params.guidance.img_cfg` | `number \| null` |
+| `defaults.sample_params.guidance.distilled_guidance` | `number` |
+| `defaults.sample_params.guidance.slg` | `object` |
+| `defaults.sample_params.guidance.slg.layers` | `array<integer>` |
+| `defaults.sample_params.guidance.slg.layer_start` | `number` |
+| `defaults.sample_params.guidance.slg.layer_end` | `number` |
+| `defaults.sample_params.guidance.slg.scale` | `number` |
+| `defaults.vae_tiling_params` | `object` |
+| `defaults.vae_tiling_params.enabled` | `boolean` |
+| `defaults.vae_tiling_params.tile_size_x` | `integer` |
+| `defaults.vae_tiling_params.tile_size_y` | `integer` |
+| `defaults.vae_tiling_params.target_overlap` | `number` |
+| `defaults.vae_tiling_params.rel_size_x` | `number` |
+| `defaults.vae_tiling_params.rel_size_y` | `number` |
+| `defaults.cache_mode` | `string` |
+| `defaults.cache_option` | `string` |
+| `defaults.scm_mask` | `string` |
+| `defaults.scm_policy_dynamic` | `boolean` |
+| `defaults.output_format` | `string` |
+| `defaults.output_compression` | `integer` |
+
+`loras`
+
+| Field | Type |
+| --- | --- |
+| `loras[].name` | `string` |
+| `loras[].path` | `string` |
+
+`limits`
+
+| Field | Type |
+| --- | --- |
+| `limits.min_width` | `integer` |
+| `limits.max_width` | `integer` |
+| `limits.min_height` | `integer` |
+| `limits.max_height` | `integer` |
+| `limits.max_batch_count` | `integer` |
+| `limits.max_queue_size` | `integer` |
+
+`features`
+
+| Field | Type |
+| --- | --- |
+| `features.init_image` | `boolean` |
+| `features.mask_image` | `boolean` |
+| `features.control_image` | `boolean` |
+| `features.ref_images` | `boolean` |
+| `features.lora` | `boolean` |
+| `features.vae_tiling` | `boolean` |
+| `features.cache` | `boolean` |
+| `features.cancel_queued` | `boolean` |
+| `features.cancel_generating` | `boolean` |
+
+#### `POST /sdcpp/v1/img_gen`
+
+Submits an async image generation job.
+
+Successful submission returns `202 Accepted`.
+
+Example response:
+
+```json
+{
+  "id": "job_01HTXYZABC",
+  "kind": "img_gen",
+  "status": "queued",
+  "created": 1775401200,
+  "poll_url": "/sdcpp/v1/jobs/job_01HTXYZABC"
+}
+```
+
+Response fields:
+
+| Field | Type |
+| --- | --- |
+| `id` | `string` |
+| `kind` | `string` |
+| `status` | `string` |
+| `created` | `integer` |
+| `poll_url` | `string` |
+
+#### `GET /sdcpp/v1/jobs/{id}`
+
+Returns current job status.
+
+Typical status codes:
+
+- `200 OK`
+- `404 Not Found`
+- `410 Gone`
+
+#### `POST /sdcpp/v1/jobs/{id}/cancel`
+
+Attempts to cancel an accepted job.
+
+Typical status codes:
+
+- `200 OK`
+- `404 Not Found`
+- `409 Conflict`
+- `410 Gone`
+
+### Canonical Request Schema
+
+The `sdcpp API` request body is the canonical native schema.
+
+Example:
+
+```json
+{
+  "prompt": "a cat sitting on a chair",
+  "negative_prompt": "",
+  "clip_skip": -1,
+  "width": 1024,
+  "height": 1024,
+  "strength": 0.75,
+  "seed": -1,
+  "batch_count": 1,
+  "auto_resize_ref_image": true,
+  "increase_ref_index": false,
+  "control_strength": 0.9,
+  "embed_image_metadata": true,
+
+  "init_image": null,
+  "ref_images": [],
+  "mask_image": null,
+  "control_image": null,
+
+  "sample_params": {
+    "scheduler": "discrete",
+    "sample_method": "euler_a",
+    "sample_steps": 28,
+    "eta": 1.0,
+    "shifted_timestep": 0,
+    "custom_sigmas": [],
+    "flow_shift": 0.0,
+    "guidance": {
+      "txt_cfg": 7.0,
+      "img_cfg": 7.0,
+      "distilled_guidance": 3.5,
+      "slg": {
+        "layers": [7, 8, 9],
+        "layer_start": 0.01,
+        "layer_end": 0.2,
+        "scale": 0.0
+      }
+    }
+  },
+
+  "lora": [],
+
+  "vae_tiling_params": {
+    "enabled": false,
+    "tile_size_x": 0,
+    "tile_size_y": 0,
+    "target_overlap": 0.5,
+    "rel_size_x": 0.0,
+    "rel_size_y": 0.0
+  },
+
+  "cache_mode": "disabled",
+  "cache_option": "",
+  "scm_mask": "",
+  "scm_policy_dynamic": true,
+
+  "output_format": "png",
+  "output_compression": 100
+}
+```
+
+### LoRA Rules
+
+- The server only accepts explicit LoRA entries from the `lora` field.
+- Prompt-embedded `<lora:...>` tags are intentionally unsupported.
+- Clients should resolve LoRA usage through the structured `lora` array.
+
+### Image Encoding Rules
+
+Any image field accepts:
+
+- a raw base64 string, or
+- a data URL such as `data:image/png;base64,...`
+
+Channel expectations:
+
+- `init_image`: 3 channels
+- `ref_images[]`: 3 channels
+- `control_image`: 3 channels
+- `mask_image`: 1 channel
+
+If omitted or null:
+
+- single-image fields map to an empty `sd_image_t`
+- array fields map to `nullptr + count = 0`
+
+### Field Mapping Summary
+
+Top-level scalar fields:
+
+| Field | Type |
+| --- | --- |
+| `prompt` | `string` |
+| `negative_prompt` | `string` |
+| `clip_skip` | `integer` |
+| `width` | `integer` |
+| `height` | `integer` |
+| `strength` | `number` |
+| `seed` | `integer` |
+| `batch_count` | `integer` |
+| `auto_resize_ref_image` | `boolean` |
+| `increase_ref_index` | `boolean` |
+| `control_strength` | `number` |
+| `embed_image_metadata` | `boolean` |
+
+Image fields:
+
+| Field | Type |
+| --- | --- |
+| `init_image` | `string \| null` |
+| `ref_images` | `array<string>` |
+| `mask_image` | `string \| null` |
+| `control_image` | `string \| null` |
+
+LoRA fields:
+
+| Field | Type |
+| --- | --- |
+| `lora[].path` | `string` |
+| `lora[].multiplier` | `number` |
+| `lora[].is_high_noise` | `boolean` |
+
+Sampling fields:
+
+| Field | Type |
+| --- | --- |
+| `sample_params.scheduler` | `string` |
+| `sample_params.sample_method` | `string` |
+| `sample_params.sample_steps` | `integer` |
+| `sample_params.eta` | `number` |
+| `sample_params.shifted_timestep` | `integer` |
+| `sample_params.custom_sigmas` | `array<number>` |
+| `sample_params.flow_shift` | `number` |
+| `sample_params.guidance.txt_cfg` | `number` |
+| `sample_params.guidance.img_cfg` | `number` |
+| `sample_params.guidance.distilled_guidance` | `number` |
+| `sample_params.guidance.slg.layers` | `array<integer>` |
+| `sample_params.guidance.slg.layer_start` | `number` |
+| `sample_params.guidance.slg.layer_end` | `number` |
+| `sample_params.guidance.slg.scale` | `number` |
+
+Other native fields:
+
+| Field | Type |
+| --- | --- |
+| `vae_tiling_params` | `object` |
+| `cache_mode` | `string` |
+| `cache_option` | `string` |
+| `scm_mask` | `string` |
+| `scm_policy_dynamic` | `boolean` |
+
+HTTP-only output fields:
+
+| Field | Type |
+| --- | --- |
+| `output_format` | `string` |
+| `output_compression` | `integer` |
+
+### Optional Field Semantics
+
+Clients should preserve unset semantics for optional sampling fields.
+
+If a user has not explicitly provided one of these fields, the client should omit it instead of injecting a guessed fallback:
+
+- `sample_params.scheduler`
+- `sample_params.sample_method`
+- `sample_params.eta`
+- `sample_params.flow_shift`
+- `sample_params.guidance.img_cfg`
+
+### Completion Result
+
+Example completed job:
+
+```json
+{
+  "id": "job_01HTXYZABC",
+  "kind": "img_gen",
+  "status": "completed",
+  "created": 1775401200,
+  "started": 1775401203,
+  "completed": 1775401215,
+  "queue_position": 0,
+  "result": {
+    "output_format": "png",
+    "images": [
+      {
+        "index": 0,
+        "b64_json": "iVBORw0KGgoAAA..."
+      }
+    ]
+  },
+  "error": null
+}
+```
+
+### Failure Result
+
+Example failed job:
+
+```json
+{
+  "id": "job_01HTXYZABC",
+  "kind": "img_gen",
+  "status": "failed",
+  "created": 1775401200,
+  "started": 1775401203,
+  "completed": 1775401204,
+  "queue_position": 0,
+  "result": null,
+  "error": {
+    "code": "generation_failed",
+    "message": "generate_image returned empty results"
+  }
+}
+```
+
+### Cancelled Result
+
+Example cancelled job:
+
+```json
+{
+  "id": "job_01HTXYZABC",
+  "kind": "img_gen",
+  "status": "cancelled",
+  "created": 1775401200,
+  "started": null,
+  "completed": 1775401202,
+  "queue_position": 0,
+  "result": null,
+  "error": {
+    "code": "cancelled",
+    "message": "job cancelled by client"
+  }
+}
+```
+
+### Validation and Retention
+
+Recommended behavior:
+
+- malformed JSON returns `400`
+- invalid image payloads return `400`
+- invalid parameter structure returns `400`
+- queue full returns `429` or `503`
+- accepted runtime failures transition the job to `failed`
+- unsupported in-progress cancellation may return `409`
+
+Recommended retention controls:
+
+- pending job limit
+- completed job TTL
+- failed job TTL
+
+### Future `vid_gen`
+
+Future `vid_gen` should reuse the same async job model:
+
+- `POST /sdcpp/v1/vid_gen`
+- `GET /sdcpp/v1/jobs/{id}`
+- `POST /sdcpp/v1/jobs/{id}/cancel`
+
+Its request body should mirror `sd_vid_gen_params_t` in the same way that `img_gen` mirrors `sd_img_gen_params_t`.