mirror of
https://github.com/huggingface/diffusers.git
synced 2026-10-24 20:59:27 +03:00
527 lines
26 KiB
Markdown
527 lines
26 KiB
Markdown
<!--Copyright 2026 The HuggingFace Team. All rights reserved.
|
|
|
|
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
the License. You may obtain a copy of the License at
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
specific language governing permissions and limitations under the License.
|
|
-->
|
|
|
|
# Command line interface
|
|
|
|
`diffusers-cli` is a command line client for running, inspecting, and packaging Diffusers pipelines.
|
|
|
|
## Available commands
|
|
|
|
| Command | Purpose |
|
|
|---|---|
|
|
| [`env`](#env) | Print environment info for bug reports. |
|
|
| [`schema`](#schema) | Inspect a pipeline's `__call__` signature without downloading weights. |
|
|
| [`run`](#run) | Run a pipeline locally or in a Hugging Face Sandbox. |
|
|
| [`serve`](#serve) | Serve a pipeline over HTTP with the Generative Media Spec API. |
|
|
| [`generate`](#generate) | Send a generation to a Generative Media Spec server and save its outputs. |
|
|
| [`custom_blocks`](#customblocks) | Package a local `ModularPipelineBlocks` subclass for the Hub. |
|
|
| [`fp16_safetensors`](#fp16safetensors) | Convert a checkpoint to fp16 `.safetensors`. |
|
|
| [`skills`](#skills) | Install pre-authored skill bundles into your AI coding agent. |
|
|
|
|
> [!TIP]
|
|
> This page does not provide details for all options under each subcommand. For the full, always current list of options for any subcommand, run `diffusers-cli <command> --help` (`diffusers-cli run --help`).
|
|
|
|
## env
|
|
|
|
Prints Python, PyTorch, Diffusers versions, CUDA info, and installed optional deps. Use it when opening an
|
|
issue so maintainers can reproduce your setup.
|
|
|
|
```bash
|
|
diffusers-cli env
|
|
```
|
|
|
|
## schema
|
|
|
|
Returns the pipeline's accepted inputs without downloading weights. Use it when building `--pipeline-kwargs` for [`run`](#run).
|
|
|
|
Only the index file is fetched. Standard pipelines read `model_index.json`. Modular pipelines read `modular_model_index.json`. Custom-block repos read `modular_config.json` and need `--trust-remote-code`, because loading them runs Hub code.
|
|
|
|
```bash
|
|
diffusers-cli --format json schema --model black-forest-labs/FLUX.1-dev
|
|
diffusers-cli schema --model my-org/my-custom-blocks --trust-remote-code
|
|
```
|
|
|
|
## run
|
|
|
|
Run a pipeline end-to-end. The CLI auto-detects standard vs modular repos. It auto-loads media inputs from URLs or local paths, saves outputs from the pipeline's return type, and can run remotely on a Hugging Face Sandbox via `--remote`.
|
|
|
|
Minimal example:
|
|
|
|
```bash
|
|
diffusers-cli run \
|
|
--model black-forest-labs/FLUX.1-dev \
|
|
--dtype bf16 \
|
|
--pipeline-kwargs '{"prompt": "an astronaut riding a horse"}'
|
|
```
|
|
|
|
### Passing pipeline arguments
|
|
|
|
`--pipeline-kwargs` takes a JSON object that's forwarded to `pipeline(**kwargs)`. String values at known
|
|
media-input keys are auto-loaded:
|
|
|
|
- Images (`image`, `last_image`, `mask_image`, `control_image`, `ip_adapter_image`, `image_2`) → `PIL.Image` via
|
|
`load_image`.
|
|
- Videos (`video`, `control_video`) → `list[PIL.Image]` via `load_video`.
|
|
- Audio (`initial_audio_waveforms`, `reference_audio`, `src_audio`) → `torch.Tensor` via `torchaudio.load`.
|
|
|
|
```bash
|
|
diffusers-cli run \
|
|
--model black-forest-labs/FLUX.2-klein-9B --dtype bf16 \
|
|
--pipeline-kwargs '{"prompt": "make the fur grey", "image": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png", "strength": 0.6}'
|
|
```
|
|
|
|
Both media keys and text keys accept a JSON array to run a batch through a single pipeline call. Each entry
|
|
in a media list is loaded individually (URL, local path, or bucket-mount path), and Diffusers processes the
|
|
whole list in one forward pass on the GPU:
|
|
|
|
```bash
|
|
diffusers-cli run \
|
|
--model black-forest-labs/FLUX.1-Kontext-dev --dtype bf16 \
|
|
--pipeline-kwargs '{
|
|
"prompt": ["make it grey", "make it pink", "make it blue"],
|
|
"image": [
|
|
"https://.../cat1.png",
|
|
"https://.../cat2.png",
|
|
"https://.../cat3.png"
|
|
]
|
|
}'
|
|
```
|
|
|
|
### Loading
|
|
|
|
Configure how the CLI loads model weights and custom pipeline code.
|
|
|
|
- `--dtype {auto, bfloat16, bf16, float16, fp16, float32, fp32}` — weight dtype.
|
|
- `--device-map <value>` — component placement. Accepts a torch device string (`cuda`, `cuda:0`, `cpu`, `mps`),
|
|
`balanced` (auto-splits components across visible GPUs), or a JSON dict for explicit per-component placement.
|
|
Auto-detected if omitted. See [device_map](../training/distributed_inference#devicemap) for more details.
|
|
- `--variant fp16` — pick a weight variant.
|
|
- `--revision <sha>` — pin a specific model revision.
|
|
- `--trust-remote-code` — allow custom code from the Hub (required for repos that ship custom pipeline classes
|
|
or modular blocks). See [Community pipelines](../using-diffusers/custom_pipeline_overview) for standard custom
|
|
pipelines and [Modular Diffusers](../modular_diffusers/overview).
|
|
- `--lora <spec>` — attach a LoRA adapter after loading. Each value is a JSON dict. Repeat the flag to
|
|
stack multiple adapters. `lora_id` is required per entry, `lora_scale` defaults to `1.0`, and
|
|
`adapter_name` is optional (auto-generated as `lora_<i>` when stacking).
|
|
- Single: `--lora '{"lora_id": "alvdansen/flux-koda", "lora_scale": 0.8}'`
|
|
- Multiple: `--lora '{"lora_id": "alvdansen/flux-koda", "lora_scale": 0.6, "adapter_name": "koda"}' --lora '{"lora_id": "Shakker-Labs/FLUX.1-dev-LoRA-AntiBlur", "lora_scale": 0.4}'`
|
|
|
|
All specs are loaded by `pipeline.load_lora_weights(...)`, then activated together with a single
|
|
`pipeline.set_adapters(names, adapter_weights=scales)` call. See [LoRA](../tutorials/using_peft_for_inference)
|
|
for a deeper walkthrough of adapter stacking, scale scheduling, and hotswapping.
|
|
|
|
### Optimizations
|
|
|
|
- `--cpu-offload {model, group, auto}` — onload target comes from `--device-map` (plain device string required for offload).
|
|
- `model` — `enable_model_cpu_offload` for standard pipelines. See [Model offloading](../optimization/memory#model-offloading).
|
|
- `group` — `enable_group_offload(offload_type="leaf_level", use_stream=True)` for standard pipelines. See [Group offloading](../optimization/memory#group-offloading).
|
|
- `auto` — modular only with [`ComponentsManager`](../modular_diffusers/components_manager). Standard modes raise on modular pipelines. `auto` raises on standard pipelines.
|
|
- `--offload-margin <size>` — device memory kept free for activations under `--cpu-offload auto`, passed to
|
|
`enable_auto_cpu_offload` as `memory_reserve_margin` (default `3GB`). Raise it when a large canvas runs out
|
|
of memory mid-forward. The offloader keeps components resident while they fit, so on a high-VRAM card the
|
|
default margin can leave too little room for the activations of a long video.
|
|
- `--attention-backend` — Hub-hosted attention kernels, auto-downloaded on first use. Choices are `default` plus the Hub backends registered in Diffusers (for example `flash_hub`, `flash_varlen_hub`, `flash_4_hub`, `sage_hub`, `sage_blackwell_hub`). Run `diffusers-cli run --help` for the current list. It only supports Transformer-based pipelines only, and is ignored with a warning on legacy UNet pipelines. See [Attention backends](../optimization/attention_backends).
|
|
- `--vae-tiling` / `--vae-slicing` — lower VAE decode VRAM. See
|
|
[VAE tiling](../optimization/memory#vae-tiling) and [VAE slicing](../optimization/memory#vae-slicing).
|
|
- `--compile [JSON]` — compile denoiser modules and the VAE decoder with
|
|
[torch.compile](../optimization/fp16#torchcompile). The CLI prefers
|
|
[regional compilation](../optimization/fp16#regional-compilation) for denoisers with repeated blocks; the VAE
|
|
decoder is always compiled whole. Bare `--compile` uses `fullgraph=true`. A JSON object is forwarded to
|
|
`torch.compile`. Not supported with `--context-parallel`.
|
|
- `--compile-mode {regional,full}` — what `--compile` compiles. `regional` (default) compiles only the repeated
|
|
blocks, which keeps the first step fast. `full` compiles the whole denoiser. `torch.compile` modes that use CUDA
|
|
graphs (`max-autotune`, `reduce-overhead`) need `--compile-mode full`: repeated blocks share one compiled graph,
|
|
so with CUDA graphs each block's output is overwritten by the next block's run.
|
|
- `--context-parallel` — Ulysses-style context parallelism on a DiT-based pipeline. Locally requires torchrun, but
|
|
under `--remote` the CLI wraps `torchrun --nproc-per-node=gpu` for you. See
|
|
[Context parallelism](../training/distributed_inference#context-parallelism).
|
|
|
|
### Modular pipelines
|
|
|
|
`run` detects a [modular repo](../modular_diffusers/overview) automatically — either because it ships a
|
|
`modular_model_index.json`, or because its `model_index.json` names a `ModularPipeline` subclass — so no flag
|
|
is needed to opt in.
|
|
|
|
Some modular repos define several *workflows* like named tasks that share components but differ in which blocks
|
|
run and which inputs they take. [MiniMax-H3](../api/pipelines/minimax_h3), for example, offers `t2va` (text to
|
|
video and audio), `fl2va` (first and/or last keyframe) and `ref2va` (an ordered mix of image, video and audio
|
|
references). Pass `--workflow` to select one:
|
|
|
|
```bash
|
|
diffusers-cli run \
|
|
--model MiniMaxAI/MiniMax-H3 --workflow fl2va \
|
|
--pipeline-kwargs '{
|
|
"prompt": "the camera pushes in slowly as rain falls",
|
|
"image": "opening-frame.png",
|
|
"last_image": "closing-frame.png",
|
|
"num_frames": 124
|
|
}' \
|
|
--output-key videos --output-key audio \
|
|
--fps 24 --sampling-rate 32000 \
|
|
--cpu-offload auto --dtype bf16
|
|
```
|
|
|
|
Selecting a workflow keeps only that task's blocks, so the pipeline declares only the components it needs and
|
|
`load_components` fetches only their subfolders. Omit `--workflow` to keep every workflow available and let the
|
|
pipeline pick per call from the inputs it is given.
|
|
|
|
```bash
|
|
diffusers-cli --format json schema --model MiniMaxAI/MiniMax-H3 --trust-remote-code
|
|
```
|
|
|
|
`--workflow` applies to modular pipelines only. It is ignored with a warning on standard pipelines.
|
|
|
|
A modular pipeline returns a [`PipelineState`](../modular_diffusers/modular_pipeline) rather than a single output object, so `--output-key` names the intermediate to save.
|
|
|
|
|
|
### Outputs
|
|
|
|
`run` detects the pipeline output type:
|
|
|
|
- `PIL.Image`/list → `<NNNN>.png` (zero-padded index, e.g. `0000.png`)
|
|
- Image sequence → `0000.mp4` (`--fps` controls framerate, default 8)
|
|
- Audio array → `0000.wav` (`--sampling-rate` controls rate)
|
|
- Anything else → JSON dump
|
|
|
|
The default output directory format is `~/.diffusers/cli/run/outputs/diffusers-run-<YYYYMMDDTHHMMSS>-<uuid>/`. Each
|
|
run gets its own subdirectory so consecutive invocations don't overwrite.
|
|
|
|
Override with `--output <path>`. How the path expands depends on its shape and the batch size:
|
|
|
|
| `--output` | 1 output | N outputs |
|
|
|---|---|---|
|
|
| _omitted_ | default dir → `0000.png` | default dir → `0000.png`, `0001.png`, `0002.png`, … |
|
|
| `./results/` (trailing `/` or an existing directory) | `./results/0000.png` | `./results/0000.png`, `./results/0001.png`, … |
|
|
| `my-cat.png` (file path) | `my-cat.png` (used verbatim) | `my-cat-0000.png`, `my-cat-0001.png`, … |
|
|
|
|
Directory outputs always use bare padded names (`0000`, `0001`, …). Explicit file paths preserve your chosen
|
|
stem and get the padded index appended when the batch produces multiple outputs.
|
|
|
|
Use `--push-to` to upload outputs to a
|
|
[Hugging Face storage bucket](https://huggingface.co/docs/hub/en/storage-buckets). It accepts an:
|
|
|
|
- HF bucket id (`<namespace>/<name>`)
|
|
- `hf://buckets/<namespace>/<name>[/<subpath>]`
|
|
[HF URI](https://huggingface.co/docs/huggingface_hub/main/en/package_reference/hf_uris)
|
|
- browser URL
|
|
|
|
A subpath is used as a folder prefix. The bucket is created if missing, and objects land under
|
|
`[<subpath>/]<run_id>/<filename>`.
|
|
|
|
```bash
|
|
# HF bucket id — files at hf://buckets/alice/edit-outputs/<run_id>/<file>
|
|
--push-to alice/edit-outputs
|
|
|
|
# URI with subpath — files at hf://buckets/alice/edit-outputs/greyscale/2026-07/<run_id>/<file>
|
|
--push-to hf://buckets/alice/edit-outputs/greyscale/2026-07
|
|
|
|
# Browser URL copy-paste from the Hub also works.
|
|
--push-to https://huggingface.co/buckets/alice/edit-outputs/tree/greyscale/2026-07
|
|
```
|
|
|
|
The table below describes remote runs. For local runs, `--push-to` uploads the locally saved output. It does
|
|
not suppress local file creation.
|
|
|
|
| `--push-to` set? | `--output` set? | Result |
|
|
|---|---|---|
|
|
| no | no | download to default local dir |
|
|
| no | yes | download to `--output` |
|
|
| yes | no | bucket only, no local download |
|
|
| yes | yes | bucket AND `--output` |
|
|
|
|
`--format` shapes the stdout metadata (paths, timing, sandbox info). It does not change the file format of
|
|
the media itself. Written images are always PNG, videos MP4, audio WAV.
|
|
|
|
### Remote execution (`--remote`)
|
|
|
|
Run the same call inside a [Hugging Face Sandbox](https://huggingface.co/docs/huggingface_hub/en/guides/sandbox)
|
|
— an isolated cloud VM the CLI drives over HTTP: it uploads inputs, runs the pipeline, downloads outputs, then
|
|
terminates the sandbox. Requires `huggingface_hub>=1.23`.
|
|
|
|
The sandbox boots from [`diffusers/diffusers-cli-cuda`](https://hub.docker.com/r/diffusers/diffusers-cli-cuda), a
|
|
prebuilt image that already ships torch, CUDA, and the CLI's dependencies, so a cold run starts generating instead
|
|
of installing first.
|
|
|
|
```bash
|
|
diffusers-cli run \
|
|
--model black-forest-labs/FLUX.1-dev --dtype bf16 \
|
|
--pipeline-kwargs '{"prompt": "an astronaut riding a horse"}' \
|
|
--remote --flavor a100-large
|
|
```
|
|
|
|
Remote flags:
|
|
|
|
- `--flavor <name>` — sandbox hardware (for example, `a10g-small`, `h200`, `rtx-pro-6000`).
|
|
- `--timeout <duration>` — max wallclock for the run command inside the sandbox (default `10m`).
|
|
- `--dependencies <pkg>` — extra pip deps (repeatable), installed on top of the image. Useful for pinning a
|
|
diffusers branch tarball or adding pipeline-specific extras.
|
|
- `--namespace <name>` — create the sandbox under a different HF org/account.
|
|
- `--image <ref>` — override the sandbox image. Must ship torch + CUDA compatible with your `--flavor`'s
|
|
driver. The CLI then installs its own dependencies on top on every cold sandbox, which the default image
|
|
avoids.
|
|
- `--volume <bucket-id>[:<mount-path>]` — mount an [HF storage bucket](https://huggingface.co/docs/hub/en/storage-buckets)
|
|
into the sandbox as a read-write directory (repeatable). Default mount path is
|
|
`/mnt/buckets/<bucket-id>`. Reference mounted files from `--pipeline-kwargs` like any other local path.
|
|
Applied only on new sandbox creation and ignored when reconnecting via `--sandbox-id`.
|
|
|
|
By default each `--remote` run is ephemeral (create → run → download → kill). To reuse a warm sandbox across
|
|
runs — keeping deps, the model weight cache, and the `torch.compile` cache on its disk — keep it alive and
|
|
reconnect:
|
|
|
|
- `--keep-alive` — don't terminate the sandbox after the run. Its id is printed.
|
|
- `--sandbox-id <id>` — reconnect to a kept-alive sandbox instead of creating a new one.
|
|
- `--idle-timeout <duration>` — auto-shutdown after this much inactivity (default `10m`). Applied only on new sandbox creation and ignored when reconnecting via `--sandbox-id`.
|
|
|
|
```bash
|
|
# First run keeps the sandbox alive and prints sandbox_id=<id>.
|
|
diffusers-cli run -m black-forest-labs/FLUX.1-dev --dtype bf16 \
|
|
--pipeline-kwargs '{"prompt": "a cat"}' --remote --flavor a100-large --keep-alive
|
|
|
|
# Reconnect for the next run — the model is already cached, so only inference runs.
|
|
diffusers-cli run -m black-forest-labs/FLUX.1-dev --dtype bf16 \
|
|
--pipeline-kwargs '{"prompt": "a dog"}' --remote --flavor a100-large --sandbox-id <id>
|
|
|
|
# Stop it when done (or let it timeout).
|
|
hf sandbox kill <id>
|
|
```
|
|
|
|
## `serve`
|
|
|
|
Serve a pipeline over HTTP. The server implements the Generative Media Spec (GMS, draft `gms/0.1`): clients
|
|
discover what the model accepts, create a generation, follow it while it runs, and read the results. It needs
|
|
the `serve` extra:
|
|
|
|
```bash
|
|
pip install "diffusers[serve]"
|
|
diffusers-cli serve --model black-forest-labs/FLUX.1-dev --dtype bf16
|
|
```
|
|
|
|
`serve` takes the same loading and optimization flags as [`run`](#run) (`--dtype`, `--device-map`,
|
|
`--cpu-offload`, `--compile`, `--lora`, ...), plus `--host` (default `127.0.0.1`) and `--port` (default `8000`).
|
|
The server has no authentication, so keep it on a trusted network.
|
|
|
|
[Modular repos](../modular_diffusers/overview) are detected the same way as in `run` and served in-process. They
|
|
report no step progress, and a cancel request only takes effect once the pipeline call returns.
|
|
|
|
| Flag | Effect |
|
|
|---|---|
|
|
| `--default-seed` | Seed used for requests that do not set one. Without it, each such request gets a random seed. |
|
|
| `--enable-cors` | Allow cross-origin requests from any origin, so a browser app on another host can call the server. |
|
|
| `--public-url` | Base URL clients reach the server at, used to build output file links. Set it behind a reverse proxy. |
|
|
| `--log-level` | Logging level for diffusers and the HTTP server (`debug`, `info`, `warning`, `error`, `critical`). |
|
|
|
|
`GET /health` returns `{"status": "ok"}`, or `503` when an SGLang or vLLM-Omni backend has exited. Every response
|
|
carries an `x-request-id` header: the one sent with the request, or a generated one.
|
|
|
|
Generations run one at a time. Creating one returns immediately with a `queued` generation; poll it, subscribe
|
|
to its events, or ask the server to hold the connection with `Prefer: wait=<seconds>`:
|
|
|
|
```bash
|
|
# What the model accepts: tasks, inputs, parameters with types and defaults.
|
|
curl localhost:8000/v1/models/black-forest-labs/FLUX.1-dev
|
|
|
|
# Create a generation and wait up to 120 seconds for it.
|
|
curl localhost:8000/v1/generations -H 'Prefer: wait=120' -d '{
|
|
"model": "black-forest-labs/FLUX.1-dev",
|
|
"inputs": {"prompt": "an astronaut riding a horse"},
|
|
"parameters": {"width": 1024, "height": 1024, "steps": 28, "seed": 42}
|
|
}'
|
|
|
|
# Follow step progress as server-sent events, or poll the generation.
|
|
curl -N localhost:8000/v1/generations/<id>/events
|
|
curl localhost:8000/v1/generations/<id>
|
|
|
|
# Cancel a queued or running generation, or delete it together with its files.
|
|
curl -X POST localhost:8000/v1/generations/<id>/cancel
|
|
curl -X DELETE localhost:8000/v1/generations/<id>
|
|
```
|
|
|
|
A completed generation lists its outputs with a download URL each (or inline base64 with
|
|
`"response_format": "b64_json"`), the seed that was used, and the model revision. Outputs are kept under
|
|
`--output-dir` and expire after one hour. Images are PNG by default (`"output_format": "image/jpeg"` or
|
|
`"image/webp"` to change it), videos MP4, audio WAV. Media inputs are passed as
|
|
`{"type": "image", "url": "https://..."}` or `{"type": "image", "base64": "...", "media_type": "image/png"}`.
|
|
|
|
Validation is strict: an unknown parameter, a value outside its declared range, or a missing input is rejected
|
|
with a JSON pointer to the offending field.
|
|
|
|
```json
|
|
{"error": {"code": "invalid_request", "message": "task 'text_to_image' declares no parameter 'cfg'", "pointer": "/parameters/cfg"}}
|
|
```
|
|
|
|
### Remote serving (`--remote`)
|
|
|
|
`--remote` starts the server in a [Hugging Face Sandbox](https://huggingface.co/docs/huggingface_hub/guides/sandbox)
|
|
instead of on the local machine. The command creates the sandbox, installs the dependencies, starts the server,
|
|
and streams its logs until you interrupt it; the sandbox is then deleted.
|
|
|
|
```bash
|
|
diffusers-cli serve --model black-forest-labs/FLUX.1-dev --dtype bf16 --remote --flavor a10g-large
|
|
```
|
|
|
|
Once the model has loaded, the command prints the sandbox id. The server is only reachable through the sandbox
|
|
proxy, which requires your Hugging Face token, so pass the id to [`generate`](#generate) rather than calling a URL
|
|
directly:
|
|
|
|
```bash
|
|
diffusers-cli generate --sandbox-id <id> --inputs '{"prompt": "a cat on the moon"}'
|
|
```
|
|
|
|
`--flavor`, `--image`, `--dependencies`, `--namespace` and `--idle-timeout` work as they do for
|
|
[`run --remote`](#remote-execution---remote). `--model` must be a Hub repo id. A local `--manifest` file is
|
|
uploaded to the sandbox.
|
|
|
|
With `--backend sglang` or `--backend vllm`, the sandbox starts from the engine's own image
|
|
(`lmsysorg/sglang:latest` or `vllm/vllm-omni:latest`) unless `--image` names another, so the engine does not need
|
|
installing:
|
|
|
|
```bash
|
|
diffusers-cli serve --model black-forest-labs/FLUX.2-klein-4B --backend sglang --remote --flavor a100-large
|
|
```
|
|
|
|
### Manifests
|
|
|
|
What a model accepts is described by its manifest. `serve` looks for one in this order:
|
|
|
|
1. the file passed with `--manifest <path>`;
|
|
2. a `gms.json` published in the model repo, served byte-for-byte;
|
|
3. a manifest derived from the pipeline's `__call__` signature.
|
|
|
|
A derived manifest is a best-effort reading of the signature: scalar arguments become parameters, prompts and
|
|
media arguments become inputs, and arguments with a core GMS name are renamed (`num_inference_steps` becomes
|
|
`steps`, `num_images_per_prompt` becomes `num_outputs`, `generator` becomes `seed`). It carries no value ranges.
|
|
Write a `gms.json` when you need ranges, presets, or a duration in seconds.
|
|
|
|
When a manifest uses a name that differs from the pipeline argument, map it per task under `x-diffusers`:
|
|
|
|
```json
|
|
"x-diffusers": {"inputs": {"caption": "prompt"}, "parameters": {"cfg": "guidance_scale"}}
|
|
```
|
|
|
|
A task is only hosted when every input it requires maps to a pipeline argument. A parameter the pipeline cannot
|
|
set is rejected with `not_implemented` unless the request sends its default value.
|
|
|
|
### Adapters
|
|
|
|
A request can apply LoRAs for that generation only. They are loaded before the call and unloaded after it:
|
|
|
|
```bash
|
|
curl localhost:8000/v1/generations -d '{
|
|
"model": "black-forest-labs/FLUX.1-dev",
|
|
"inputs": {"prompt": "a tiny cat"},
|
|
"adapters": [{"type": "lora", "path": "hf:alvdansen/littletinies", "scale": 0.8}]
|
|
}'
|
|
```
|
|
|
|
Request adapters are turned off when the server starts with `--compile` or `--lora`; such requests fail with
|
|
`not_implemented`.
|
|
|
|
### SGLang and vLLM-Omni backends
|
|
|
|
`--backend sglang` and `--backend vllm` run the model in [SGLang Diffusion](https://docs.sglang.io/) or
|
|
[vLLM-Omni](https://docs.vllm.ai/projects/vllm-omni) instead of in-process. `serve` starts `sglang serve` or
|
|
`vllm serve --omni` as a child process on a local port, waits for it to become healthy, and translates GMS
|
|
requests to the engine's OpenAI-compatible image and video routes. Clients only ever talk to the GMS endpoints
|
|
of `serve`. The engine must be installed separately.
|
|
|
|
```bash
|
|
diffusers-cli serve --model Qwen/Qwen-Image --backend sglang --backend-args '--num-gpus 2'
|
|
diffusers-cli serve --model Wan-AI/Wan2.2-T2V-A14B-Diffusers --backend vllm
|
|
```
|
|
|
|
The pipeline flags (`--dtype`, `--cpu-offload`, `--compile`, ...) do not apply to these backends: they are
|
|
ignored with a warning. Pass the engine's own flags with `--backend-args`. Compared with the in-process backend, they serve image and video
|
|
outputs only, accept `image` as the only media input, expose the parameters both engines share (size, steps,
|
|
guidance scale, seed, number of frames, fps, number of images), report no step progress, and apply no request
|
|
adapters.
|
|
|
|
## `generate`
|
|
|
|
Send a generation to a server started with [`serve`](#serve), or to any other Generative Media Spec server, and
|
|
save its outputs. Unlike [`run`](#run), `generate` loads no pipeline: it reads the model's discovery document,
|
|
submits the request, polls the generation until it finishes, and downloads the files.
|
|
|
|
```bash
|
|
diffusers-cli generate --url http://localhost:8000 --inputs '{"prompt": "a cat on the moon"}'
|
|
|
|
diffusers-cli generate --url http://localhost:8000 \
|
|
--inputs '{"prompt": "make the fur grey", "image": "cat.png"}' \
|
|
--parameters '{"steps": 8, "seed": 0}' --output outputs/
|
|
```
|
|
|
|
`--inputs` and `--parameters` use the names the server declares, not pipeline argument names (`steps`, not
|
|
`num_inference_steps`). An image, mask, video or audio input can be an http(s) URL or a local file path; local
|
|
files are sent inline as base64. `--task`, `--preset`, `--adapters` and `--output-format` map to the request
|
|
fields of the same name. `--model` is only needed when the server hosts more than one model.
|
|
|
|
Outputs are saved to `~/.diffusers/cli/generate/outputs/<generation id>/` unless `--output` names a directory.
|
|
`--token` sends a bearer token in the `Authorization` header, for servers behind an authenticating proxy; no
|
|
token is sent by default. Interrupting the command cancels the generation on the server.
|
|
|
|
For a server started with [`serve --remote`](#remote-serving---remote), pass `--sandbox-id <id>` instead of
|
|
`--url`. The request then goes through the sandbox proxy, authenticated with your Hugging Face token.
|
|
|
|
## `custom_blocks`
|
|
|
|
Package a local `ModularPipelineBlocks` subclass for upload to the Hub. Reads a Python file, AST-scans it for
|
|
classes deriving from `ModularPipelineBlocks` or one of its composite subclasses (`SequentialPipelineBlocks`,
|
|
`AutoPipelineBlocks`, `ConditionalPipelineBlocks`, `LoopSequentialPipelineBlocks`), instantiates the chosen one, and
|
|
calls `save_pretrained` in the current working directory.
|
|
|
|
```bash
|
|
# Package the first block found in ./block.py
|
|
diffusers-cli custom_blocks
|
|
|
|
# Point at a different file / pick a specific class
|
|
diffusers-cli custom_blocks --block_module_name my_block.py --block_class_name MyDenoiseBlock
|
|
```
|
|
|
|
The block class must be instantiable with zero constructor args and hardcodes defaults in `__init__` or read
|
|
config from the pipeline `state` at call time.
|
|
|
|
## `fp16_safetensors`
|
|
|
|
> [!CAUTION]
|
|
> This command is now deprecated and will be removed in a future version.
|
|
|
|
Convert a checkpoint on the Hub to fp16 `.safetensors` and push the result. Useful for shrinking a repo's
|
|
weight size for faster loading. See `diffusers-cli fp16_safetensors --help` for the exact args.
|
|
|
|
## `skills`
|
|
|
|
Install skills from the diffusers repo ([`.ai/skills/`](https://github.com/huggingface/diffusers/tree/main/.ai/skills)).
|
|
|
|
```bash
|
|
# Install a single skill
|
|
diffusers-cli skills add "<skill name>"
|
|
|
|
# Install every skill in the registry
|
|
diffusers-cli skills add --all
|
|
|
|
# List available skills with a one-line summary of each
|
|
diffusers-cli skills list
|
|
|
|
# Preview a skill's SKILL.md without installing
|
|
diffusers-cli skills preview diffusers-cli
|
|
|
|
# Refetch and reinstall every managed skill
|
|
diffusers-cli skills update
|
|
|
|
# Install to the user-level directory instead of the current project
|
|
diffusers-cli skills add diffusers-cli --global
|
|
|
|
# Install for one agent instead of detecting it from the environment
|
|
diffusers-cli skills add --all --claude # or --codex / --cursor
|
|
```
|
|
|
|
Without a target flag, the CLI installs for the agent that launched it, or for every agent when it cannot tell. Claude Code gets a plugin bundle under `.claude/skills/diffusers/` (namespaced as `/diffusers:<skill name>`). Codex and Cursor get `.agents/skills/<skill name>/`.
|