- Shard the transformer with `TensorParallelConfig` on CUDA, TPU and Neuron, with
remote-code transformers loaded via `--trust-remote-code`.
- Require `--seed`; reject `--device-map`, `--cpu-offload`, `--lora`,
`--context-parallel` and modular pipelines.
- Wrap `--remote` runs in torchrun.
- Load standard pipelines without an indexed `device_map` (e.g. `cuda:<LOCAL_RANK>`)
and place components after loading, fixing `--context-parallel` on ranks > 0.
fix(image_processor): preserve 3-tuple return contract when mask is None (#14470)
Ensure InpaintProcessor.preprocess returns (image, None, postprocessing_kwargs) when mask is None, preserving the expected 3-tuple return contract and signature consistency across all preprocess branches. Consolidate unit tests into a parameterized test covering with-mask, without-mask, and crop configurations.
* restructure guides as skill references
* reorganize AGENTS.md and references
* clean up
* clean up
* clean up
* fix AGENTS.md references and add check_ai to Makefile
* clean up
* update paths
* clean up
* [CLI] Clean up and improvements to `diffusers-cli`
Modular pipelines:
- `custom_blocks` also writes `modular_model_index.json`, so a packaged repo is
loadable by `ModularPipeline.from_pretrained` (and runnable by `run`), not just
importable as blocks.
- Add `--cpu-offload auto`, which offloads through the pipeline's
`ComponentsManager`, plus `--offload-margin` to set its `memory_reserve_margin`.
- Add `--workflow` passthrough; warn and ignore it on standard pipelines.
- Route `model_index.json` repos whose `_class_name` is modular to the modular path.
Output saving:
- Resolve every media field on the output object (`images`, `frames`, `audios`,
`audio`) instead of the first match, keeping the batch dimension. Fixes video
output being misrouted through the image branch and LTX2's audio being dropped.
- Fix the `output_type="pt"` video transpose: `postprocess_video` returns
`(B, F, C, H, W)`, not `(B, C, F, H, W)`.
- Save `(B, C, samples)` waveforms as multi-channel wavs (StableAudio).
- Write each video's frames to `<stem>-frames/<NNNN>.png`. Frames need no video
backend, so they replace the `.pt` fallback when `export_to_video` fails.
- Report frame directories as a single path so hundreds of frames don't flood the
terminal; `--push-to` and the sandbox download preserve subfolder structure.
- Raise on unrecognized payloads instead of dumping JSON/pickle.
Remote:
- Add `imageio`, `imageio-ffmpeg`, and `peft` to the default sandbox deps.
- Raise sandbox `start_timeout` to 600s; the 120s default expires while a cold node
pulls the pytorch base image.
* Apply suggestion from @sayakpaul
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* update
* update
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* download kernels when users request for it.
* migrate tests/others to pytest
* Revert "download kernels when users request for it."
This reverts commit 6fe34ade6b.
---------
Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>
* Remove JAX/Flax support
Deletes all Flax models, schedulers, pipelines, and loading utilities,
the from_flax loading path, flax/jax packaging extras, Flax CI filters,
Flax training examples, and Flax content in translated docs.
Hub compatibility is preserved: Flax class names in old model_index.json
files are still stripped to their PyTorch equivalents, and *.msgpack
files in repos are still ignored during download.
* Drop Flax class-name remapping from model_index.json loading
Flax-saved checkpoint metadata is no longer recognized: class names like
FlaxStableDiffusionPipeline in model_index.json now fail to resolve
instead of being remapped to their PyTorch equivalents. *.msgpack files
in hub repos are still ignored during download.
* Restore Flax training examples
The example scripts are standalone and still work with diffusers<=0.39.x.
A warning note is added to each Flax README section pointing out the
version requirement now that the library itself no longer ships Flax.
* update
* Repoint scheduler default-config tests off the Flax fixture
* Repoint example training tests off the Flax fixture
* fix(randn_tensor): compare device.type, not torch.device to str, when suppressing MPS info log
When a CPU generator is passed with an MPS target, randn_tensor intentionally skips
the 'generator was on cpu, tensor will be moved to <device>' info log — MPS doesn't
support device-side generators, so the suggestion to create one on MPS would be
misleading. The guard was written as `if device != "mps"`, but a few lines
earlier `device` is coerced to a `torch.device` object, and
`torch.device("mps") == "mps"` is False (torch.device's __eq__ with a string
returns NotImplemented, falling back to identity — they're different types).
Result: the guard is effectively always True, so MPS users get the spurious log
whenever they pass a CPU generator — the opposite of the documented intent.
Fix: compare `device.type` (a str) against "mps". Added a regression test in
tests/others/test_utils.py that exercises both the MPS and non-MPS paths via
`assertLogs` on the diffusers logger.
* refactor: use CaptureLogger instead of assertLogs in randn_tensor test
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
* add attention backend tests.
* remove existing tests/others/test_attention_backends.py file
* modify generate_model_tests.py
* remove native.
* account for _keep_in_fp32_modules
* don't skip when exception is raised.
* use is_kernels_available()
* mark with compile.
* move rtol and atol to methods as defaults.
* Apply suggestions from code review
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* up
* up
* fix(freeu): run FFT in float32 for float16 inputs to avoid ComplexHalf
`fourier_filter` already upcasts `bfloat16` inputs to `float32` before
calling `torch.fft.fftn`, because PyTorch's FFT does not support bf16.
The same is true for `float16`: depending on the PyTorch version,
`fftn` either
- produces the experimental `torch.complex32` (ComplexHalf) dtype and
emits a `UserWarning: ComplexHalf support is experimental`, or
- raises `RuntimeError: Unsupported dtype Half` outright.
Both paths were reachable from FreeU with half-precision models
(e.g. `sd-turbo` + `fp16` + `enable_freeu`) as reported in #12504.
Extend the existing upcast branch to cover `float16` too. The function
already downcasts the result back to `x_in.dtype` at the end, so the
externally observable dtype is unchanged.
Closes#12504.
* Address review: generalize upcast to non-float32 + fix ruff F821
- Apply @sayakpaul's suggestion: use `elif x.dtype != torch.float32:`
so any non-float32 dtype (bf16, fp16, and future half-precision
dtypes) is upcast to float32 before the FFT.
- Drop the `"torch.Tensor"` return annotation on the test helper
that triggered ruff F821 in CI (torch is imported inside the
method body, not at module scope).
* drop python 3.8
* remove list, tuple, dict from typing
* fold Unions into |
* up
* fix a bunch and please me.
* up
* up
* up
* up
* up
* up
* enforce 3.10.0.
* up
* up
* up
* up
* up
* up
* up
* up
* Update setup.py
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
* up.
* python 3.10.
* ifx
* up
* up
* up
* up
* final
* up
* fix typing utils.
* up
* up
* up
* up
* up
* up
* fix
* up
* up
* up
* up
* up
* up
* handle modern types.
* up
* up
* fix ip adapter type checking.
* up
* up
* up
* up
* up
* up
* up
* revert docstring changes.
* keep deleted files deleted.
* keep deleted files deleted.
---------
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
* Add check for WindowsPath in to_json_string
On Windows, os.path.join returns a WindowsPath. to_json_string does not convert this from a WindowsPath to a string. Added check for WindowsPath to to_json_saveable.
* Remove extraneous convert to string in test_check_path_types (tests/others/test_config.py)
* Fix style issues in tests/others/test_config.py
* Add unit test to test_config.py to verify that PosixPath and WindowsPath (depending on system) both work when converted to JSON
* Remove distinction between PosixPath and WindowsPath in ConfigMixIn.to_json_string(). Conditional now tests for Path, and uses Path.as_posix() to convert to string.
---------
Co-authored-by: Vincent Dovydaitis <vincedovy@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* Add support for _foreach operations and non-blocking to EMAModel
* default foreach to false
* add non-blocking EMA offloading to SD1.5 T2I example script
* fix whitespace
* move foreach to cli argument
* linting
* Update README.md re: EMA weight training
* correct args.foreach_ema
* add tests for foreach ema
* code quality
* add foreach to from_pretrained
* default foreach false
* fix linting
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: drhead <a@a.a>
* find & replace all FloatTensors to Tensor
* apply formatting
* Update torch.FloatTensor to torch.Tensor in the remaining files
* formatting
* Fix the rest of the places where FloatTensor is used as well as in documentation
* formatting
* Update new file from FloatTensor to Tensor
* feat: explicitly tag to diffusers when using push_to_hub
* remove tags.
* reset repo.
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
* fix: tests
* fix: push_to_hub behaviour for tagging from save_pretrained
* Apply suggestions from code review
Co-authored-by: Lucain <lucainp@gmail.com>
* Apply suggestions from code review
Co-authored-by: Lucain <lucainp@gmail.com>
* import fixes.
* add library name to existing model card.
* add: standalone test for generate_model_card
* fix tests for standalone method
* moved library_name to a better place.
* merge create_model_card and generate_model_card.
* fix test
* address lucain's comments
* fix return identation
* Apply suggestions from code review
Co-authored-by: Lucain <lucainp@gmail.com>
* address further comments.
* Update src/diffusers/pipelines/pipeline_utils.py
Co-authored-by: Lucain <lucainp@gmail.com>
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Lucain <lucainp@gmail.com>
