- Shard the transformer with `TensorParallelConfig` on CUDA, TPU and Neuron, with
remote-code transformers loaded via `--trust-remote-code`.
- Require `--seed`; reject `--device-map`, `--cpu-offload`, `--lora`,
`--context-parallel` and modular pipelines.
- Wrap `--remote` runs in torchrun.
- Load standard pipelines without an indexed `device_map` (e.g. `cuda:<LOCAL_RANK>`)
and place components after loading, fixing `--context-parallel` on ranks > 0.
* feat: add torchtpu
* feat:draft TorchTPU support
* fix: wan overflow issue + compile mode error on sdxl
* doc: enhance with TorchTPU doc
* doc: enhance with TorchTPU doc
* style: remove unused imports flagged by ruff
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs: remove Debug Eager and Fused Eager sections from tpu.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: add TP for TPU
* style: fix import sorting in TPU test scripts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* style: ruff format TPU test scripts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: style
* fix: test for native 4 devices
* fix: propagate TPU device fixes to Flux/Flux2/Wan-family copies; register tpu.md; drop redundant execution_device check
- Propagate the text_encoder.device-based fix (introduced for TPU CPU-offload
support) from FluxPipeline/Flux2KleinPipeline/WanPipeline into their
`# Copied from` copies (flux/*, flux2_klein_inpaint, visualcloze, anyflow,
chronoedit, lucy_edit, skyreels_v2/*). SDXL-family copies of
StableDiffusionXLPipeline.encode_prompt are intentionally left untouched;
they'll be handled in a follow-up PR that fixes device placement for every
pipeline component (not just text encoders).
- Register docs/source/en/optimization/tpu.md in _toctree.yml (was breaking
the docs build: "not present in the table of contents").
- Remove the redundant "prefer non-CPU, non-meta component" loop from
DiffusionPipeline._execution_device: PR #14383 already fixed this in
DiffusionPipeline.device, which _execution_device falls back to. Verified
on TPU hardware that _execution_device still resolves correctly for a
split-placement pipeline after the removal.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* tests: cleanup
* fix: fix compile mode
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* test: remove flux2 e2e test
* doc: apply suggestions
* doc: apply suggestions
* review: remove monkey patch
* review: revert neuron-specific changes in the tests
* review: apply suggestions
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* doc: add tpu to tp doc
* review: remove unnecessary for tpu
* test: assert every _tp_plan parameter is sharded after a TP load
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* removal: delete redundant tp shard helpers
* removal: drop TPU workarounds no longer needed after recent main changes
* doc: keep the FLUX.2-dev text encoder off a single TPU chip in the TP example
* doc: encode the prompt on CPU in the TPU TP example so it runs end to end
* test: tighten TPU TP tolerance and simplify the TPU TP worker
* doc: use enable_model_cpu_offload() in the TPU eager example
* doc: shard both FLUX.2-dev text encoder and transformer in the TPU TP example
* doc: run the text encoders on TPU in the compiled example
* test: drop the redundant TPU sync in the TP worker
* test: trim comments in the TPU TP tests
* test: run the TPU TP test with mp.spawn like the CUDA one
* test: pin the TPU TP test to 4 chips
* Update docs/source/en/optimization/tpu.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
* doc: use DistributedConfig and dtype, and drop the torch_tpu import in the TPU examples
* test: skip the TPU TP test for models without a _tp_plan
* doc: correct the warning of recompilation
* fix: clone the SD3 timestep so torch.compile doesn't recompile every step, thus we can use it for the tpu.md compile mode snippet
* doc: update advice for compile mode when one chip doesn't fit
* test: pass the TPU TP tolerances as test arguments
* review: apply suggestions to the doc
* review: view compilation patch for tpu-only
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: JingyaHuang <huang_jingya@outlok.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* [LoRA] add LoKr adapter support (Z-Image, Flux2/Klein)
Adds loading of LoKr (LyCORIS Kronecker product) adapters:
- `load_lora_adapter` detects `lokr_` keys and injects a peft `LoKrConfig`,
inferred from the tensor shapes via `_create_lokr_config` (decompose
factor, per-module rank/alpha patterns).
- State dict conversions for the formats in the wild: ai-toolkit Z-Image
(dotted diffusers paths under `diffusion_model.`), ai-toolkit BFL Flux2
(fused qkv), LyCORIS underscore format, and bare dotted diffusers paths.
- BFL fused-QKV LoKr cannot be split exactly into separate Q/K/V Kronecker
factors, so `Flux2LoraLoaderMixin.load_lora_weights` fuses the model's
QKV projections and maps the adapter 1:1 (exact).
- Alpha follows the LyCORIS convention: scaling applies only to
rank-decomposed factors and is baked into the weights at conversion.
Fixes#13221
* [LoRA] LoKr review follow-ups: adapter-agnostic error message, refuse fused-QKV load over an unfused adapter
- "Invalid adapter checkpoint. We currently support LoRA and LoKr." replaces the
message that still said "LoRA checkpoint" and described the substring check.
- Loading a fused-QKV LoKr checkpoint now refuses when an adapter is already
injected on to_q/to_k/to_v or add_{q,k,v}_proj: fuse_qkv_projections() would
replace those modules and orphan it. Covered by a test.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* [LoRA] rename the LoKr tests to test_lokr_conversion.py
The file exercises checkpoint conversion and loading for LoKr adapters, not LoRA.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* [LoRA] LoKr review follow-ups: scope to Z-Image/Flux2, model-level fused QKV, LoKrTesterMixin
- Only the Z-Image and Flux2 loader mixins accept LoKr checkpoints, the two that
convert them; the other mixins are back to their LoRA-only check.
- The fused-QKV handling moves to `_maybe_fuse_qkv_projections_for_lokr` in
peft_utils, called from `load_lora_adapter`, so the model fuses its projections
whichever entry point loads the adapter. The quantized-model error explains why.
- `_bake_lokr_alpha` becomes `_bake_lokr_alpha_` as it edits the dict in place.
- The BFL Flux2 converter no longer accepts expanded diffusers block names under
`diffusion_model.`: no LoKr checkpoint uses that layout; diffusers-named
checkpoints have no prefix and take the generic converter.
- The LyCORIS converter names the keys it does not recognize.
- Tests move from tests/lora to a `LoKrTesterMixin` in tests/models/testing_utils,
with the checkpoint formats tested on the Flux2 and Z-Image model test classes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* [LoRA] mark ZImageLoraLoaderMixin.load_lora_weights as copied from Flux2
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
fix(image_processor): preserve 3-tuple return contract when mask is None (#14470)
Ensure InpaintProcessor.preprocess returns (image, None, postprocessing_kwargs) when mask is None, preserving the expected 3-tuple return contract and signature consistency across all preprocess branches. Consolidate unit tests into a parameterized test covering with-mask, without-mask, and crop configurations.
* Bump doc-builder pin to 9fc8a41 in build_documentation.yml
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Bump doc-builder pin to 9fc8a41 in build_pr_documentation.yml
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Bump doc-builder pin to 9fc8a41 in upload_pr_documentation.yml
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* implement Kandinsky 6
- add TI2VA and SR pipelines
- additonally implement PiflowScheduler and MMAudioVAE
* update docs for Kandinsky 6
* remove redundant bigvgan code and manual checkpoint loading inside model
code
* add missing _no_split_modules
* use attention processor in SR
* fix unguarded dependencies
- removed: einops, pydantic
- lazy: torchvision, av, librosa
* remove dead manual weight loading code; flattent MMAudioVAE structure
* move attention processors to transformer files
* remove duplicate code in K6 latent_upscaler
* address 1st batch of comments
* patch transformer k6
* patch SR
* patch pipeline style
* replace TimestepEmbedding and fix batching
* refactor ti2va dit and pipeline
* refactor piflow scheduler
* refactor SR
* patch sr transformer autocast
* major refactor
* untrack notebooks directory
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* patch mmaudio VAE
* patch mmaudio VAE
* fix inits
* fix flex attn
* refactoring
* remove __all__
* separate MMAudio VAE and Vocoder
* fix naming
* refactor checkpointing, magcache, transformer, sr vae for K6
* refactor
* address apply_scale_shift_norm problem
* update docs
* apply style chanes
* address review feedback on transformer, SR VAE, and flex attention
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* address comments
* fix critical bugs
* Revert checkpoint-breaking module renames from 28c5326da
Commit 28c5326da (today, 14:03 +0300) renamed Kandinsky6FusedTransformerDecoderBlock's
videoT/audioT submodules to video_dec_block/audio_dec_block, and replaced both
Kandinsky6TimeEmbeddings and Kandinsky6SRTimeEmbeddings's hand-rolled in_layer/out_layer
with diffusers' Timesteps/TimestepEmbedding, purely for naming clarity. No checkpoint
conversion was re-run to match, so every published K6 checkpoint (unchanged since Sep 27,
confirmed via identical blob SHA256 across recent Hub commits) still ships the old names.
Verified against kandinskylab/Kandinsky-6.0-Pro-distill-5s-Diffusers and
kandinskylab/Kandinsky-6.0-VSR-5s-Diffusers: this was silently dropping ~65% of the main
transformer's weights (every visual_transformer_blocks.*.{video,audio}T.* parameter) and
the SR transformer's time embeddings, then crashing with "Cannot copy out of meta tensor"
on enable_model_cpu_offload(). After this revert, all checkpoint keys match exactly
(4471/4471 for the main transformer, 459/459 for the SR transformer).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Revert checkpoint-breaking latent-upscaler renames from 372f29088
Same class of regression as 9ae71f0d0, this time in Kandinsky6SRLatentUpscalerX2Branch
and Kandinsky6SRLatentUpscalerOutputHead: today's "address comments" commit (18:48 +0300)
dropped the private_ prefix from private_mid_blocks/private_upsample/private_blocks/
private_output_proj, and converted OutputHead from nn.Sequential to a plain nn.Module with
named norm/activation/conv submodules, again without a matching checkpoint re-conversion.
Verified against kandinskylab/Kandinsky-6.0-VSR-5s-Diffusers's latent_upscaler: this left
every Kandinsky6SRLatentUpscalerX2Branch parameter (and the OutputHead-based output_proj
used standalone by the x2/x4 branches) unfilled on the meta device, crashing
enable_model_cpu_offload() the same way as the main transformer did before 9ae71f0d0.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* move vocoder and LU
* add coppied from resolve i2v cond mode
* rename variables
* fix lu rename bug
* Update src/diffusers/pipelines/kandinsky6/pipeline_kandinsky6_ti2va.py
Co-authored-by: YiYi Xu <yixu310@gmail.com>
* Update src/diffusers/models/transformers/transformer_kandinsky6.py
Co-authored-by: YiYi Xu <yixu310@gmail.com>
* address yiyixuxu's comments
* fix audio channels bug
* fix tests
* Fix CI failures on the Kandinsky6 PR
- tests/others/test_utils.py: assert the caller file path without assuming the checkout directory is named `diffusers`.
- .github/workflows/pr_tests.yml: raise PYTEST_TIMEOUT to 300 s for the example and pipeline CPU steps. Both run several workers with four threads each on an 8-vCPU runner, and the checkpointing example tests run several training subprocesses in sequence.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Avoid staging LFS dedup race in the org push test
The second push in TestModelPushToHub.test_push_to_hub_in_organization sent the same model bytes as the first push. The staging server can deduplicate those bytes against the first push's LFS object and then reject the commit with "LFS pointer pointed to a file that does not exist". Change the weights before the second push so its bytes differ.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Revert "Avoid staging LFS dedup race in the org push test"
This reverts commit 35e6cc4ef.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Revert "Fix CI failures on the Kandinsky6 PR"
This reverts commit 60cad4341.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Apply suggestion from @yiyixuxu
* Apply suggestion from @yiyixuxu
* Apply batched suggestions from code review
Co-authored-by: YiYi Xu <yixu310@gmail.com>
* remove review-only files from the Kandinsky6 PR
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCdbvRpL9fv3h3WwSUPpfS
* Apply suggestion from @yiyixuxu
* restore .gitignore to match main
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCdbvRpL9fv3h3WwSUPpfS
* simplify the mono audio handling in _write_audio
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCdbvRpL9fv3h3WwSUPpfS
* remove the no-op set_attention_backend calls from the Kandinsky6 docs
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCdbvRpL9fv3h3WwSUPpfS
* give the Kandinsky6 transformer its own output class instead of reusing LTX-2's
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCdbvRpL9fv3h3WwSUPpfS
* fix the stale pipeline exports and add return annotations for the Kandinsky6 pipelines
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCdbvRpL9fv3h3WwSUPpfS
* address feedback from @leffff
---------
Co-authored-by: Denis Koposov <denis.koposov@phystech.edu>
Co-authored-by: leffff <levnovitskiy@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Lev Novitskiy <57654885+leffff@users.noreply.github.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
* [core] Shard tensor-parallel checkpoints on load and save
Stream each rank's slice of a tensor-parallel checkpoint straight off disk
instead of materializing the full checkpoint on every rank and resharding it
afterwards, and gather the shards back on save.
- `from_pretrained(..., parallel_config=TensorParallelConfig(...))` resolves
the shard specs on the still-meta model, then slices each safetensors tensor
before the dtype cast, so host memory peaks at ~1/tp_degree of the checkpoint.
- `save_pretrained` all-gathers the DTensors into an ordinary checkpoint, or
writes a distributed checkpoint with `dcp=True` so no full tensor is ever
formed. The writing `tp_degree` is recorded, since a packed weight's stored
layout is interleaved by it.
- Factor the plan interpretation out of the Neuron pre-shard path into shared
`TPShardSpec` / `resolve_tp_shard_specs` / `_local_shard` / `_hooks_only_styles`
helpers, so both backends and both the load and save paths shard identically.
* Raise when tensor parallelism is combined with quantization, offloading or LoRA
Addresses the remaining two items of the review on #13718: tensor parallelism was rejected
alongside quantization and `device_map` only on the `from_pretrained` streaming path, while
`enable_parallelism` — which the quantization error message itself recommended — accepted a
quantized, offloaded or adapter-injected model and sharded it anyway.
- Add `_check_tp_model_state`, called from `apply_tensor_parallel`, the one chokepoint every TP
entry point funnels through. It rejects a model that is quantized, group-offloaded, placed by
accelerate (`device_map` or CPU offload), or has PEFT layers injected. Placed before the
device-type check so the reported reason is the useful one.
- Guard the reverse order too: `enable_group_offload`, the two pipeline CPU-offload methods, and
`load_lora_adapter` now refuse a tensor-parallel model.
- `save_pretrained` refuses a quantized tensor-parallel model. Previously the `dcp=True` branch
returned before the quantizer's serialization step, writing shards with no quantization
metadata and no error.
- The DCP load guard checked the `quantization_config` kwarg only, so a pre-quantized checkpoint
directory loaded silently; check the config's own entry too, and add the missing `_tp_plan`
check that otherwise surfaced as a raw `AttributeError`.
- Correct the `from_pretrained` message and the doc sentence that pointed at `enable_parallelism`
as a way to shard a quantized model.
The new tests are the first tensor-parallel tests that need neither an accelerator nor more than
one rank: every case asserts a raise before any collective, so they run single-process on gloo.
* review: remove tp save/ dcp related
* review: add a new helper .
* review: moved the TP checks into enable_parallelism
* review: move the warning under caution block
* review: add loading time numbers
* review: move tp size dividende check
* review: delete test
* review: consolidate tp checks
* review: consolidate the tp checks, all in tensor_parallel.py
* review: improve conditionals
* review: debug info
* review: check weights format before instantiating the model
* review: restore duplicated _find_mismatched_keys since irrelevant
* revert: delete duplicated helper
* review: update doc for loading benchmark of flux2-dev
* review: improve the raising check
* review: add an args section
* Test tensor-parallel loading from sharded checkpoints
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* [LoRA] convert Z-Image LoRAs that only carry the fused `qkv` and bare `out` attention keys
A LoRA trained on Z-Image's original module names (musubi-tuner, LyCORIS)
has `attention.qkv` and `attention.out` and nothing else. The non-diffusers
converter dropped both on the assumption that split `to.q/k/v` and `to_out.0`
keys are also present (the Anime-Z layout), which emptied such a LoRA and then
raised on its leftover `.alpha` keys. Split the fused key into to_q/to_k/to_v
like the single-file converter does and map bare `out` to `to_out.0` when no
split keys exist; the Anime-Z case keeps skipping the redundant keys.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* [LoRA] review follow-ups: shorter comment with the layout's origin, drop the private-method test
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: christopher5106 <christopher@scenario3d.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Both layout steps build position_ids in float64 on CPU and then move it to
the execution device with a plain .to(device), which raises on MPS. Pass the
dtype through maybe_adjust_dtype_for_device so the transfer lands as float32
on mps/npu/neuron and is unchanged everywhere else. The fp64 construction is
untouched, and MiniMaxH3RotaryPosEmbed.forward already casts position_ids to
float32, so no device sees different rotary angles.
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* Fix TaylorSeer cache crash when the feature shape changes between steps
* Add QwenImage 2.1 x TaylorSeer tests covering the KV-cache prefill shape change
---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
* [LoRA] add LoRA training for Qwen Image 2.1
Add DreamBooth LoRA training for Qwen-Image 2.1, text-to-image and
image-to-image, with fast tests and a README section.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011qnMKe5MVc7B4XXHGPvXuZ
* default rank and alpha to 16 and document the ratio
`--rank` and `--lora_alpha` are independent arguments, so raising the rank
alone leaves the update scaled by `lora_alpha / rank`. Default both to 16,
which keeps the scale at 1, and add a README section explaining the ratio.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011qnMKe5MVc7B4XXHGPvXuZ
* fix validation when embeddings are precomputed or only the final pass runs
Two fixes from review:
`QwenImage21ValidationPipeline` substituted the cached image-pad mask after
calling the base `encode_prompt`, but that call raises for a missing mask when
it is handed `prompt_embeds` together with a condition image, so the
substitution never ran and image-to-image validation failed. Pass the mask in
before the call instead.
`--final_validation_prompt` was accepted by the final-inference guard, but the
prompt embeddings were only built under `--validation_prompt`, so the final pass
reached a pipeline whose text encoder is `None` with no prompt at all. Build the
embeddings for whichever prompt is set.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011qnMKe5MVc7B4XXHGPvXuZ
* fix encoding under --offload without --cache_latents
`vae.encode` sat outside the `offload_models` block, so the context manager had
already moved the VAE back to the CPU by the time it ran, while the prepared
dataloader hands the batch over on the accelerator:
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type
(torch.FloatTensor) should be the same
Move the call inside the block and cover that flag combination, which is the
only path that encodes pixels inside the training loop. The image-to-image
trainer already had it right.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011qnMKe5MVc7B4XXHGPvXuZ
* add LoRA tests for the Qwen-Image 2.1 pipeline
Follow the Flux layout and reuse `LoraTesterMixin` and `LoraMemoryTesterMixin`,
so the adapters these trainers produce are covered by the pipeline's own
loading, fusing and memory-offload tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011qnMKe5MVc7B4XXHGPvXuZ
* fix caching a prompt mask that `encode_prompt` returns as None
`encode_prompt` drops the prompt mask when nothing in the batch is padded,
which is the common case for `--caption_column` datasets since captions in a
bucket often tokenize to the same length. The cache is filled per sample, so
slicing that `None` raised during latent caching:
prompt_embeds_mask_cache[idx] = prompt_embeds_mask[i : i + 1]
TypeError: 'NoneType' object is not subscriptable
Materialize the mask before filling the cache, and cover the path with a test:
none of the existing ones passed `--caption_column`, which is why this only
showed up on a real dataset.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011qnMKe5MVc7B4XXHGPvXuZ
* pass rank and alpha explicitly in the README dog example
The dog example was written for the old 4/4 defaults; now that both default
to 16, spell out 4/4 so the documented command keeps training the size it
was tuned for.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011qnMKe5MVc7B4XXHGPvXuZ
* drop copy-paste leftovers from other trainers
`--output_dir` defaulted to `hidream-dreambooth-lora`, the weight-decay help
mentioned UNet params, and the VAE cast comment talked about the Flux VAE.
All three were inherited verbatim from the trainer this was derived from.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011qnMKe5MVc7B4XXHGPvXuZ
* lower the README dog example to lr 1e-4
At 2e-4 the example memorizes the five training images: prompted scenes
collapse to the training backdrop rather than following the prompt. 1e-4
is what the image-to-image example already uses.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011qnMKe5MVc7B4XXHGPvXuZ
* skip the LoRA scale test for the Qwen-Image 2.1 dummy
Halving the LoRA scale moves this dummy transformer's output by less than the
tolerance the assertion allows, so the two outputs compare equal and the test
fails on CPU. Skip it with the measurement recorded rather than loosening a
tolerance shared with every other pipeline's LoRA tests.
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
A tag is mutable: whoever controls the action's repository can move
`v4` to different code, and the next run picks it up with no diff to
review. Pinning to the commit SHA freezes the code that runs today.
Co-authored-by: hf-security-analysis[bot] <265538906+hf-security-analysis[bot]@users.noreply.github.com>
A job with no `permissions:` block inherits whatever the repository
hands out. Each block added here grants what that job's own steps
need and nothing more.
Co-authored-by: hf-security-analysis[bot] <265538906+hf-security-analysis[bot]@users.noreply.github.com>
Use hf auth login in the LPL example error message
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
- TransformerSpatioTemporalModel.forward: no num_frames parameter
- get_2d_rotary_pos_embed_lumina: parameters are len_h / len_w, not grid
- AutoencoderKLHunyuanImageRefiner / AutoencoderKLHunyuanVideo15.tiled_decode:
no return_dict parameter; they return a plain tensor
Co-authored-by: revanth-045 <revanthreddych159@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
