8c3dcd6da1
* Add LoRA support for MiniMax-H3 * Address review: Copied-from loader, LoRA badge, docstring caveats, drop generic scale test * Move the MiniMax-H3 LoRA tests to tests/lora * Support the remaining published MiniMax-H3 LoRA layouts Three more published H3 LoRAs did not load. Two were key-layout gaps and the third was the reason neither was noticed: a layout that reaches no module at all used to return without an exception. - musubi-tuner writes one flattened `lora_unet_` name per module. It is un-flattened against the H3 module vocabulary, since `qkv_proj`, `token_refiner`, `final_layer` and the output heads carry underscores that are not path separators, and then goes through the existing kohya path. - one producer publishes its own converter's output: diffusers module names with peft's `.default.` infix left in and no component prefix. The infix is dropped and the prefix added. - a state dict that filters to nothing in both partitions now warns instead of loading as a silent no-op, in the wording `load_lora_adapter` uses for the same situation. A fourth loaded, but at the wrong strength. One producer ships no `.alpha` scalars and records the alpha it trained with in the file's own `__metadata__` instead, under `alpha`. Its 8-step turbo LoRA pairs that entry's 8 with rank 128, an effective scale of 0.0625, so synthesizing `alpha == rank` applied the adapter 16x too strong. That entry is now read as the uniform network alpha. Per-module scalars still win when a file carries both, since the converter has already folded them into the weights, and a non-numeric value is warned about and ignored. `__metadata__` only exists on a file, so `_fetch_state_dict` hands it back on request. * Simplify the MiniMax-H3 LoRA comments * Trim the MiniMax-H3 LoRA tests to loading and effectiveness * Reference the alpha convention in the MiniMax-H3 LoRA docstring * Drop a redundant format comment * Test only diffusers-written state dicts * Rely on get_peft_kwargs for alpha-less files * Drop the peft version guards after the main cleanup --------- Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
7.4 KiB
LoRA
LoRA is a fast and lightweight training method that inserts and trains a significantly smaller number of parameters instead of all the model parameters. This produces a smaller file (~100 MBs) and makes it easier to quickly train a model to learn a new concept. LoRA weights are typically loaded into the denoiser, text encoder or both. The denoiser usually corresponds to a UNet ([UNet2DConditionModel], for example) or a Transformer ([SD3Transformer2DModel], for example). There are several classes for loading LoRA weights:
- [
StableDiffusionLoraLoaderMixin] provides functions for loading and unloading, fusing and unfusing, enabling and disabling, and more functions for managing LoRA weights. This class can be used with any model. - [
StableDiffusionXLLoraLoaderMixin] is a Stable Diffusion (SDXL) version of the [StableDiffusionLoraLoaderMixin] class for loading and saving LoRA weights. It can only be used with the SDXL model. - [
SD3LoraLoaderMixin] provides similar functions for Stable Diffusion 3. - [
FluxLoraLoaderMixin] provides similar functions for Flux. - [
CogVideoXLoraLoaderMixin] provides similar functions for CogVideoX. - [
Mochi1LoraLoaderMixin] provides similar functions for Mochi. - [
AuraFlowLoraLoaderMixin] provides similar functions for AuraFlow. - [
LTXVideoLoraLoaderMixin] provides similar functions for LTX-Video. - [
SanaLoraLoaderMixin] provides similar functions for Sana. - [
HeliosLoraLoaderMixin] provides similar functions for HunyuanVideo. - [
HunyuanVideoLoraLoaderMixin] provides similar functions for HunyuanVideo. - [
Lumina2LoraLoaderMixin] provides similar functions for Lumina2. - [
WanLoraLoaderMixin] provides similar functions for Wan. - [
SkyReelsV2LoraLoaderMixin] provides similar functions for SkyReels-V2. - [
CogView4LoraLoaderMixin] provides similar functions for CogView4. - [
AmusedLoraLoaderMixin] is for the [AmusedPipeline]. - [
AnimaLoraLoaderMixin] provides similar functions for Anima. - [
AceStepLoraLoaderMixin] provides similar functions for ACE-Step. - [
HiDreamImageLoraLoaderMixin] provides similar functions for HiDream Image - [
QwenImageLoraLoaderMixin] provides similar functions for Qwen Image. - [
ZImageLoraLoaderMixin] provides similar functions for Z-Image. - [
Flux2LoraLoaderMixin] provides similar functions for Flux2. - [
ErnieImageLoraLoaderMixin] provides similar functions for Ernie-Image. - [
LTX2LoraLoaderMixin] provides similar functions for Flux2. - [
MiniMaxH3LoraLoaderMixin] provides similar functions for MiniMax-H3. - [
LoraBaseMixin] provides a base class with several utility methods to fuse, unfuse, unload, LoRAs and more.
Tip
To learn more about how to load LoRA weights, see the LoRA loading guide.
LoraBaseMixin
autodoc loaders.lora_base.LoraBaseMixin
StableDiffusionLoraLoaderMixin
autodoc loaders.lora_pipeline.StableDiffusionLoraLoaderMixin
StableDiffusionXLLoraLoaderMixin
autodoc loaders.lora_pipeline.StableDiffusionXLLoraLoaderMixin
SD3LoraLoaderMixin
autodoc loaders.lora_pipeline.SD3LoraLoaderMixin
FluxLoraLoaderMixin
autodoc loaders.lora_pipeline.FluxLoraLoaderMixin
Flux2LoraLoaderMixin
autodoc loaders.lora_pipeline.Flux2LoraLoaderMixin
ErnieImageLoraLoaderMixin
autodoc loaders.lora_pipeline.ErnieImageLoraLoaderMixin
LTX2LoraLoaderMixin
autodoc loaders.lora_pipeline.LTX2LoraLoaderMixin
CogVideoXLoraLoaderMixin
autodoc loaders.lora_pipeline.CogVideoXLoraLoaderMixin
Mochi1LoraLoaderMixin
autodoc loaders.lora_pipeline.Mochi1LoraLoaderMixin
AuraFlowLoraLoaderMixin
autodoc loaders.lora_pipeline.AuraFlowLoraLoaderMixin
LTXVideoLoraLoaderMixin
autodoc loaders.lora_pipeline.LTXVideoLoraLoaderMixin
SanaLoraLoaderMixin
autodoc loaders.lora_pipeline.SanaLoraLoaderMixin
HeliosLoraLoaderMixin
autodoc loaders.lora_pipeline.HeliosLoraLoaderMixin
HunyuanVideoLoraLoaderMixin
autodoc loaders.lora_pipeline.HunyuanVideoLoraLoaderMixin
Lumina2LoraLoaderMixin
autodoc loaders.lora_pipeline.Lumina2LoraLoaderMixin
CogView4LoraLoaderMixin
autodoc loaders.lora_pipeline.CogView4LoraLoaderMixin
WanLoraLoaderMixin
autodoc loaders.lora_pipeline.WanLoraLoaderMixin
SkyReelsV2LoraLoaderMixin
autodoc loaders.lora_pipeline.SkyReelsV2LoraLoaderMixin
AmusedLoraLoaderMixin
autodoc loaders.lora_pipeline.AmusedLoraLoaderMixin
AnimaLoraLoaderMixin
autodoc loaders.lora_pipeline.AnimaLoraLoaderMixin
AceStepLoraLoaderMixin
autodoc loaders.lora_pipeline.AceStepLoraLoaderMixin
HiDreamImageLoraLoaderMixin
autodoc loaders.lora_pipeline.HiDreamImageLoraLoaderMixin
QwenImageLoraLoaderMixin
autodoc loaders.lora_pipeline.QwenImageLoraLoaderMixin
ZImageLoraLoaderMixin
autodoc loaders.lora_pipeline.ZImageLoraLoaderMixin
CosmosLoraLoaderMixin
autodoc loaders.lora_pipeline.CosmosLoraLoaderMixin
KandinskyLoraLoaderMixin
autodoc loaders.lora_pipeline.KandinskyLoraLoaderMixin
Ideogram4LoraLoaderMixin
autodoc loaders.lora_pipeline.Ideogram4LoraLoaderMixin
Krea2LoraLoaderMixin
autodoc loaders.lora_pipeline.Krea2LoraLoaderMixin
MiniMaxH3LoraLoaderMixin
autodoc loaders.lora_pipeline.MiniMaxH3LoraLoaderMixin
LoraBaseMixin
autodoc loaders.lora_base.LoraBaseMixin