3jSSKrSSKJr SSKrSSKJr SSKJrJrJ r J r J r J r J r JrJrJrJr SSKJrJrJrJrJrJr SSKJrJrJrJrJrJrJ r J!r!J"r"J#r#J$r$J%r%J&r&J'r'J(r(J)r)J*r* S r+\"S S 5(a6\ "5(a*\ "S S 5(a\"5(a\"S S5(aSr+\RX"\-5r.Sr/Sr0Sr1Sr2SS0r3Sr4"SS\5r5"SS\5r6"SS\5r7"SS\5r8"SS \5r9"S!S"\55r:"S#S$\5r;"S%S&\5r<"S'S(\5r="S)S*\5r>"S+S,\5r?"S-S.\5r@"S/S0\5rA"S1S2\5rB"S3S4\5rC"S5S6\5rD"S7S8\5rE"S9S:\5rF"S;S<\5rG"S=S>\5rH"S?S@\5rI"SASB\5rJ"SCSD\5rK"SESF\55rLg)GN)Callable)validate_hf_hub_args) USE_PEFT_BACKEND deprecateget_submodule_by_nameis_bitsandbytes_availableis_gguf_availableis_peft_availableis_peft_versionis_torch_versionis_transformers_availableis_transformers_versionlogging)LORA_WEIGHT_NAMELORA_WEIGHT_NAME_SAFE LoraBaseMixin_fetch_state_dict_load_lora_into_text_encoder_pack_dict_with_prefix)+_convert_bfl_flux_control_lora_to_diffusers&_convert_fal_kontext_lora_to_diffusers(_convert_hunyuan_video_lora_to_diffusers&_convert_kohya_flux2_lora_to_diffusers%_convert_kohya_flux_lora_to_diffusers%_convert_musubi_wan_lora_to_diffusers._convert_non_diffusers_flux2_lora_to_diffusers0_convert_non_diffusers_hidream_lora_to_diffusers(_convert_non_diffusers_lora_to_diffusers-_convert_non_diffusers_ltx2_lora_to_diffusers-_convert_non_diffusers_ltxv_lora_to_diffusers0_convert_non_diffusers_lumina2_lora_to_diffusers-_convert_non_diffusers_qwen_lora_to_diffusers,_convert_non_diffusers_wan_lora_to_diffusers0_convert_non_diffusers_z_image_lora_to_diffusers%_convert_xlabs_flux_lora_to_diffusers"_maybe_map_sgm_blocks_to_diffusersF>=z1.9.00.13.1>z4.45.2T text_encoderunet transformer connectors x_embedder in_channelsc[5(aSSKJn [5(aSSKJn UR RRS:HnUR RRS:HnUR RRS:HnU(a[5(d [S5eU(a[5(d [S5eU(a[5(d [S 5eS nUR RRS :XaS n[[S 5(a([RR5ROSnU(dU(apW"U(aUR R!U5O UR U(aUR R"O UR$UR&S9R(n OmU(aPW"U(aUR R!U5O UR 5n U R!UR&5n OUR R(n U(aU R+5n U $)Nr)dequantize_bnb_weight)dequantize_gguf_tensor Params4bit Int8Params GGUFParameterz~The checkpoint seems to have been quantized with `bitsandbytes` (4bits). Install `bitsandbytes` to load quantized checkpoints.z~The checkpoint seems to have been quantized with `bitsandbytes` (8bits). Install `bitsandbytes` to load quantized checkpoints.zfThe checkpoint seems to have been quantized with `gguf`. Install `gguf` to load quantized checkpoints.FcpuT acceleratorcuda)statedtype)r quantizers.bitsandbytesr3r quantizers.gguf.utilsr4weight __class____name__ ValueErrordevicetypehasattrtorchr9current_acceleratorto quant_stater;r<datar8) modelmoduler3r4is_bnb_4bit_quantizedis_bnb_8bit_quantizedis_gguf_quantized weight_on_cpurC module_weights Y/home/wildlama/miniconda3/lib/python3.13/site-packages/diffusers/loaders/lora_pipeline.py*_maybe_dequantize_weight_for_expanded_lorarSTs ""CB"MM33<< L"MM33<< L //88OK%>%@%@ M  %>%@%@ M  !2!4!4 t  M }}  E) =DUM=Z=ZU   2 2 4 9 9`fF 5-(5FMM  V $6==/D&--++&,,++  $  .(5FMM  V $6== &((5  ** %))+ cP^\rSrSrSrSS/r\r\r SS\ \ \ \ R4-S\ S-S \4S jjr\\S\ \ \ \ R4-4S j55r\SS \4S jj5r\S S \4Sjj5r\S!S\ \R,-S\ \ \ R.R0\ R-4S\ \ \ R.R04S\S\ S\S\4Sjj5rSS/S SS4S\\ S\S\S\\ S-4U4SjjjrSS/4S\\ 4U4SjjjrSrU=r $)"StableDiffusionLoraLoaderMixinz Load LoRA layers into Stable Diffusion [`UNet2DConditionModel`] and [`CLIPTextModel`](https://huggingface.co/docs/transformers/model_doc/clip#transformers.CLIPTextModel). r-r,NF%pretrained_model_name_or_path_or_dict adapter_namehotswapc [(d [S5eURS[5nU(a[ SS5(d [S5e[ U[ 5(aUR5nSUS'UR"U40UD6upgn[SUR555n U (d [S 5eURUU[US 5(d[XR5O URUUUUUS 9 UR!UU[US 5(d[XR"5O UR$UR&UUUUUS 9 g)aw Load LoRA weights specified in `pretrained_model_name_or_path_or_dict` into `self.unet` and `self.text_encoder`. All kwargs are forwarded to `self.lora_state_dict`. See [`~loaders.StableDiffusionLoraLoaderMixin.lora_state_dict`] for more details on how the state dict is loaded. See [`~loaders.StableDiffusionLoraLoaderMixin.load_lora_into_unet`] for more details on how the state dict is loaded into `self.unet`. See [`~loaders.StableDiffusionLoraLoaderMixin.load_lora_into_text_encoder`] for more details on how the state dict is loaded into `self.text_encoder`. Parameters: pretrained_model_name_or_path_or_dict (`str` or `os.PathLike` or `dict`): See [`~loaders.StableDiffusionLoraLoaderMixin.lora_state_dict`]. adapter_name (`str`, *optional*): Adapter name to be used for referencing the loaded adapter model. If not specified, it will use `default_{i}` where i is the total number of adapters being loaded. low_cpu_mem_usage (`bool`, *optional*): Speed up model loading by only loading the pretrained LoRA weights and not initializing the random weights. hotswap (`bool`, *optional*): Defaults to `False`. Whether to substitute an existing (LoRA) adapter with the newly loaded adapter in-place. This means that, instead of loading an additional adapter, this will take the existing adapter weights and replace them with the weights of the new adapter. This can be faster and more memory efficient. However, the main advantage of hotswapping is that when the model is compiled with torch.compile, loading the new adapter does not require recompilation of the model. When using hotswapping, the passed `adapter_name` should be the name of an already loaded adapter. If the new adapter and the old adapter have different ranks and/or LoRA alphas (i.e. scaling), you need to call an additional method before loading the adapter: ```py pipeline = ... # load diffusers pipeline max_rank = ... # the highest rank among all LoRAs that you want to load # call *before* compiling and loading the LoRA adapter pipeline.enable_lora_hotswap(target_rank=max_rank) pipeline.load_lora_weights(file_name) # optionally compile the model now ``` Note that hotswapping adapters of the text encoder is not yet supported. There are some further limitations to this technique, which are documented here: https://huggingface.co/docs/peft/main/en/package_reference/hotswap kwargs (`dict`, *optional*): See [`~loaders.StableDiffusionLoraLoaderMixin.lora_state_dict`]. )PEFT backend is required for this method.low_cpu_mem_usager)r*q`low_cpu_mem_usage=True` is not compatible with this `peft` version. Please update it with `pip install -U peft`.Treturn_lora_metadatac3,# UH nSU;v M g7floraN.0keys rR CStableDiffusionLoraLoaderMixin.load_lora_weights..K9J## 9JSInvalid LoRA checkpoint. Make sure all LoRA param names contain `'lora'` substring.r-network_alphasr-rYmetadata _pipeliner]rZr,)rmr, lora_scalerYrornr]rZN)rrBpop_LOW_CPU_MEM_USAGE_DEFAULT_LORAr isinstancedictcopylora_state_dictallkeysload_lora_into_unetrEgetattr unet_namer-load_lora_into_text_encodertext_encoder_namer,rp selfrXrYrZkwargsr] state_dictrmrnis_correct_formats rRload_lora_weights0StableDiffusionLoraLoaderMixin.load_lora_weightss[p HI I"JJ':<[\ _T8%D%DD  ;T B B4Y4^4^4` 1*.%&/3/C/CDi/tms/t, HK9JKK rs s   )6=dF6K6K~~.QUQZQZ%/ !  (( )400!'='=>""%/ ) rTc PURSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n URS S5n SnU cSn SnSSS.n[UU U UUUUUUU UUS9 unn[SU55nU(aESn[R U5 UR 5VVs0sHunnSU;dMUU_M nnnSn[ SUR555(aU b [UU 5n[U5unnU (aUUU4nU$UU4nU$s snnf) Return state dict for lora weights and the network alphas. > [!WARNING] > We support loading A1111 formatted LoRA checkpoints in a limited capacity. > > This function is experimental and might change in the future. Parameters: pretrained_model_name_or_path_or_dict (`str` or `os.PathLike` or `dict`): Can be either: - A string, the *model id* (for example `google/ddpm-celebahq-256`) of a pretrained model hosted on the Hub. - A path to a *directory* (for example `./my_model_directory`) containing the model weights saved with [`ModelMixin.save_pretrained`]. - A [torch state dict](https://pytorch.org/tutorials/beginner/saving_loading_models.html#what-is-a-state-dict). cache_dir (`str | os.PathLike`, *optional*): Path to a directory where a downloaded pretrained model configuration is cached if the standard cache is not used. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist. proxies (`dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, for example, `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request. local_files_only (`bool`, *optional*, defaults to `False`): Whether to only load local model weights and configuration files or not. If set to `True`, the model won't be downloaded from the Hub. token (`str` or *bool*, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, the token generated from `diffusers-cli login` (stored in `~/.huggingface`) is used. revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, a commit id, or any identifier allowed by Git. subfolder (`str`, *optional*, defaults to `""`): The subfolder location of a model file within a larger model repository on the Hub or locally. weight_name (`str`, *optional*, defaults to None): Name of the serialized state dict file. return_lora_metadata (`bool`, *optional*, defaults to False): When enabled, additionally return the LoRA adapter metadata, typically found in the state dict. cache_dirNforce_downloadFproxieslocal_files_onlytokenrevision subfolder weight_name unet_configuse_safetensorsr_Tattn_procs_weightspytorch file_type framework rXrrrrrrrrr user_agent allow_picklec3,# UH nSU;v M g7f dora_scaleNrcreks rRrgAStableDiffusionLoraLoaderMixin.lora_state_dict..I#Jz!LA$5zrj!It seems like you are using a DoRA checkpoint that is not compatible in Diffusers at the moment. So, we are going to filter out the keys associated to 'dora_scale` from the state dict. If you think this is a mistake please open an issue https://github.com/huggingface/diffusers/issues/new.rc3# UH^nURS5=(dA URS5=(d) URS5=(d URS5v M` g7flora_te_ lora_unet_ lora_te1_ lora_te2_N startswithrs rRrgrQ^ '  Z(-<< --<< ,-<< ,- 'A&A( rqranyloggerwarningitemsrwrxr(r clsrXrrrrrrrrrrrr_rrrrnis_dora_scale_presentwarn_msgrvrmouts rRrv.StableDiffusionLoraLoaderMixin.lora_state_dictshJJ{D1 $4e<**Y-!::&8$? 7D)::j$/JJ{D1 jj5 jj5  **%6=%zz*@%H  ""OL#7iP 02W#+-)!%  H!$#Jz#J J {H NN8 $+5+;+;+=W+=41aUVAV$!Q$+=JW   __&   &? KX )QR\)] &J8Lz>84 T^_mRn )X / F"F"c [(d [S5eU(a[SS5(d [S5e[R SUR S35 UR UUR UUUUUUS9 g a This will load the LoRA layers specified in `state_dict` into `unet`. Parameters: state_dict (`dict`): A standard state dict containing the lora layer parameters. The keys can either be indexed directly into the unet or prefixed with an additional `unet` which can be used to distinguish between text encoder lora layers. network_alphas (`dict[str, float]`): The value of the network alpha used for stable learning and preventing underflow. This value has the same meaning as the `--network_alpha` option in the kohya-ss trainer script. Refer to [this link](https://github.com/darkstorm2150/sd-scripts/blob/main/docs/train_network_README-en.md#execute-learning). unet (`UNet2DConditionModel`): The UNet model to load the LoRA layers into. adapter_name (`str`, *optional*): Adapter name to be used for referencing the loaded adapter model. If not specified, it will use `default_{i}` where i is the total number of adapters being loaded. low_cpu_mem_usage (`bool`, *optional*): Speed up model loading only loading the pretrained LoRA weights and not initializing the random weights. hotswap (`bool`, *optional*): See [`~loaders.StableDiffusionLoraLoaderMixin.load_lora_weights`]. metadata (`dict`): Optional LoRA adapter metadata. When supplied, the `LoraConfig` arguments of `peft` won't be derived from the state dict. r\r)r*r^Loading .)prefixrmrYrnror]rZNrrBr rinfor{load_lora_adapter rrrmr-rYror]rZrns rRry2StableDiffusionLoraLoaderMixin.load_lora_into_unetcsL HI I _T8%D%DD   hs}}oQ/0  ==)%/  rT?c >[UUUUUURUU UUU S9 ga} This will load the LoRA layers specified in `state_dict` into `text_encoder` Parameters: state_dict (`dict`): A standard state dict containing the lora layer parameters. The key should be prefixed with an additional `text_encoder` to distinguish between unet lora layers. network_alphas (`dict[str, float]`): The value of the network alpha used for stable learning and preventing underflow. This value has the same meaning as the `--network_alpha` option in the kohya-ss trainer script. Refer to [this link](https://github.com/darkstorm2150/sd-scripts/blob/main/docs/train_network_README-en.md#execute-learning). text_encoder (`CLIPTextModel`): The text encoder model to load the LoRA layers into. prefix (`str`): Expected prefix of the `text_encoder` in the `state_dict`. lora_scale (`float`): How much to scale the output of the lora linear layer before it is added with the output of the regular lora layer. adapter_name (`str`, *optional*): Adapter name to be used for referencing the loaded adapter model. If not specified, it will use `default_{i}` where i is the total number of adapters being loaded. low_cpu_mem_usage (`bool`, *optional*): Speed up model loading by only loading the pretrained LoRA weights and not initializing the random weights. hotswap (`bool`, *optional*): See [`~loaders.StableDiffusionLoraLoaderMixin.load_lora_weights`]. metadata (`dict`): Optional LoRA adapter metadata. When supplied, the `LoraConfig` arguments of `peft` won't be derived from the state dict. ) rrmrpr,rr}rYrnror]rZNrr} rrrmr,rrprYror]rZrns rRr|:StableDiffusionLoraLoaderMixin.load_lora_into_text_encoders4X %!)!%!33%/ rTsave_directoryunet_lora_layerstext_encoder_lora_layersis_main_processr save_functionsafe_serializationc 0n 0n U(aX*UR'XUR'U(aX:UR'XUR'U (d [S5eURUU U UUUUS9 g)ux Save the LoRA parameters corresponding to the UNet and text encoder. Arguments: save_directory (`str` or `os.PathLike`): Directory to save LoRA parameters to. Will be created if it doesn't exist. unet_lora_layers (`dict[str, torch.nn.Module]` or `dict[str, torch.Tensor]`): State dict of the LoRA layers corresponding to the `unet`. text_encoder_lora_layers (`dict[str, torch.nn.Module]` or `dict[str, torch.Tensor]`): State dict of the LoRA layers corresponding to the `text_encoder`. Must explicitly pass the text encoder LoRA state dict because it comes from 🤗 Transformers. is_main_process (`bool`, *optional*, defaults to `True`): Whether the process calling this is the main process or not. Useful during distributed training and you need to call this function on all processes. In this case, set `is_main_process=True` only on the main process to avoid race conditions. save_function (`Callable`): The function to use to save the state dictionary. Useful during distributed training when you need to replace `torch.save` with another method. Can be configured with the environment variable `DIFFUSERS_SAVE_MODE`. safe_serialization (`bool`, *optional*, defaults to `True`): Whether to save the model using `safetensors` or the traditional PyTorch way with `pickle`. unet_lora_adapter_metadata: LoRA adapter metadata associated with the unet to be serialized with the state dict. text_encoder_lora_adapter_metadata: LoRA adapter metadata associated with the text encoder to be serialized with the state dict. zOYou must pass at least one of `unet_lora_layers` or `text_encoder_lora_layers`.r lora_layers lora_metadatarrrrN)r{r}rB_save_lora_weights) rrrrrrrrunet_lora_adapter_metadata"text_encoder_lora_adapter_metadatarrs rRsave_lora_weights0StableDiffusionLoraLoaderMixin.save_lora_weightssN  )9 &+E#-- ( #1I-- .3U#// 0no o )#'+#'1  rT componentsrp safe_fusing adapter_namesc 0>[TU]"SUUUUS.UD6 g)a Fuses the LoRA parameters into the original parameters of the corresponding blocks. > [!WARNING] > This is an experimental API. Args: components: (`list[str]`): list of LoRA-injectable components to fuse the LoRAs into. lora_scale (`float`, defaults to 1.0): Controls how much to influence the outputs with the LoRA parameters. safe_fusing (`bool`, defaults to `False`): Whether to check fused weights for NaN values before fusing and if values are NaN not fusing them. adapter_names (`list[str]`, *optional*): Adapter names to be used for fusing. If nothing is passed, all active adapters will be fused. Example: ```py from diffusers import DiffusionPipeline import torch pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16 ).to("cuda") pipeline.load_lora_weights("nerijs/pixel-art-xl", weight_name="pixel-art-xl.safetensors", adapter_name="pixel") pipeline.fuse_lora(lora_scale=0.7) ``` rrprrNrcsuper fuse_lorarrrprrrr@s rRr(StableDiffusionLoraLoaderMixin.fuse_loras.F  !!#'    rTc *>[TU]"SSU0UD6 g)aS Reverses the effect of [`pipe.fuse_lora()`](https://huggingface.co/docs/diffusers/main/en/api/loaders#diffusers.loaders.LoraBaseMixin.fuse_lora). > [!WARNING] > This is an experimental API. Args: components (`list[str]`): list of LoRA-injectable components to unfuse LoRA from. unfuse_unet (`bool`, defaults to `True`): Whether to unfuse the UNet LoRA parameters. unfuse_text_encoder (`bool`, defaults to `True`): Whether to unfuse the text encoder LoRA parameters. If the text encoder wasn't monkey-patched with the LoRA parameters then it won't have any effect. rNrcr unfuse_lorarrrr@s rRr*StableDiffusionLoraLoaderMixin.unfuse_loraDs 2J=d3i==rTrVc^\rSrSrSr/SQr\r\r SS\ \ \ \ R4-S\ S-S\4S jjr\\S\ \ \ \ R4-4S j55r\SS\4S jj5r\S S\4S jj5r\S!S\ \R,-S\ \ \ R.R0\ R-4S\ \ \ R.R0\ R-4S\ \ \ R.R0\ R-4S\S\ S\S\4Sjj5r/SQS SS4S\\ S\S\S\\ S-4U4Sjjjr/SQ4S\\ 4U4SjjjrSrU=r $)" StableDiffusionXLLoraLoaderMixiniUa3 Load LoRA layers into Stable Diffusion XL [`UNet2DConditionModel`], [`CLIPTextModel`](https://huggingface.co/docs/transformers/model_doc/clip#transformers.CLIPTextModel), and [`CLIPTextModelWithProjection`](https://huggingface.co/docs/transformers/model_doc/clip#transformers.CLIPTextModelWithProjection). )r-r,text_encoder_2NFrXrYrZc [(d [S5eURS[5nU(a[ SS5(d [S5e[ U[ 5(aUR5nSUS'UR"U4SURR0UD6upgn[S UR555n U (d [S 5eURUUURUUUUUS 9 URUUURUR UR"UUUUUS 9 URUUUR$UR S 3UR"UUUUUS 9 g)U See [`~loaders.StableDiffusionLoraLoaderMixin.load_lora_weights`] for more details. r\r]r)r*r^Tr_rc3,# UH nSU;v M g7frarcrds rRrgEStableDiffusionXLLoraLoaderMixin.load_lora_weights..rirjrkrl rmr,rrprYrnror]rZ_2N)rrBrqrrr rsrtrurvr-configrwrxryr|r,r}rprr~s rRr2StableDiffusionXLLoraLoaderMixin.load_lora_weights`s HI I"JJ':<[\ _T8%D%DD  ;T B B4Y4^4^4` 1*.%&/3/C/C 10  ((0 0 , H  K9JKK rs s   )%/ !  (( )**))%/ )  (( ),,,,-R0%/ ) rTc PURSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n URS S5n SnU cSn SnSSS.n[UU U UUUUUUU UUS9 unn[SU55nU(aESn[R U5 UR 5VVs0sHunnSU;dMUU_M nnnSn[ SUR555(aU b [UU 5n[U5unnU (aUUU4nU$UU4nU$s snnf)rrNrFrrrrrrrrr_Trrrrc3,# UH nSU;v M g7frrcrs rRrgCStableDiffusionXLLoraLoaderMixin.lora_state_dict..rrjrrc3# UH^nURS5=(dA URS5=(d) URS5=(d URS5v M` g7frrrs rRrgr rrrrs rRrv0StableDiffusionXLLoraLoaderMixin.lora_state_dictsjJJ{D1 $4e<**Y-!::&8$? 7D)::j$/JJ{D1 jj5 jj5  **%6=%zz*@%H  ""OL#7iP 02W#+-)!%  H!$#Jz#J J {H NN8 $+5+;+;+=W+=41aUVAV$!Q$+=JW   __&   &? KX )QR\)] &J8Lz>84 T^_mRn )Xrc [(d [S5eU(a[SS5(d [S5e[R SUR S35 UR UUR UUUUUUS9 grrrs rRry4StableDiffusionXLLoraLoaderMixin.load_lora_into_unetsN HI I _T8%D%DD   hs}}oQ/0  ==)%/  rTrc >[UUUUUURUU UUU S9 grrrs rRr|[TU]"SUUUUS.UD6 gzM See [`~loaders.StableDiffusionLoraLoaderMixin.fuse_lora`] for more details. rNrcrrs rRr*StableDiffusionXLLoraLoaderMixin.fuse_lora-  !!#'    rTc *>[TU]"SSU0UD6 gzO See [`~loaders.StableDiffusionLoraLoaderMixin.unfuse_lora`] for more details. rNrcrrs rRr,StableDiffusionXLLoraLoaderMixin.unfuse_lora .rrjrrrqrrrrrrrXrrrrrrrrrrr_rrrrnrrrrrs rRrv"SD3LoraLoaderMixin.lora_state_dicts|JJ{D1 $4e<**Y-!::&8$? 7D)::j$/JJ{D1 jj5  **%6=%zz*@%H  ""OL#7iP 02W#+-)!%  H!$#Jz#J J {H NN8 $+5+;+;+=W+=41aUVAV$!Q$+=JW(.;rirjrkr.r.rYrnror]rZNrr)rrBrqrrr rsrtrurvrwrxload_lora_into_transformerrErztransformer_namer.r|r,r}rpr rrXrYrZrr]rrnrs rRr$SD3LoraLoaderMixin.load_lora_weights s HI I"JJ':<[\ h!?!?D  ;T B B4Y4^4^4` 1*.%&#334Yd]cd K9JKK rs s '' DKDR_D`D`&;&;[UUUUUURUU UUU S9 grrrs rRr|.SD3LoraLoaderMixin.load_lora_into_text_encoderrrTrtransformer_lora_layersrrrrrrc 0n 0n U(aX,UR'XUR'U(aX[TU]"SUUUUS.UD6 grrrs rRrSD3LoraLoaderMixin.fuse_lorarrTc *>[TU]"SSU0UD6 grrrs rRrSD3LoraLoaderMixin.unfuse_lorar rTrcrrrr!)!rArrrrrTRANSFORMER_NAMEr3rr}rrrrtrFrrvrrr2r|rrrrrrrrrrrrrs@rRr#r#s1O')2/2T#u||:K5L/L22n ? /2T#u||:K5L/L?  ? B    <7 7 7 r NRNRPT $"&#'*.+/-1- bkk)- "&c588??U\\+I&I!J- #'sEHHOOell,J'J"K - %)ehhoo .L)L$M -  - -  - !- - d!R!*.  I    Cy4'   (3d=d3i==rTr#c^\rSrSrSrS/r\r\\ S\ \ \ \ R4-4Sj55rSS\ \ \ \ R4-S\ S-S \4S jjr\SS \4S jj5r\SS \ \R&-S \ \ \ R(R*\ R-4S\S\ S\S\S\ S-4Sjj5rS/SSS4S\\ S\S\S\\ S-4U4SjjjrSS/4S\\ 4U4SjjjrSrU=r$) AuraFlowLoraLoaderMixinizX Load LoRA layers into [`AuraFlowTransformer2DModel`] Specific to [`AuraFlowPipeline`]. r.rXc URSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n Sn U cS n S n SSS.n[UU U UUUUUUU UU S9 unn[SU55nU(aESn[R U5 UR 5VVs0sHunnSU;dMUU_M nnnU (aUU4nU$UnU$s snnf)r%rNrFrrrrrrrr_Trrrrc3,# UH nSU;v M g7frrcrs rRrg:AuraFlowLoraLoaderMixin.lora_state_dict..>rrjrrr(r)s rRrv'AuraFlowLoraLoaderMixin.lora_state_dict|JJ{D1 $4e<**Y-!::&8$? 7D)::j$/JJ{D1 jj5  **%6=%zz*@%H  ""OL#7iP 02W#+-)!%  H!$#Jz#J J {H NN8 $+5+;+;+=W+=41aUVAV$!Q$+=JW(.crirjrkr.r1NrrBrqrrr rsrtrurvrwrxr2rErzr3r.r4s rRr)AuraFlowLoraLoaderMixin.load_lora_weightsH HI I"JJ':<[\ h!?!?D  ;T B B4Y4^4^4` 1*.%&#334Yd]cd K9JKK rs s '' DKDR_D`D`&;&;[TU]"SUUUUS.UD6 grrrs rRr!AuraFlowLoraLoaderMixin.fuse_lorarrTr,c *>[TU]"SSU0UD6 grrrs rRr#AuraFlowLoraLoaderMixin.unfuse_lorar rTrcrrNTNNTNrArrrrrrHr3rrrrtrFrrvrrr2rrrrrrrrrrrrrs@rRrJrJs,_'2/2T#u||:K5L/L22p$( ' /2T#u||:K5L/L' Dj'  ' R    < NR $"&#'9= bkk) "&c588??U\\+I&I!J       ! ,0$;  H"/!*.  I    Cy4'   (4A.2Q=d3i==rTrJcL^\rSrSrSrSS/r\r\r /SQr \ \ S-S\ \\ \R 4-S\4S jj55rS.S\ \\ \R 4-S \ S -S \4S jjr\ S/S \4Sjj5r\ S.S\\ \R 44Sjj5r\ S0S \4Sjj5r\ S1S\ \R0-S\\ \R2R4\R -4S\\ \R2R44S\S\ S\S\4Sjj5rS/SSS 4S\\ S\S\S\\ S -4U4SjjjrSS/4S\\ 4U4S jjjr S-U4S!jjr!\ S2S\R2R4S\4S"jj5r"\ S#5r#\$S3S$S%S&S'S(\ SS)4S*jj5r%\$S4S+j5r&S,r'U=r($)5FluxLoraLoaderMixiniz Load LoRA layers into [`FluxTransformer2DModel`], [`CLIPTextModel`](https://huggingface.co/docs/transformers/model_doc/clip#transformers.CLIPTextModel). Specific to [`FluxPipeline`]. r.r,)norm_qnorm_k norm_added_q norm_added_kFrX return_alphasc URSS5nURSS5nURSS5nURSS5nURSS5nURSS5n URS S5n URS S5n URS S5n URS S5n SnU cS n S nSSS.n[UU U UUUUUU U UUS9 unn[SU55nU(aESn[R U5 UR 5VVs0sHunnSU;dMUU_M nnn[SU55nU(a[ U5nURUUSUU S9$[SU55nU(a[U5nURUUSUU S9$[SU55nU(a[U5nURUUSUU S9$[SU55nU(a[U5nURUUSUU S9$[UR55n0nUHnSU;dM URU5n[R"U5(a[R "U5(d[#U[$5(aURU5UU'M}['SUS35e U(dU (aURUUUUU S9$U$s snnf)r%rNrFrrrrrrrr_Trrrrc3,# UH nSU;v M g7frrcrs rRrg6FluxLoraLoaderMixin.lora_state_dict.. rrjrrc3,# UH nSU;v M g7fz.lora_down.weightNrcrs rRrgrnDA*a/rj)rnalphasrkreturn_metadatac3,# UH nSU;v M g7f) processorNrcrs rRrgrns<A{a'rjc3,# UH nSU;v M g7f)zquery_norm.scaleNrcrs rRrgrn)sIj/14jrjc3,# UH nSU;v M g7f) base_modelNrcrs rRrgrn4sC 1\Q. rjalphazThe alpha key (zU) seems to be incorrect. If you think this error is unexpected, please open as issue.)rqrrrrrr_prepare_outputsr'rrrrxgetrF is_tensoris_floating_pointrsrrB)rrXrkrrrrrrrrrrr_rrrrnrrrris_kohyais_xlabsis_bfl_controlis_fal_kontextrxrm alpha_values rRrv#FluxLoraLoaderMixin.lora_state_dictsLJJ{D1 $4e<**Y-!::&8$? 7D)::j$/JJ{D1 jj5  **%6=%zz*@%H  ""OL#7iP 02W#+-)!%  H!$#Jz#J J {H NN8 $+5+;+;+=W+=41aUVAV$!Q$+=JWDDD >zJJ''!+ 4 ( <<< >zJJ''!+ 4 ( IjII DZPJ''!+ 4 ( C CC ? KJ''!+ 4 ( JOO%&A!|(nnQ/ OOK00U5L5L[5Y5Y^h__)3q(9N1%$)!-BC 0''!%+ 4 (  UXs  K".K"NrYrZc ^^ [(d [S5eURS[5nU(a[ SS5(d [S5e[ U[ 5(aUR5nSUS'TR"U4SS0UD6upgn[S UR555n [U4S jUR555n U (dU (d [S 5e[UR55V s0sHAn U RTRS 35(dM(S U ;dM0XRU 5_MC n n [UR55V ^ s0sH`m T RTRS 35(dM([U 4SjTR55(dMNT URT 5_Mb n n [!TS5(d[#TTR5O TR$nSn['U 5S:aTR)XU 5nU(a[*R-S5 ['U 5S:a.TR/XS9n U Hm UR1T U T 05 M TR3UUUUUTUUS9 ['U 5S:aTR5U USS9UlTR9UUTR:TR<TR>UUTUUS9 gs sn fs sn f)rr\r]r)r*r^Tr_rkc3,# UH nSU;v M g7frarcrds rRrg8FluxLoraLoaderMixin.load_lora_weights..wsG5FcFcM5Frjc3T># UHnTRH o"U;v M M g7fN!_control_lora_supported_norm_keys)rerfnorm_keyrs rRrgrzs& '8IoIoXOIoO'8%(rkrrbc3,># UH oT;v M g7frrc)rerrs rRrgrsY2XhM2Xsr.FraThe LoRA weights contain parameters that have different shapes that expected by the transformer. As a result, the state_dict of the transformer has been expanded to match the LoRA parameter shapes. To get a comprehensive list of parameter names that were modified, enable debug logging.)r.rv)rmr.rYrnror]rZ)r.discard_original_layersrN) rrBrqrrr rsrtrurvrrxrrr3r{rrErzr.len/_maybe_expand_transformer_param_shape_or_error_rr_maybe_expand_lora_state_dictupdater2_load_norm_into_transformer_transformer_norm_layersr|r,r}rp)rrXrYrZrr]rrmrn has_lora_keys has_norm_keysrtransformer_lora_state_dicttransformer_norm_state_dictr.has_param_with_expanded_shapes` ` rRr%FluxLoraLoaderMixin.load_lora_weightsZs+ HI I"JJ':<[\ _T8%D%DD  ;T B B4Y4^4^4` 1*.%&/3/C/C 10 AE0 IO0 , HGZ__5FGG  '1'8  rs s*//+,' ,||t445Q78 !=Cq[ !A~~a , $' *//+,' ,||t445Q78 !Y$2X2XYY !Az~~a , $' CJ$P]B^B^gdD$9$9:dhdtdt (-% * +a /,0,`,`:U- ) ) KKk  * +a /*.*L*L'+M+ '1!!1&A!&D"EF1 '' )#%/ (  * +a /373S3S+'(-4T4K 0 (( )**))%/ ) e' ' s$ 'K;6K;>K;1'L"LLc U(a[SS5(d [S5e[RSURS35 UR UUUUUUUS9 gr8r)r*r^rrr9Nr: rrrmr.rYrnror]rZs rRr2.FluxLoraLoaderMixin.load_lora_into_transformersl _T8%D%DD   hs334A67%% )%/ & rTreturnc.^U=(d TRn[UR55HBnURS5SU:XdMUR U5XR US35'MD UR 5n[UR55n[UR55n[X- 5n U (a[RSU S35 U HnUR U5 M 0n U(d+UR5HnXeR5X'M [RS5 URUSS9n [U SS5n U (a)[U4SjU 55(a[S U S 35eU $) NrrzUnsupported keys found in state dict when trying to load normalization layers into the transformer. The following keys will be ignored: aWThe provided state dict contains normalization layers in addition to LoRA layers. The normalization layers will directly update the state_dict of the transformer as opposed to the LoRA layers that will co-exist separately until the "fuse_lora()" method is called. That is to say, the normalization layers will always be directly fused into the transformer and can only be unfused if `discard_original_layers=True` is passed. This might also have implications when dealing with multiple LoRAs. If you notice something unexpected, please open an issue: https://github.com/huggingface/diffusers/issues.Fstrictunexpected_keysc3T># UHnTRH o"U;v M M g7frr)rerrrs rRrgBFluxLoraLoaderMixin._load_norm_into_transformer..s"p/Q#JoJohq=Jo=/rzFound zJ as unexpected keys while trying to load norm layers into the transformer.)r3rrxsplitrq removeprefixrsetrrclonerload_state_dictrzrrB) rrr.rrrftransformer_state_dicttransformer_keysstate_dict_keys extra_keysoverwritten_layers_state_dictincompatible_keysrs ` rRr/FluxLoraLoaderMixin._load_norm_into_transformers/3// )*Cyy~a F*=G^^C=P ++vhaL9:+ "-!7!7!95::<=joo/0/<=  NN\]g\hhij C NN3 )+%&!(5K5P5V5V5X-2)   y (77 57Q!"35FM p/ppp _--wx-,rTrc >[UUUUUURUU UUU S9 grrrs rRr|/FluxLoraLoaderMixin.load_lora_into_text_encoderrrTrr?rrrrrc 0n 0n U(aX*UR'XUR'U(aX:UR'XUR'U (d [S5eURUU U UUUUS9 g)u Save the LoRA parameters corresponding to the UNet and text encoder. Arguments: save_directory (`str` or `os.PathLike`): Directory to save LoRA parameters to. Will be created if it doesn't exist. transformer_lora_layers (`dict[str, torch.nn.Module]` or `dict[str, torch.Tensor]`): State dict of the LoRA layers corresponding to the `transformer`. text_encoder_lora_layers (`dict[str, torch.nn.Module]` or `dict[str, torch.Tensor]`): State dict of the LoRA layers corresponding to the `text_encoder`. Must explicitly pass the text encoder LoRA state dict because it comes from 🤗 Transformers. is_main_process (`bool`, *optional*, defaults to `True`): Whether the process calling this is the main process or not. Useful during distributed training and you need to call this function on all processes. In this case, set `is_main_process=True` only on the main process to avoid race conditions. save_function (`Callable`): The function to use to save the state dictionary. Useful during distributed training when you need to replace `torch.save` with another method. Can be configured with the environment variable `DIFFUSERS_SAVE_MODE`. safe_serialization (`bool`, *optional*, defaults to `True`): Whether to save the model using `safetensors` or the traditional PyTorch way with `pickle`. transformer_lora_adapter_metadata: LoRA adapter metadata associated with the transformer to be serialized with the state dict. text_encoder_lora_adapter_metadata: LoRA adapter metadata associated with the text encoder to be serialized with the state dict. r[rN)r3r}rBr) rrr?rrrrrrBrrrs rRr%FluxLoraLoaderMixin.save_lora_weightsQsP  "0G,, -2S#.. / #1I-- .3U#// 0uv v )#'+#'1  rTrrprrc l>[US5(d[XR5O URn[US5(a[[ UR [ 5(a<[UR R55S:a[RS5 [TU]0"SUUUUS.UD6 g)r%r.rraThe provided state dict contains normalization layers in addition to LoRA layers. The normalization layers will be directly updated the state_dict of the transformer as opposed to the LoRA layers that will co-exist separately until the 'fuse_lora()' method is called. That is to say, the normalization layers will always be directly fused into the transformer and can only be unfused if `discard_original_layers=True` is passed.rNrc) rErzr3r.rsrrtrrxrrrr)rrrprrrr.r@s rRrFluxLoraLoaderMixin.fuse_lorasCJ$P]B^B^gd$9$9:dhdtdt K!; < <;??FFK88==?@1D KKr   !!#'    rTc >[US5(d[XR5O URn[US5(a+UR(aUR URSS9 [ TU]"SSU0UD6 g)a Reverses the effect of [`pipe.fuse_lora()`](https://huggingface.co/docs/diffusers/main/en/api/loaders#diffusers.loaders.LoraBaseMixin.fuse_lora). > [!WARNING] > This is an experimental API. Args: components (`list[str]`): list of LoRA-injectable components to unfuse LoRA from. r.rFrrNrc)rErzr3r.rrrr)rrrr.r@s rRrFluxLoraLoaderMixin.unfuse_lorasrCJ$P]B^B^gd$9$9:dhdtdt ; : ; ; @d@d  ' ' (L(LUZ ' [ [TU]5 [US5(d[XR5O UR n[US5(a2UR (a!URUR SS9 SUlU(Ga?[USS5Gb/URn[5nUH<nURS5(dMURURSS55 M> UR5GHupg[U[R R"5(dM1Xd;dM8UR$R&nUR(bUR(R&OSn U SLn UR+S 5upn UR-U 5nX6S3nUR.S UR.S nn[R0"S 5 [R R#UUU UR2S 9nSSS5 SU0nU bUR5SX6S305 WRUSSS9 [7XU5 AU [8;dGMT[8U n[;UR.S 5n[UR<U5n[7UR<UU5 [>RASUSUSUS 35 GM ggg!,(df  N=f)a Unloads the LoRA parameters. Args: reset_to_overwritten_params (`bool`, defaults to `False`): Whether to reset the LoRA-loaded modules to their original params. Refer to the [Flux documentation](https://huggingface.co/docs/diffusers/main/en/api/pipelines/flux) to learn more. Examples: ```python >>> # Assuming `pipeline` is already loaded with the LoRA parameters. >>> pipeline.unload_lora_weights() >>> ... ``` r.rFrN_overwritten_params.weightrrrmetabiasr<r?r.biasT)assignrSet the  attribute of the model to  from )!runload_lora_weightsrErzr3r.rrrrendswithaddreplace named_modulesrsrFrLinearr?rJr rpartition get_submoduleshaperCr<rsetattr"_MODULE_NAME_TO_ATTRIBUTE_MAP_FLUXintrrr)rreset_to_overwritten_paramsr.overwritten_params module_names param_namenamerLrQ module_biasrparent_module_name_current_module_name parent_modulecurrent_param_weight in_features out_featuresoriginal_moduletmp_state_dictattribute_name new_value old_valuer@s rRr'FluxLoraLoaderMixin.unload_lora_weightss" #%BI$P]B^B^gd$9$9:dhdtdt ; : ; ; @d@d  ' ' (L(LUZ ' [37K 0 &7;@UW[+\+h!,!@!@ 5L0 &&y11 $$Z%7%7 2%FG1!, 9 9 ; fehhoo664;O$*MM$6$6M6&+>$/$=$=>P$QM+=g>N+O(0D0J0J10MOcOiOijkOlKf-*/((//'(!%"/"5"5 +:+.'/0D%EN".&--v7IFRW.7Y.Z[#33N4X\3]MP&*.PP)KL_)`$'(<(B(B1(E$F $+K,>,>$O  2 2NIN &~&66QR[Q\\bclbmmnoA!<,i &&.-s +K  K c 0nUbURU5 UbURU5 U=(d URn[UR55HBnUR S5SU:XdMUR U5XVR US35'MD Sn0n[USS5SLn [US5n UR5GHup[U [RR5(dM1U RRn U R bU R ROSnUSLnU (aU R#SS5OU nUS 3nUS 3nUU;aMUUR$S nUUR$SnUR'XS 9n[)U5UU4:XaMUunnSnUU:aUS U SUSU3- nUU:a USUSUS3- nOUS- nU(a[*R-U5 UU:d UU:dGMASnU R/S5unnnUR1U5nU (a [3X5n [R4"S5 [RRUUXR6S9nSSS5 [R8"WRRU R4U R6S9n[)SU55nU UU'SU0n UbUU S'UR;U SSS9 [=UUU5 A U[>;a[>Un![AURRR$S 5n"[URBU!5n#[=URBU!U"5 [*RESU!SU"SU#S35 XUS3'UcGMXUS3'GM [GU5S:aXl$U$!,(df  GNJ=f)z Control LoRA expands the shape of the input layer from (3072, 64) to (3072, 128). This method handles that and generalizes things a bit so that any parameter that needs expansion receives appropriate treatment. NrrF peft_config hf_quantizerz .base_layerr.lora_A.weight.lora_B.weightr)rK base_modulez:Expanding the nn.Linear input/output features for module="z" because the provided LoRA checkpoint contains higher number of features than expected. The number of input_features will be expanded from z to z:, and the number of output features will be expanded from Trr)rCr<c3:# UHn[SU5v M g7f)rN)slice)redims rRrgVFluxLoraLoaderMixin._maybe_expand_transformer_param_shape_or_error_..bs"P&+>$/$=$=>P$QM#(RS^(g f-*/((//'DH[H[+:+."'!1!1'..33MP&*.PP)KL_)`$'(>(>(C(C(I(I!(L$M $+K,>,>$O  2 2NIN &~&66QR[Q\\bclbmmnoKX*=)>g'FG".LW.A-B%+HIo8r ! "Q &.@ +**S.-s *O  O c[5nUR5nURS3nUVs/sH)ofRS5(dMUS[ S5*PM+ nnUVs/sH(oR U5(dMU[ U5SPM* nn[ [U55n[ UR5VV s1sHupUiM sn n5n [U5[U 5- n U (a[RSU S35 UGH3n X;aM U RUS5S3U;aU RUS5S3OU RUS5S3n XMnX%U S3nURXS9nUS URS :aURS URS 4n[R"UURS 9nUSS2SURS 24R!U5 UX%U S3'UR#U 5 GMUS URS :dGM[%S U S URS35e U(a&[R'SURSUS35 U$s snfs snfs sn nf)NrrzFound unexpected modules: z. These will be ignored.rz.base_layer.weightr)rKbase_weight_param_namerrrCzThis LoRA param (z*.lora_A.weight) has an incompatible shape zk. Please open an issue to file for a feature request - https://github.com/huggingface/diffusers/issues/new.zGThe following LoRA modules were zero padded to match the state dict of z: zn. Please open an issue if you think this was unexpected - https://github.com/huggingface/diffusers/issues/new.)rrr3rrrsortedrrrrrrrFzerosrCcopy_rNotImplementedErrorr)rr.rvexpanded_module_namesrrrflora_module_namesrrtransformer_module_namesunexpected_modulesrbase_param_namebase_weight_param lora_A_parambase_module_shaperexpanded_state_dict_weights rRr1FluxLoraLoaderMixin._maybe_expand_lora_state_dicts #!,!7!7!9(()+6E 5Dc UeHf )C(3'(( )_  >Oj=NTRaRabhRi0T#f+-0=Nj"3'8#9:#){?X?X?Z*[?ZGD4?Z*[#\  !23c:R6SS  LL56H5IIab c"A&ii+,,>?CYY99VR());< &"-.g6  !7 G *XaS+GHL!$ ; ;+ ; v  #l&8&8&;;%++A.0A0G0G0JK-2[[GXG_G_-`**1.E 0B0B10E.E+EFLL\Z@Z(1#^ <=%))!,"1% (:(:1(==)'s*TUaUgUgThiTU-#4 ! KKYZ]ZnZnYooqsHrIIwx S k*[sI7 I7$I<I<J rKztorch.nn.Modulerztorch.nn.Linearrz torch.Sizec"S[R4SjnUbU"UR5$UbXURS5(d[ SU<S35eUR SS5Sn[ X5nU"UR5$[ S5e) Nr?cURRS:XaURR$URRS:Xa UR$UR$)Nr5r7)r@rArIr quant_shape)r?s rR_get_weight_shapeFFluxLoraLoaderMixin._calculate_module_shape.._get_weight_shapesR((L8))///!!**o=)))||#rTrzaInvalid `base_weight_param_name` passed as it does not end with '.weight' base_weight_param_name=rrrzBEither `base_module` or `base_weight_param_name` must be provided.)rFrr?rrBrsplitr)rKrrr module_path submodules rRr+FluxLoraLoaderMixin._calculate_module_shapes  $ell $  "$[%7%78 8 # /)229== xaw`yyz{177 1EaHK-eAI$Y%5%56 6]^^rTcU/nU(aURU5 U(aURU5 U(dU(a [U5$U$r)appendr)rrnrrrkrsoutputss rRrz$FluxLoraLoaderMixin._prepare_outputss:,  NN6 "  NN8 $"/?uW~SSrTrc)FrNNNFFrr)NNN)NN)NFF))rArrrrrrHr3rr}rrrrrtrFrrrvrr2rr|rrrrrrrrrrrrr staticmethodrrzrrrs@rRrfrfs,^<')(\%$y/2T#u||:K5L/Lyyy|$( d /2T#u||:K5L/Ld Djd  d L    >  % 3- c5<< 3-3-j7 7 7 r NR?C $"&#'*.+/< bkk)< "&c588??U\\+I&I!J< #'sEHHOO';"< <  <  <  < !< < @"/!*.  I    Cy4'   @4A.2Q=d3i=="BH y+XX__y+  y+y+v..`*.&*_ _&_!$_  __4TTrTrfc2\rSrSrSS/r\r\r\ SS\ 4Sjj5r \ SS\ 4Sjj5r \ SS\ \R-S \\ \R$R&4S \\ \R$R&4S \ S \ S \S\ 4Sjj5rSrg)AmusedLoraLoaderMixinir.r,NrZc U(a[SS5(d [S5e[RSURS35 UR UUUUUUUS9 grr:rs rRr20AmusedLoraLoaderMixin.load_lora_into_transformersl _T8%D%DD   hs334A67%% )%/ & rTc >[UUUUUURUU UUU S9 grrrs rRr|1AmusedLoraLoaderMixin.load_lora_into_text_encoder rrTrrr?rrrrc &0nU(dU(d [S5eU(a*URURX0R55 U(a*URURX R55 UR UUUUUUS9 g)un Save the LoRA parameters corresponding to the UNet and text encoder. Arguments: save_directory (`str` or `os.PathLike`): Directory to save LoRA parameters to. Will be created if it doesn't exist. unet_lora_layers (`dict[str, torch.nn.Module]` or `dict[str, torch.Tensor]`): State dict of the LoRA layers corresponding to the `unet`. text_encoder_lora_layers (`dict[str, torch.nn.Module]` or `dict[str, torch.Tensor]`): State dict of the LoRA layers corresponding to the `text_encoder`. Must explicitly pass the text encoder LoRA state dict because it comes from 🤗 Transformers. is_main_process (`bool`, *optional*, defaults to `True`): Whether the process calling this is the main process or not. Useful during distributed training and you need to call this function on all processes. In this case, set `is_main_process=True` only on the main process to avoid race conditions. save_function (`Callable`): The function to use to save the state dictionary. Useful during distributed training when you need to replace `torch.save` with another method. Can be configured with the environment variable `DIFFUSERS_SAVE_MODE`. safe_serialization (`bool`, *optional*, defaults to `True`): Whether to save the model using `safetensors` or the traditional PyTorch way with `pickle`. r[)rrrrrrN)rBr pack_weightsr3r}write_lora_layers) rrrr?rrrrrs rRr'AmusedLoraLoaderMixin.save_lora_weights; sB '+Cuv v "   c../FH\H\] ^ #   c../GI^I^_ ` !)+#'1  rTrcr r)NNTNNT)rArrrrrHr3rr}rrr2r|rrrrtrFrrrrrrcrTrRr#r#s+^<')   >7 7 7 r@D>B $"&#'3 bkk)3 #'sEHHOO';"<3 "&c588??&:!; 3  3  3  3 !3 3 rTr#c^\rSrSrSrS/r\r\\ S\ \ \ \ R4-4Sj55rSS\ \ \ \ R4-S\ S-S \4S jjr\SS \4S jj5r\SS \ \R&-S \ \ \ R(R*\ R-4S\S\ S\S\S\ S-4Sjj5rS/SSS4S\\ S\S\S\\ S-4U4SjjjrS/4S\\ 4U4SjjjrSrU=r$)CogVideoXLoraLoaderMixinir z[ Load LoRA layers into [`CogVideoXTransformer3DModel`]. Specific to [`CogVideoXPipeline`]. r.rXc URSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n Sn U cS n S n SSS.n[UU U UUUUUUU UU S9 unn[SU55nU(aESn[R U5 UR 5VVs0sHunnSU;dMUU_M nnnU (aUU4nU$UnU$s snnf)r%rNrFrrrrrrrr_Trrrrc3,# UH nSU;v M g7frrcrs rRrg;CogVideoXLoraLoaderMixin.lora_state_dict.. rrjrrr(r)s rRrv(CogVideoXLoraLoaderMixin.lora_state_dictz rOr+NFrYrZc [(d [S5eURS[5nU(a[ SS5(a [S5e[ U[ 5(aUR5nSUS'UR"U40UD6upg[SUR555nU(d [S 5eURU[US 5(d[XR5O URUUUUUS 9 g ) rr\r]r-r.r^Tr_c3,# UH nSU;v M g7frarcrds rRrg=CogVideoXLoraLoaderMixin.load_lora_weights.. rirjrkr.r1NrSr4s rRr*CogVideoXLoraLoaderMixin.load_lora_weights rUrTc U(a[SS5(a [S5e[RSURS35 UR USUUUUUS9 gr7r:r;s rRr23CogVideoXLoraLoaderMixin.load_lora_into_transformer rXrTrr?rrrrrBc 0n0n U(aX(UR'XyUR'U(d [S5eURUUU UUUUS9 grZrAr\s rRr*CogVideoXLoraLoaderMixin.save_lora_weights sd  "0G,, -2S#.. /uv v )#'+#'1  rTrrrprrc 0>[TU]"SUUUUS.UD6 grrrs rRr"CogVideoXLoraLoaderMixin.fuse_lora rrTc *>[TU]"SSU0UD6 grrrs rRr$CogVideoXLoraLoaderMixin.unfuse_lora/ r rTrcrrrcrdrs@rRr-r-r s,_'2/2T#u||:K5L/L22n$( ' /2T#u||:K5L/L' Dj'  ' R    <NR $"&#'9= bkk) "&c588??U\\+I&I!J       ! ,0$;  F"/!*.  I    Cy4'   &4A/=d3i==rTr-c^\rSrSrSrS/r\r\\ S\ \ \ \ R4-4Sj55rSS\ \ \ \ R4-S\ S-S \4S jjr\SS \4S jj5r\SS \ \R&-S \ \ \ R(R*\ R-4S\S\ S\S\S\ S-4Sjj5rS/SSS4S\\ S\S\S\\ S-4U4SjjjrS/4S\\ 4U4SjjjrSrU=r$)Mochi1LoraLoaderMixini6 zS Load LoRA layers into [`MochiTransformer3DModel`]. Specific to [`MochiPipeline`]. r.rXc URSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n Sn U cS n S n SSS.n[UU U UUUUUUU UU S9 unn[SU55nU(aESn[R U5 UR 5VVs0sHunnSU;dMUU_M nnnU (aUU4nU$UnU$s snnf)r%rNrFrrrrrrrr_Trrrrc3,# UH nSU;v M g7frrcrs rRrg8Mochi1LoraLoaderMixin.lora_state_dict..l rrjrrr(r)s rRrv%Mochi1LoraLoaderMixin.lora_state_dict> rOr+NFrYrZc [(d [S5eURS[5nU(a[ SS5(a [S5e[ U[ 5(aUR5nSUS'UR"U40UD6upg[SUR555nU(d [S 5eURU[US 5(d[XR5O URUUUUUS 9 g ) rr\r]r-r.r^Tr_c3,# UH nSU;v M g7frarcrds rRrg:Mochi1LoraLoaderMixin.load_lora_weights.. rirjrkr.r1NrSr4s rRr'Mochi1LoraLoaderMixin.load_lora_weightsv rUrTc U(a[SS5(a [S5e[RSURS35 UR USUUUUUS9 gr7r:r;s rRr20Mochi1LoraLoaderMixin.load_lora_into_transformer rXrTrr?rrrrrBc 0n0n U(aX(UR'XyUR'U(d [S5eURUUU UUUUS9 grZrAr\s rRr'Mochi1LoraLoaderMixin.save_lora_weights r^rTrrrprrc 0>[TU]"SUUUUS.UD6 grrrs rRrMochi1LoraLoaderMixin.fuse_lora rrTc *>[TU]"SSU0UD6 grrrs rRr!Mochi1LoraLoaderMixin.unfuse_lora r rTrcrrrcrdrs@rRr?r?6 ,_'2/2T#u||:K5L/L22p$( ' /2T#u||:K5L/L' Dj'  ' R    < NR $"&#'9= bkk) "&c588??U\\+I&I!J       ! ,0$;  H"/!*.  I    Cy4'   (4A/=d3i==rTr?c^\rSrSrSrS/r\r\\ S\ \ \ \ R4-4Sj55rSS\ \ \ \ R4-S\ S-S \4S jjr\SS \4S jj5r\SS \ \R&-S \ \ \ R(R*\ R-4S\S\ S\S\S\ S-4Sjj5rS/SSS4S\\ S\S\S\\ S-4U4SjjjrS/4S\\ 4U4SjjjrSrU=r$)LTXVideoLoraLoaderMixini zT Load LoRA layers into [`LTXVideoTransformer3DModel`]. Specific to [`LTXPipeline`]. r.rXc URSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n Sn U cS n S n SSS.n[UU U UUUUUUU UU S9 unn[SU55nU(aESn[R U5 UR 5VVs0sHunnSU;dMUU_M nnn[SU55nU(a [ U5nU (aUU4nU$UnU$s snnf)r%rNrFrrrrrrrr_Trrrrc3,# UH nSU;v M g7frrcrs rRrg:LTXVideoLoraLoaderMixin.lora_state_dict..3 rrjrrc3B# UHoRS5v M g7fdiffusion_model.Nrrs rRrgrU9 %[PZ1ll3E&F&FPZ)rqrrrrrr"rrXrrrrrrrrrrr_rrrrnrrrris_non_diffusers_formatrs rRrv'LTXVideoLoraLoaderMixin.lora_state_dict sJJ{D1 $4e<**Y-!::&8$? 7D)::j$/JJ{D1 jj5  **%6=%zz*@%H  ""OL#7iP 02W#+-)!%  H!$#Jz#J J {H NN8 $+5+;+;+=W+=41aUVAV$!Q$+=JW"%%[PZ%["[ "FzRJ(.\ rirjrkr.r1NrSr4s rRr)LTXVideoLoraLoaderMixin.load_lora_weightsA rUrTc U(a[SS5(a [S5e[RSURS35 UR USUUUUUS9 gr7r:r;s rRr22LTXVideoLoraLoaderMixin.load_lora_into_transformerj rXrTrr?rrrrrBc 0n0n U(aX(UR'XyUR'U(d [S5eURUUU UUUUS9 grZrAr\s rRr)LTXVideoLoraLoaderMixin.save_lora_weights r^rTrrrprrc 0>[TU]"SUUUUS.UD6 grrrs rRr!LTXVideoLoraLoaderMixin.fuse_lora rrTc *>[TU]"SSU0UD6 grrrs rRr#LTXVideoLoraLoaderMixin.unfuse_lora r rTrcrrrcrdrs@rRrRrR ,_'6/2T#u||:K5L/L66x$( ' /2T#u||:K5L/L' Dj'  ' R    < NR $"&#'9= bkk) "&c588??U\\+I&I!J       ! ,0$;  H"/!*.  I    Cy4'   (4A/=d3i==rTrRc^\rSrSrSrSS/r\r\r \ \ S\ \ \ \R4-4Sj55rSS\ \ \ \R4-S \ S-S \4S jjr\ SS \S \ 4S jj5r\ S S\ \R*-S\ \ \R,R.\R-4S\S\ S\S\S\ S-4Sjj5rS/SSS4S\\ S\S\S\\ S-4U4SjjjrS/4S\\ 4U4SjjjrSrU=r$)!LTX2LoraLoaderMixini zV Load LoRA layers into [`LTX2VideoTransformer3DModel`]. Specific to [`LTX2Pipeline`]. r.r/rXc VURSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n Sn U cS n S n SSS.n[UU U UUUUUUU UU S9 unn[SU55nU(aESn[R U5 UR 5VVs0sHunnSU;dMUU_M nnnUn[SU55n[SU55nU(a [ U5nU(a[ US5nURU5 U (aUU4nU$UnU$s snnf)r%rNrFrrrrrrrr_Trrrrc3,# UH nSU;v M g7frrcrs rRrg6LTX2LoraLoaderMixin.lora_state_dict.. rrjrrc3B# UHoRS5v M g7frWrrs rRrgrp rYrZc3B# UHoRS5v M g7f)ztext_embedding_projection.Nrrs rRrgrp sZ1LL)EFFPZrZtext_embedding_projection)rqrrrrrr!r)rrXrrrrrrrrrrr_rrrrnrrrrfinal_state_dictr\ has_connectorconnectors_state_dictrs rRrv#LTX2LoraLoaderMixin.lora_state_dict sJJ{D1 $4e<**Y-!::&8$? 7D)::j$/JJ{D1 jj5  **%6=%zz*@%H  ""OL#7iP 02W#+-)!%  H!$#Jz#J J {H NN8 $+5+;+;+=W+=41aUVAV$!Q$+=JW%"%%[PZ%["[Z[[ "LZX  $Q7% !  # #$9 :.B* IY Xs  F%.F%NFrYrZc [(d [S5eURS[5nU(a[ SS5(a [S5e[ U[ 5(aUR5nSUS'UR"U40UD6upg[SUR555nU(d [S 5eUR5V V s0sH,upU RURS 35(dM*X_M. n n n UR5V V s0sH,upU RURS 35(dM*X_M. n n n URU [!US 5(d[#XR5O UR$UUUUUS 9 U (aQURU [!US 5(d[#XR5O UR&UUUUUURS9 ggs sn n fs sn n f)rr\r]r-r.r^Tr_c3,# UH nSU;v M g7frarcrds rRrg8LTX2LoraLoaderMixin.load_lora_weights..- rirjrkrr.r1r/)r.rYrnror]rZrN)rrBrqrrr rsrtrurvrwrxrrr3connectors_namer2rErzr.r/) rrXrYrZrr]rrnrrrtransformer_peft_state_dictconnectors_peft_state_dicts rRr%LTX2LoraLoaderMixin.load_lora_weights s HI I"JJ':<[\ h!?!?D  ;T B B4Y4^4^4` 1*.%&#334Yd]cd K9JKK rs s(--/' /TQ1<<4CXCXBYYZ@[3\DAD/ $' 8B7G7G7I%v7ItqQ\\]a]q]q\rrsZtMudad7I"%v '' 'DKDR_D`D`&;&;*>?__)!"3++ ,  &' &ws)G1G1)G7 G7rc U(a[SS5(a [S5e[RSUS35 UR USUUUUUUS9 g)r8r-r.r^rrN)rmrYrnror]rZr)r rBrrr) rrr.rYror]rZrnrs rRr2.LTX2LoraLoaderMixin.load_lora_into_transformerL sh h!?!?D   hvha()%% %/ & rTrr?rrrrrBc 0n0n U(aX(UR'XyUR'U(d [S5eURUUU UUUUS9 grZrAr\s rRr%LTX2LoraLoaderMixin.save_lora_weightsm r^rTrrrprrc 0>[TU]"SUUUUS.UD6 grrrs rRrLTX2LoraLoaderMixin.fuse_lora rrTc *>[TU]"SSU0UD6 grrrs rRrLTX2LoraLoaderMixin.unfuse_lora r rTrcr)NNFFNr.rc) rArrrrrrHr3LTX2_CONNECTOR_NAMEr{rrrrtrFrrvrrr2rrrrrrrrrrrrrs@rRrmrm s,\:')O</2T#u||:K5L/L<<B$( 8/2T#u||:K5L/L8Dj8 8t #    @ NR $"&#'9= bkk) "&c588??U\\+I&I!J       ! ,0$;  H"/!*.  I    Cy4'   (4A/=d3i==rTrmc^\rSrSrSrS/r\r\\ S\ \ \ \ R4-4Sj55rSS\ \ \ \ R4-S\ S-S \4S jjr\SS \4S jj5r\SS \ \R&-S \ \ \ R(R*\ R-4S\S\ S\S\S\ S-4Sjj5rS/SSS4S\\ S\S\S\\ S-4U4SjjjrS/4S\\ 4U4SjjjrSrU=r$)SanaLoraLoaderMixini zQ Load LoRA layers into [`SanaTransformer2DModel`]. Specific to [`SanaPipeline`]. r.rXc URSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n Sn U cS n S n SSS.n[UU U UUUUUUU UU S9 unn[SU55nU(aESn[R U5 UR 5VVs0sHunnSU;dMUU_M nnnU (aUU4nU$UnU$s snnf)r%rNrFrrrrrrrr_Trrrrc3,# UH nSU;v M g7frrcrs rRrg6SanaLoraLoaderMixin.lora_state_dict.. rrjrrr(r)s rRrv#SanaLoraLoaderMixin.lora_state_dict rOr+NFrYrZc [(d [S5eURS[5nU(a[ SS5(a [S5e[ U[ 5(aUR5nSUS'UR"U40UD6upg[SUR555nU(d [S 5eURU[US 5(d[XR5O URUUUUUS 9 g ) rr\r]r-r.r^Tr_c3,# UH nSU;v M g7frarcrds rRrg8SanaLoraLoaderMixin.load_lora_weights.. rirjrkr.r1NrSr4s rRr%SanaLoraLoaderMixin.load_lora_weights rUrTc U(a[SS5(a [S5e[RSURS35 UR USUUUUUS9 gr7r:r;s rRr2.SanaLoraLoaderMixin.load_lora_into_transformer rXrTrr?rrrrrBc 0n0n U(aX(UR'XyUR'U(d [S5eURUUU UUUUS9 grZrAr\s rRr%SanaLoraLoaderMixin.save_lora_weights5 r^rTrrrprrc 0>[TU]"SUUUUS.UD6 grrrs rRrSanaLoraLoaderMixin.fuse_loraY rrTc *>[TU]"SSU0UD6 grrrs rRrSanaLoraLoaderMixin.unfuse_loram r rTrcrrrcrdrs@rRrr rPrTrc^\rSrSrSrS/r\r\\ S\ \ \ \ R4-4Sj55rSS\ \ \ \ R4-S\ S-S \4S jjr\SS \4S jj5r\SS \ \R&-S \ \ \ R(R*\ R-4S\S\ S\S\S\ S-4Sjj5rS/SSS4S\\ S\S\S\\ S-4U4SjjjrS/4S\\ 4U4SjjjrSrU=r$)HeliosLoraLoaderMixinit zs Load LoRA layers into [`HeliosTransformer3DModel`]. Specific to [`HeliosPipeline`] and [`HeliosPyramidPipeline`]. r.rXc (URSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n Sn U cS n S n SSS.n[UU U UUUUUUU UU S9 unn[SU55(a [U5nO"[SU55(a [ U5n[SU55nU(aESn[ R U5 UR5VVs0sHunnSU;dMUU_M nnnU (aUU4nU$UnU$s snnf)r%rNrFrrrrrrrr_Trrrrc3B# UHoRS5v M g7frWrrs rRrg8HeliosLoraLoaderMixin.lora_state_dict.. DA||.//rZc3B# UHoRS5v M g7frNrrs rRrgr @Zl++ZrZc3,# UH nSU;v M g7frrcrs rRrgr rrjrrrqrrr%rrrrr)s rRrv%HeliosLoraLoaderMixin.lora_state_dict| JJ{D1 $4e<**Y-!::&8$? 7D)::j$/JJ{D1 jj5  **%6=%zz*@%H  ""OL#7iP 02W#+-)!%  H DD D DEjQJ @Z@ @ @>zJJ ##Jz#J J {H NN8 $+5+;+;+=W+=41aUVAV$!Q$+=JW(. rirjrkr.r1NrSr4s rRr'HeliosLoraLoaderMixin.load_lora_weights s HI I"JJ':<[\ h!?!?D  ;T B B4Y4^4^4` 1*.%&#334Yd]cd K9JKK rs s '' DKDR_D`D`&;&;[TU]"SUUUUS.UD6 grrrs rRrHeliosLoraLoaderMixin.fuse_lora"rrTc *>[TU]"SSU0UD6 grrrs rRr!HeliosLoraLoaderMixin.unfuse_lora6r rTrcrrrcrdrs@rRrrt s,_'6/2T#u||:K5L/L66v$( & /2T#u||:K5L/L& Dj&  & P    < NR $"&#'9= bkk) "&c588??U\\+I&I!J       ! ,0$;  H"/!*.  I    Cy4'   (4A/=d3i==rTrc^\rSrSrSrS/r\r\\ S\ \ \ \ R4-4Sj55rSS\ \ \ \ R4-S\ S-S \4S jjr\SS \4S jj5r\SS \ \R&-S \ \ \ R(R*\ R-4S\S\ S\S\S\ S-4Sjj5rS/SSS4S\\ S\S\S\\ S-4U4SjjjrS/4S\\ 4U4SjjjrSrU=r$)HunyuanVideoLoraLoaderMixini=za Load LoRA layers into [`HunyuanVideoTransformer3DModel`]. Specific to [`HunyuanVideoPipeline`]. r.rXc URSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n Sn U cS n S n SSS.n[UU U UUUUUUU UU S9 unn[SU55nU(aESn[R U5 UR 5VVs0sHunnSU;dMUU_M nnn[SU55nU(a [ U5nU (aUU4nU$UnU$s snnf)r%rNrFrrrrrrrr_Trrrrc3,# UH nSU;v M g7frrcrs rRrg>HunyuanVideoLoraLoaderMixin.lora_state_dict..rrrjrrc3,# UH nSU;v M g7f) img_attn_qkvNrcrs rRrgrxs'PZ!(;Zrj)rqrrrrrr)rrXrrrrrrrrrrr_rrrrnrrrris_original_hunyuan_videors rRrv+HunyuanVideoLoraLoaderMixin.lora_state_dictEsJJ{D1 $4e<**Y-!::&8$? 7D)::j$/JJ{D1 jj5  **%6=%zz*@%H  ""OL#7iP 02W#+-)!%  H!$#Jz#J J {H NN8 $+5+;+;+=W+=41aUVAV$!Q$+=JW$''PZ'P$P! $A*MJ(.rirjrkr.r1NrSr4s rRr-HunyuanVideoLoraLoaderMixin.load_lora_weightsrUrTc U(a[SS5(a [S5e[RSURS35 UR USUUUUUS9 gr7r:r;s rRr26HunyuanVideoLoraLoaderMixin.load_lora_into_transformerrXrTrr?rrrrrBc 0n0n U(aX(UR'XyUR'U(d [S5eURUUU UUUUS9 grZrAr\s rRr-HunyuanVideoLoraLoaderMixin.save_lora_weightsr^rTrrrprrc 0>[TU]"SUUUUS.UD6 grrrs rRr%HunyuanVideoLoraLoaderMixin.fuse_lorarrTc *>[TU]"SSU0UD6 grrrs rRr'HunyuanVideoLoraLoaderMixin.unfuse_lorar rTrcrrrcrdrs@rRrr=rkrTrc^\rSrSrSrS/r\r\\ S\ \ \ \ R4-4Sj55rSS\ \ \ \ R4-S\ S-S \4S jjr\SS \4S jj5r\SS \ \R&-S \ \ \ R(R*\ R-4S\S\ S\S\S\ S-4Sjj5rS/SSS4S\\ S\S\S\\ S-4U4SjjjrS/4S\\ 4U4SjjjrSrU=r$)Lumina2LoraLoaderMixiniz_ Load LoRA layers into [`Lumina2Transformer2DModel`]. Specific to [`Lumina2Text2ImgPipeline`]. r.rXc URSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n Sn U cS n S n SSS.n[UU U UUUUUUU UU S9 unn[SU55nU(aESn[R U5 UR 5VVs0sHunnSU;dMUU_M nnn[SU55nU(a [ U5nU (aUU4nU$UnU$s snnf)r%rNrFrrrrrrrr_Trrrrc3,# UH nSU;v M g7frrcrs rRrg9Lumina2LoraLoaderMixin.lora_state_dict..=rrjrrc3B# UHoRS5v M g7frWrrs rRrgrDQjLL);<.grirjrkr.r1NrSr4s rRr(Lumina2LoraLoaderMixin.load_lora_weightsLrUrTc U(a[SS5(a [S5e[RSURS35 UR USUUUUUS9 gr7r:r;s rRr21Lumina2LoraLoaderMixin.load_lora_into_transformerurXrTrr?rrrrrBc 0n0n U(aX(UR'XyUR'U(d [S5eURUUU UUUUS9 grZrAr\s rRr(Lumina2LoraLoaderMixin.save_lora_weightsr^rTrrrprrc 0>[TU]"SUUUUS.UD6 grrrs rRr Lumina2LoraLoaderMixin.fuse_lorarrTc *>[TU]"SSU0UD6 grrrs rRr"Lumina2LoraLoaderMixin.unfuse_lorar rTrcrrrcrdrs@rRrrs,_'7/2T#u||:K5L/L77z$( ' /2T#u||:K5L/L' Dj'  ' R    < NR $"&#'9= bkk) "&c588??U\\+I&I!J       ! ,0$;  H"/!*.  I    Cy4'   (4A/=d3i==rTrc^\rSrSrSrS/r\r\\ S\ \ \ \ R4-4Sj55rSS\ \ \ \ R4-S\ S-S \4S jjr\SS \4S jj5r\SS \ \R&-S \ \ \ R(R*\ R-4S\S\ S\S\S\ S-4Sjj5rS/SSS4S\\ S\S\S\\ S-4U4SjjjrS/4S\\ 4U4SjjjrSrU=r$)KandinskyLoraLoaderMixiniz9 Load LoRA layers into [`Kandinsky5Transformer3DModel`], r.rXc URSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n Sn U cS n S n SSS.n[UU U UUUUUUU UU S9 unn[SU55nU(aESn[R U5 UR 5VVs0sHunnSU;dMUU_M nnnU (aUU4nU$UnU$s snnf)r%rNrFrrrrrrrr_Trrrrc3,# UH nSU;v M g7frrcrs rRrg;KandinskyLoraLoaderMixin.lora_state_dict.. rrjrrr(r)s rRrv(KandinskyLoraLoaderMixin.lora_state_dictrOr+NFrYrZc [(d [S5eURS[5nU(a[ SS5(a [S5e[ U[ 5(aUR5nSUS'UR"U40UD6upg[SUR555nU(d [S 5eURU[US 5(d[XR5O URUUUUUS 9 g ) rr\r]r-r.r^Tr_c3,# UH nSU;v M g7frarcrds rRrg=KandinskyLoraLoaderMixin.load_lora_weights../rirjrkr.r1NrSr4s rRr*KandinskyLoraLoaderMixin.load_lora_weightsrUrTc U(a[SS5(a [S5e[RSURS35 UR USUUUUUS9 gr7r:r;s rRr23KandinskyLoraLoaderMixin.load_lora_into_transformer=rXrTrr?rrrrrBc 0n0n U(aX(UR'XyUR'U(d [S5eURUUU UUUUS9 grZrAr\s rRr*KandinskyLoraLoaderMixin.save_lora_weights]r^rTrrrprrc 0>[TU]"SUUUUS.UD6 grrrs rRr"KandinskyLoraLoaderMixin.fuse_lorarrTc *>[TU]"SSU0UD6 grrrs rRr$KandinskyLoraLoaderMixin.unfuse_lorar rTrcrrrcrdrs@rRrrrPrTrc.^\rSrSrSrSS/r\r\\ S\ \ \ \ R4-4Sj55r\S\ RR 4Sj5rSS\ \ \ \ R4-S \ S-S \4S jjr\SS \4S jj5r\S S\ \R,-S\ \ \ RR \ R-4S\S\ S\S\S\ S-4Sjj5rS/SS S4S\\ S\S\S\\ S-4U4SjjjrS/4S\\ 4U4SjjjrSrU=r$)!WanLoraLoaderMixinizo Load LoRA layers into [`WanTransformer3DModel`]. Specific to [`WanPipeline`] and `[WanImageToVideoPipeline`]. r. transformer_2rXc (URSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n Sn U cS n S n SSS.n[UU U UUUUUUU UU S9 unn[SU55(a [U5nO"[SU55(a [ U5n[SU55nU(aESn[ R U5 UR5VVs0sHunnSU;dMUU_M nnnU (aUU4nU$UnU$s snnf)r%rNrFrrrrrrrr_Trrrrc3B# UHoRS5v M g7frWrrs rRrg5WanLoraLoaderMixin.lora_state_dict..rrZc3B# UHoRS5v M g7frrrs rRrgrrrZc3,# UH nSU;v M g7frrcrs rRrgrrrjrrrr)s rRrv"WanLoraLoaderMixin.lora_state_dictrrc 6URRcU$URn[SU55(GaV[ UVs1sH1nSU;dM UR S5SR S5SiM3 sn5n[SU55=(a [SU55n[SU55nU(aU$[ U5Hn[S S /S S /5HupS US3n S US3n X;dX;aM[R"US US3US9US USU S3'[R"US US3US9US USU S3'S US3n U(dMxX;dMX-n[R"UUS9US USU S3'M M U$s snf)Nc3B# UHoRS5v M g7ftransformer.blocks.Nrrs rRrgDWanLoraLoaderMixin._maybe_expand_t2v_lora_for_i2v..GJq||122JrZblocks.rrrc3,# UH nSU;v M g7f add_k_projNrcrs rRrgrDAla/rjc3,# UH nSU;v M g7f add_v_projNrcrs rRrgrLshrcd\]^M^hrrjc3,# UH nSU;v M g7f .lora_B.biasNrcrs rRrgrC 1>Q. rjk_imgv_imgrrr.attn2.to_k.lora_A.weight.attn2.to_k.lora_B.weightr.attn2.rr.attn2.to_k.lora_B.biasr  r image_dimrCrrrrangeziprFrrr.r target_devicer num_blocks is_i2v_lorahas_biasiocref_key_lora_Aref_key_lora_Bref_key_lora_B_biasref_lora_B_bias_tensors rR_maybe_expand_t2v_lora_for_i2v1WanLoraLoaderMixin._maybe_expand_t2v_lora_for_i2vs    ' ' / #** GJG G G:h:aYbfgYgAaggi0399#>qA:hiJDDDsLshrLsIsKC CCH!!:&' 2\<4PQDA':1#=V%WN':1#=V%WN%7>;[ TYTdTd"%8;T#UV_lUJ!4QCwqcPQUZTdTd"%8;T#UV_lUJ!4QCwqcPQ-@sBY*Z'x$7$E1;1P.V[VfVf2#0W %871#\#RS+R'6Ei  F*FNFrYrZc f[(d [S5eURS[5nU(a[ SS5(a [S5e[ U[ 5(aUR5nSUS'UR"U40UD6upgUR[US5(d[XR5O URUS 9n[S UR555nU(d [S 5eURS S 5n U (aS[US5(d"[!S[#U5R$S35eUR'UUR(UUUUUS9 gUR'U[US5(d[XR5O URUUUUUS9 g)rr\r]r-r.r^Tr_r.r.rc3,# UH nSU;v M g7frarcrds rRrg7WanLoraLoaderMixin.load_lora_weights..-rirjrkload_into_transformer_2Fr'' object has no attribute transformer_2Note that Wan2.1 models do not have a transformer_2 component.Ensure the model has a transformer_2 component before setting load_into_transformer_2=True.r1NrrBrqrrr rsrtrurvr"rErzr3r.rwrxAttributeErrorrDrAr2r rrXrYrZrr]rrnrr)s rRr$WanLoraLoaderMixin.load_lora_weights HI I"JJ':<[\ h!?!?D  ;T B B4Y4^4^4` 1*.%&#334Yd]cd 88DKDR_D`D`&;&;[TU]"SUUUUS.UD6 grrrs rRrWanLoraLoaderMixin.fuse_lorarrTc *>[TU]"SSU0UD6 grrrs rRrWanLoraLoaderMixin.unfuse_lorar rTrcrrrcrArrrrrrHr3rrrrtrFrrvrrr"rrr2rrrrrrrrrrrs@rRrrs,_='6/2T#u||:K5L/L66p-XX__--d$( ?/2T#u||:K5L/L?Dj? ?B    < NR $"&#'9= bkk) "&c588??U\\+I&I!J       ! ,0$;  H"/!*.  I    Cy4'   (4A/=d3i==rTrc,^\rSrSrSrS/r\r\\ S\ \ \ \ R4-4Sj55r\S\ RR 4Sj5rSS\ \ \ \ R4-S \ S-S \4S jjr\SS \4S jj5r\SS \ \R,-S\ \ \ RR \ R-4S\S\ S\S\S\ S-4Sjj5rS/SSS4S\\ S\S\S\\ S-4U4SjjjrS/4S\\ 4U4SjjjrSrU=r$) SkyReelsV2LoraLoaderMixiniz9 Load LoRA layers into [`SkyReelsV2Transformer3DModel`]. r.rXc (URSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n Sn U cS n S n SSS.n[UU U UUUUUUU UU S9 unn[SU55(a [U5nO"[SU55(a [ U5n[SU55nU(aESn[ R U5 UR5VVs0sHunnSU;dMUU_M nnnU (aUU4nU$UnU$s snnf)r%rNrFrrrrrrrr_Trrrrc3B# UHoRS5v M g7frWrrs rRrg.rrZc3B# UHoRS5v M g7frrrs rRrgr>rrZc3,# UH nSU;v M g7frrcrs rRrgr>rrjrrrr)s rRrv)SkyReelsV2LoraLoaderMixin.lora_state_dictsJJ{D1 $4e<**Y-!::&8$? 7D)::j$/JJ{D1 jj5  **%6=%zz*@%H  ""OL#7iP 02W#+-)!%  H DD D DEjQJ @Z@ @ @>zJJ ##Jz#J J {H NN8 $+5+;+;+=W+=41aUVAV$!Q$+=JW(.rrZrrrrc3,# UH nSU;v M g7frrcrs rRrgrDrrjc3,# UH nSU;v M g7frrcrs rRrgrDrrjc3,# UH nSU;v M g7fr rcrs rRrgrDr rjr r rrrrrrrrrrr rrs rRr"8SkyReelsV2LoraLoaderMixin._maybe_expand_t2v_lora_for_i2vs    ' ' / #** GJG G G:h:aYbfgYgAaggi0399#>qA:hiJDDDsLshrLsIsKC CCH!!:&' 2\<4PQDA':1#=V%WN':1#=V%WN%7>;[ TYTdTd"%8;T#UV_lUJ!4QCwqcPQUZTdTd"%8;T#UV_lUJ!4QCwqcPQ-@sBY*Z'x$7$E1;1P.V[VfVf2#0W %871#\#RS+R'6Eir$NFrYrZc f[(d [S5eURS[5nU(a[ SS5(a [S5e[ U[ 5(aUR5nSUS'UR"U40UD6upgUR[US5(d[XR5O URUS 9n[S UR555nU(d [S 5eURS S 5n U (aS[US5(d"[!S[#U5R$S35eUR'UUR(UUUUUS9 gUR'U[US5(d[XR5O URUUUUUS9 g)rr\r]r-r.r^Tr_r.r&c3,# UH nSU;v M g7frarcrds rRrg>SkyReelsV2LoraLoaderMixin.load_lora_weights..Brirjrkr)Frr*r+r1Nr,r.s rRr+SkyReelsV2LoraLoaderMixin.load_lora_weights#r0rTc U(a[SS5(a [S5e[RSURS35 UR USUUUUUS9 gr7r:r;s rRr24SkyReelsV2LoraLoaderMixin.load_lora_into_transformerdrXrTrr?rrrrrBc 0n0n U(aX(UR'XyUR'U(d [S5eURUUU UUUUS9 grZrAr\s rRr+SkyReelsV2LoraLoaderMixin.save_lora_weightsr^rTrrrprrc 0>[TU]"SUUUUS.UD6 grrrs rRr#SkyReelsV2LoraLoaderMixin.fuse_lorarrTc *>[TU]"SSU0UD6 grrrs rRr%SkyReelsV2LoraLoaderMixin.unfuse_lorar rTrcrrrcr9rs@rRr;r;s,_'6/2T#u||:K5L/L66p-XX__--f$( ?/2T#u||:K5L/L?Dj? ?B    < NR $"&#'9= bkk) "&c588??U\\+I&I!J       ! ,0$;  H"/!*.  I    Cy4'   (4A/=d3i==rTr;c^\rSrSrSrS/r\r\\ S\ \ \ \ R4-4Sj55rSS\ \ \ \ R4-S\ S-S \4S jjr\SS \4S jj5r\SS \ \R&-S \ \ \ R(R*\ R-4S\S\ S\S\S\ S-4Sjj5rS/SSS4S\\ S\S\S\\ S-4U4SjjjrS/4S\\ 4U4SjjjrSrU=r$)CogView4LoraLoaderMixinizT Load LoRA layers into [`WanTransformer3DModel`]. Specific to [`CogView4Pipeline`]. r.rXc URSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n Sn U cS n S n SSS.n[UU U UUUUUUU UU S9 unn[SU55nU(aESn[R U5 UR 5VVs0sHunnSU;dMUU_M nnnU (aUU4nU$UnU$s snnf)r%rNrFrrrrrrrr_Trrrrc3,# UH nSU;v M g7frrcrs rRrg:CogView4LoraLoaderMixin.lora_state_dict..rrjrrr(r)s rRrv'CogView4LoraLoaderMixin.lora_state_dictrOr+NFrYrZc [(d [S5eURS[5nU(a[ SS5(a [S5e[ U[ 5(aUR5nSUS'UR"U40UD6upg[SUR555nU(d [S 5eURU[US 5(d[XR5O URUUUUUS 9 g ) rr\r]r-r.r^Tr_c3,# UH nSU;v M g7frarcrds rRrg.rirjrkr.r1NrSr4s rRr)CogView4LoraLoaderMixin.load_lora_weightsrUrTc U(a[SS5(a [S5e[RSURS35 UR USUUUUUS9 gr7r:r;s rRr22CogView4LoraLoaderMixin.load_lora_into_transformer,rXrTrr?rrrrrBc 0n0n U(aX(UR'XyUR'U(d [S5eURUUU UUUUS9 grZrAr\s rRr)CogView4LoraLoaderMixin.save_lora_weightsLr^rTrrrprrc 0>[TU]"SUUUUS.UD6 grrrs rRr!CogView4LoraLoaderMixin.fuse_loraprrTc *>[TU]"SSU0UD6 grrrs rRr#CogView4LoraLoaderMixin.unfuse_lorar rTrcrrrcrdrs@rRrVrVrPrTrVc^\rSrSrSrS/r\r\\ S\ \ \ \ R4-4Sj55rSS\ \ \ \ R4-S\ S-S \4S jjr\SS \4S jj5r\SS \ \R&-S \ \ \ R(R*\ R-4S\S\ S\S\S\ S-4Sjj5rS/SSS4S\\ S\S\S\\ S-4U4SjjjrS/4S\\ 4U4SjjjrSrU=r$)HiDreamImageLoraLoaderMixiniza Load LoRA layers into [`HiDreamImageTransformer2DModel`]. Specific to [`HiDreamImagePipeline`]. r.rXc URSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n Sn U cS n S n SSS.n[UU U UUUUUUU UU S9 unn[SU55nU(aESn[R U5 UR 5VVs0sHunnSU;dMUU_M nnn[SU55nU(a [ U5nU (aUU4nU$UnU$s snnf)r%rNrFrrrrrrrr_Trrrrc3,# UH nSU;v M g7frrcrs rRrg>HiDreamImageLoraLoaderMixin.lora_state_dict..rrjrrc3,# UH nSU;v M g7f)diffusion_modelNrcrs rRrgrks%Qj&71&.rirjrkr.r1NrSr4s rRr-HiDreamImageLoraLoaderMixin.load_lora_weightsrUrTc U(a[SS5(a [S5e[RSURS35 UR USUUUUUS9 gr7r:r;s rRr26HiDreamImageLoraLoaderMixin.load_lora_into_transformerrXrTrr?rrrrrBc 0n0n U(aX(UR'XyUR'U(d [S5eURUUU UUUUS9 grZrAr\s rRr-HiDreamImageLoraLoaderMixin.save_lora_weightsr^rTrrrprrc 0>[TU]"SUUUUS.UD6 grrrs rRr%HiDreamImageLoraLoaderMixin.fuse_lora;rrTc *>[TU]"SSU0UD6 grrrs rRr'HiDreamImageLoraLoaderMixin.unfuse_loraOr rTrcrrrcrdrs@rRrhrhrkrTrhc^\rSrSrSrS/r\r\\ S\ \ \ \ R4-4Sj55rSS\ \ \ \ R4-S\ S-S \4S jjr\SS \4S jj5r\SS \ \R&-S \ \ \ R(R*\ R-4S\S\ S\S\S\ S-4Sjj5rS/SSS4S\\ S\S\S\\ S-4U4SjjjrS/4S\\ 4U4SjjjrSrU=r$)QwenImageLoraLoaderMixiniVz[ Load LoRA layers into [`QwenImageTransformer2DModel`]. Specific to [`QwenImagePipeline`]. r.rXc |URSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n Sn U cS n S n SSS.n[UU U UUUUUUU UU S9 unn[SU55nU(aESn[R U5 UR 5VVs0sHunnSU;dMUU_M nnn[SU55n[SU55n[SU55n[SU55nU(dU(dU(dU(a [ U5nU (aUU4nU$UnU$s snnf)r%rNrFrrrrrrrr_Trrrrc3,# UH nSU;v M g7frrcrs rRrg;QwenImageLoraLoaderMixin.lora_state_dict..rrjrrc3B# UHoRS5v M g7fz.alphaNrrs rRrgrHZzz(33ZrZc3B# UHoRS5v M g7frrrs rRrgrK 1LL66 rZc3B# UHoRS5v M g7frWrrs rRrgr!WJq,,/A"B"BJrZc3,# UH nSU;v M g7fzdefault.Nrcrs rRrgr>:a*/:rj)rqrrrrrr$rrXrrrrrrrrrrr_rrrrnrrrrhas_alphas_in_sd has_lora_unethas_diffusion_model has_defaultrs rRrv(QwenImageLoraLoaderMixin.lora_state_dict^sJJ{D1 $4e<**Y-!::&8$? 7D)::j$/JJ{D1 jj5  **%6=%zz*@%H  ""OL#7iP 02W#+-)!%  H!$#Jz#J J {H NN8 $+5+;+;+=W+=41aUVAV$!Q$+=JWHZHHK KK !!WJ!WW>:>> }0C{FzRJ(.rirjrkr.r1NrSr4s rRr*QwenImageLoraLoaderMixin.load_lora_weightsrUrTc U(a[SS5(a [S5e[RSURS35 UR USUUUUUS9 gr7r:r;s rRr23QwenImageLoraLoaderMixin.load_lora_into_transformerrXrTrr?rrrrrBc 0n0n U(aX(UR'XyUR'U(d [S5eURUUU UUUUS9 grZrAr\s rRr*QwenImageLoraLoaderMixin.save_lora_weightsr^rTrrrprrc 0>[TU]"SUUUUS.UD6 grrrs rRr"QwenImageLoraLoaderMixin.fuse_lora rrTc *>[TU]"SSU0UD6 grrrs rRr$QwenImageLoraLoaderMixin.unfuse_lorar rTrcrrrcrdrs@rRr|r|V,_'9/2T#u||:K5L/L99~$( ' /2T#u||:K5L/L' Dj'  ' R    < NR $"&#'9= bkk) "&c588??U\\+I&I!J       ! ,0$;  H"/!*.  I    Cy4'   (4A/=d3i==rTr|c^\rSrSrSrS/r\r\\ S\ \ \ \ R4-4Sj55rSS\ \ \ \ R4-S\ S-S \4S jjr\SS \4S jj5r\SS \ \R&-S \ \ \ R(R*\ R-4S\S\ S\S\S\ S-4Sjj5rS/SSS4S\\ S\S\S\\ S-4U4SjjjrS/4S\\ 4U4SjjjrSrU=r$)ZImageLoraLoaderMixini$zU Load LoRA layers into [`ZImageTransformer2DModel`]. Specific to [`ZImagePipeline`]. r.rXc |URSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n Sn U cS n S n SSS.n[UU U UUUUUUU UU S9 unn[SU55nU(aESn[R U5 UR 5VVs0sHunnSU;dMUU_M nnn[SU55n[SU55n[SU55n[SU55nU(dU(dU(dU(a [ U5nU (aUU4nU$UnU$s snnf)r%rNrFrrrrrrrr_Trrrrc3,# UH nSU;v M g7frrcrs rRrg8ZImageLoraLoaderMixin.lora_state_dict..Yrrjrrc3B# UHoRS5v M g7frrrs rRrgr_rrZc3B# UHoRS5v M g7frrrs rRrgr`rrZc3B# UHoRS5v M g7frWrrs rRrgrarrZc3,# UH nSU;v M g7frrcrs rRrgrbrrj)rqrrrrrr&rs rRrv%ZImageLoraLoaderMixin.lora_state_dict,sJJ{D1 $4e<**Y-!::&8$? 7D)::j$/JJ{D1 jj5  **%6=%zz*@%H  ""OL#7iP 02W#+-)!%  H!$#Jz#J J {H NN8 $+5+;+;+=W+=41aUVAV$!Q$+=JWHZHHK KK !!WJ!WW>:>> }0C{I*UJ(.rirjrkr.r1NrSr4s rRr'ZImageLoraLoaderMixin.load_lora_weightsjrUrTc U(a[SS5(a [S5e[RSURS35 UR USUUUUUS9 gr7r:r;s rRr20ZImageLoraLoaderMixin.load_lora_into_transformerrXrTrr?rrrrrBc 0n0n U(aX(UR'XyUR'U(d [S5eURUUU UUUUS9 grZrAr\s rRr'ZImageLoraLoaderMixin.save_lora_weightsr^rTrrrprrc 0>[TU]"SUUUUS.UD6 grrrs rRrZImageLoraLoaderMixin.fuse_lorarrTc *>[TU]"SSU0UD6 grrrs rRr!ZImageLoraLoaderMixin.unfuse_lorar rTrcrrrcrdrs@rRrr$rrTrc^\rSrSrSrS/r\r\\ S\ \ \ \ R4-4Sj55rSS\ \ \ \ R4-S\ S-S \4S jjr\SS \4S jj5r\SS \ \R&-S \ \ \ R(R*\ R-4S\S\ S\S\S\ S-4Sjj5rS/SSS4S\\ S\S\S\\ S-4U4SjjjrS/4S\\ 4U4SjjjrSrU=r$)Flux2LoraLoaderMixinizS Load LoRA layers into [`Flux2Transformer2DModel`]. Specific to [`Flux2Pipeline`]. r.rXc URSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n Sn U cS n S n SSS.n[UU U UUUUUUU UU S9 unn[SU55nU(aESn[R U5 UR 5VVs0sHunnSU;dMUU_M nnn[SU55nU(a[ U5nU (aUU4nU$UnU$[SU55nU(a6UR 5VVs0sHunnURSS5U_M nnn[SU55nU(a [U5nU (aUU4nU$UnU$s snnfs snnf)r%rNrFrrrrrrrr_Trrrrc3,# UH nSU;v M g7frrcrs rRrg7Flux2LoraLoaderMixin.lora_state_dict..'rrjrrc3,# UH nSU;v M g7frprcrs rRrgr-rqrjc3B# UHoRS5v M g7fbase_model.model.Nrrs rRrgr4S 1\\*=>> rZrrXc3B# UHoRS5v M g7frWrrs rRrgr8rrZ) rqrrrrrrrr)rrXrrrrrrrrrrr_rrrrnrrrrr~ris_peft_format is_ai_toolkits rRrv$Flux2LoraLoaderMixin.lora_state_dicts&JJ{D1 $4e<**Y-!::&8$? 7D)::j$/JJ{D1 jj5  **%6=%zz*@%H  ""OL#7iP 02W#+-)!%  H!$#Jz#J J {H NN8 $+5+;+;+=W+=41aUVAV$!Q$+=JWDDD ? KJ,@:x(CJGQCJS SS \f\l\l\no\nTXTUWX!))$79KLaO\nJoQjQQ G SJ(.[rirjrkr.r1NrSr4s rRr&Flux2LoraLoaderMixin.load_lora_weights@rUrTc U(a[SS5(a [S5e[RSURS35 UR USUUUUUS9 gr7r:r;s rRr2/Flux2LoraLoaderMixin.load_lora_into_transformerirXrTrr?rrrrrBc 0n0n U(aX(UR'XyUR'U(d [S5eURUUU UUUUS9 grZrAr\s rRr&Flux2LoraLoaderMixin.save_lora_weightsr^rTrrrprrc 0>[TU]"SUUUUS.UD6 grrrs rRrFlux2LoraLoaderMixin.fuse_lorarrTc *>[TU]"SSU0UD6 grrrs rRr Flux2LoraLoaderMixin.unfuse_lorar rTrcrrrcrdrs@rRrrs,_'A/2T#u||:K5L/LAAN$( ' /2T#u||:K5L/L' Dj'  ' R    < NR $"&#'9= bkk) "&c588??U\\+I&I!J       ! ,0$;  H"/!*.  I    Cy4'   (4A/=d3i==rTrc^\rSrSrSrS/r\r\\ S\ \ \ \ R4-4Sj55rSS\ \ \ \ R4-S\ S-S \4S jjr\SS \4S jj5r\SS \ \R&-S \ \ \ R(R*\ R-4S\S\ S\S\S\ S-4Sjj5rS/SSS4S\\ S\S\S\\ S-4U4SjjjrS/4S\\ 4U4SjjjrSrU=r$)ErnieImageLoraLoaderMixiniz] Load LoRA layers into [`ErnieImageTransformer2DModel`]. Specific to [`ErnieImagePipeline`]. r.rXc URSS5nURSS5nURSS5nURSS5nURSS5nURSS5nURS S5n URS S5n URS S5n URS S5n Sn U cS n S n SSS.n[UU U UUUUUUU UU S9 unn[SU55nU(aESn[R U5 UR 5VVs0sHunnSU;dMUU_M nnn[SU55nU(a6UR 5VVs0sHunnUR SS5U_M nnn[SU55nU(a6UR 5VVs0sHunnUR SS5U_M nnnU (aUU4nU$UnU$s snnfs snnfs snnf)r%rNrFrrrrrrrr_Trrrrc3,# UH nSU;v M g7frrcrs rRrg.rrjrrc3B# UHoRS5v M g7frrrs rRrgrrrZrrXc3B# UHoRS5v M g7frWrrs rRrgr s']R\Q 5G(H(HR\rZz transformer.)rqrrrrrr)rrXrrrrrrrrrrr_rrrrnrrrrris_diffusion_model_prefixrs rRrv)ErnieImageLoraLoaderMixin.lora_state_dicts!JJ{D1 $4e<**Y-!::&8$? 7D)::j$/JJ{D1 jj5  **%6=%zz*@%H  ""OL#7iP 02W#+-)!%  H!$#Jz#J J {H NN8 $+5+;+;+=W+=41aUVAV$!Q$+=JWS SS \f\l\l\no\nTXTUWX!))$79KLaO\nJo %(']R\']$]! $WaWgWgWijWitqRS!))$6GJWiJj(ErnieImageLoraLoaderMixin.load_lora_weights...rirjrkr.r1NrSr4s rRr+ErnieImageLoraLoaderMixin.load_lora_weightsrUrTc U(a[SS5(a [S5e[RSURS35 UR USUUUUUS9 gr7r:r;s rRr24ErnieImageLoraLoaderMixin.load_lora_into_transformer<rXrTrr?rrrrrBc 0n0n U(aX(UR'XyUR'U(d [S5eURUUU UUUUS9 grZrAr\s rRr+ErnieImageLoraLoaderMixin.save_lora_weights\r^rTrrrprrc 0>[TU]"SUUUUS.UD6 grrrs rRr#ErnieImageLoraLoaderMixin.fuse_lorarrTc *>[TU]"SSU0UD6 grrrs rRr%ErnieImageLoraLoaderMixin.unfuse_lorar rTrcrrrcrdrs@rRrrs,_'>/2T#u||:K5L/L>>H$( ' /2T#u||:K5L/L' Dj'  ' R    < NR $"&#'9= bkk) "&c588??U\\+I&I!J       ! ,0$;  H"/!*.  I    Cy4'   (4A/=d3i==rTrc(^\rSrSrU4SjrSrU=r$)LoraLoaderMixinicD>Sn[SSU5 [TU]"U0UD6 g)NzLoraLoaderMixin is deprecated and this will be removed in a future version. Please use `StableDiffusionLoraLoaderMixin`, instead.rz1.0.0)rr__init__)rargsrdeprecation_messager@s rRrLoraLoaderMixin.__init__s,b#W.AB $)&)rTrc)rArrrrrrrs@rRrrs **rTr)MrtypingrrFhuggingface_hub.utilsrutilsrrrr r r r r rrr lora_baserrrrrrlora_conversion_utilsrrrrrrrrr r!r"r#r$r%r&r'r(rr get_loggerrArrrrHrrrSrVrr#rJrfr#r-r?rRrmrrrrrrr;rVrhr|rrrrrcrTrRrs4  6    *#(D'"" D( + + % ' ' #C 2 2*.'   H %"   "&2M%B".bM=]M=`E=}E=P h=h=V E=mE=PE T-E TTU :U pA=}A=HE=ME=PH=mH=V`=-`=FE=-E=PF=MF=RH=-H=VI=]I=XE=}E=PO=O=dR= R=jE=mE=PH=-H=VK=}K=\K=MK=\S==S=lP= P=f*4*rT