3jSSKrSSKJr SSKrSSKrSSKrSSKJ r SSK J r J r JrJr SSKJrJr SSKJr SSKJr SS KJr SS KJr SS KJr SS KJrJrJ r J!r! SS K"J#r# SSK$J%r% SSK&J'r' \"5(a SSK(J)s J*r+ Sr,OSr,\ RZ"\.5r/\"5(aSSK0r0\ RZ"\.5r/Sr1Sr2Sr3Sr4"SS\%\5r5g)N)Callable) functional) CLIPTextModel CLIPTokenizer"Qwen2_5_VLForConditionalGenerationQwen2VLProcessor)MultiPipelineCallbacksPipelineCallback)VaeImageProcessor)KandinskyLoraLoaderMixin) AutoencoderKL)Kandinsky5Transformer3DModel)FlowMatchEulerDiscreteScheduler)is_ftfy_availableis_torch_xla_availableloggingreplace_example_docstring) randn_tensor)DiffusionPipeline)KandinskyImagePipelineOutputTFa< Examples: ```python >>> import torch >>> from diffusers import Kandinsky5T2IPipeline >>> # Available models: >>> # kandinskylab/Kandinsky-5.0-T2I-Lite-sft-Diffusers >>> # kandinskylab/Kandinsky-5.0-T2I-Lite-pretrain-Diffusers >>> model_id = "kandinskylab/Kandinsky-5.0-T2I-Lite-sft-Diffusers" >>> pipe = Kandinsky5T2IPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) >>> pipe = pipe.to("cuda") >>> prompt = "A cat and a dog baking a cake together in a kitchen." >>> output = pipe( ... prompt=prompt, ... negative_prompt="", ... height=1024, ... width=1024, ... num_inference_steps=50, ... guidance_scale=3.5, ... ).frames[0] ``` c[5(a[R"U5n[R"[R"U55nUR 5$)z Copied from https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/wan/pipeline_wan.py Clean text using ftfy if available and unescape HTML entities. )rftfyfix_texthtmlunescapestriptexts o/home/wildlama/miniconda3/lib/python3.13/site-packages/diffusers/pipelines/kandinsky5/pipeline_kandinsky_t2i.py basic_cleanr#Ws> }}T" ==t, -D ::<cV[R"SSU5nUR5nU$)z Copied from https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/wan/pipeline_wan.py Normalize whitespace in text by replacing multiple spaces with single space. z\s+ )resubrr s r"whitespace_cleanr)cs& 66&#t $D ::text_encoder_2->transformer->vae)latentsprompt_embeds_qwenprompt_embeds_clipnegative_prompt_embeds_qwennegative_prompt_embeds_clip transformervae text_encoder tokenizertext_encoder_2 tokenizer_2 schedulerc >[TU]5 URUUUUUUUS9 SUlSUlSUl[ UR S9Ul/SQUlg)N)r4r5r6r7r8r9r:z<|im_start|>system You are a promt engineer. Describe the image by detailing the color, shape, size, texture, quantity, text, spatial relationships of the objects and background:<|im_end|> <|im_start|>user {}<|im_end|>))vae_scale_factor))r?))rAr@))rCrB))rErD) super__init__register_modulesprompt_template prompt_template_encode_start_idxvae_scale_factor_spatialr image_processor resolutions) selfr4r5r6r7r8r9r: __class__s r"rGKandinsky5T2IPipeline.__init__sn  #%)#   02-()%0$B_B_`wr$Npromptdevicemax_sequence_lengthdtypec U=(d URnU=(d URRnUVs/sHoPRR U5PM nnUR U-nUR USSSSS9SnURSU:a[U5HsupXUR Sn UR RXS- S5n [U 5S :dMGU S[U 5*Xi'[RS US U 35 Mu UR USSUS SS S 9RU5n URU SS S S9SSSS2UR S24nU SSS2UR S24n[R"UR!S5S S9n["R$"USS S9R[R&S9nURU5U4$s snf)a Encode prompt using Qwen2.5-VL text encoder. This method processes the input prompt through the Qwen2.5-VL model to generate text embeddings suitable for image generation. Args: prompt list[str]: Input list of prompts device (torch.device): Device to run encoding on max_sequence_length (int): Maximum sequence length for tokenization dtype (torch.dtype): Data type for embeddings Returns: tuple[torch.Tensor, torch.Tensor]: Text embeddings and cumulative sequence lengths Nptlongest)r!imagesvideosreturn_tensorspadding input_idsrrzXThe following part of your input was truncated because `max_sequence_length` is set to z tokens: T)r!rYrZ max_length truncationr[r\)r] return_dictoutput_hidden_states hidden_statesattention_maskr)dim)rr)value)rU)_execution_devicer6rUrIformatrJr7shape enumeratedecodelenloggerwarningtotorchcumsumsumFpadint32)rNrRrSrTrUp full_textsmax_allowed_lenuntruncated_idsir!tokens removed_textinputsembedsre cu_seqlenss r"_encode_prompt_qwen)Kandinsky5T2IPipeline._encode_prompt_qwens1,14110**00>DEf**11!4f E??BUU.. )     $ 6$Z0(+D,Q,QTVW#~~44VRSJMNN/0 ,I 1&   "V* ""[)!%#     !$"G"G"II K  01!T5Z5Z5\2\]\\."4"4Q"7Q? UU:vQ7:::M yy++SFs$G*c U=(d URnU=(d URRnURUSSSSSS9R U5nUR"S0UD6SnUR U5$)a Encode prompt using CLIP text encoder. This method processes the input prompt through the CLIP model to generate pooled embeddings that capture semantic information. Args: prompt (str | list[str]): Input prompt or list of prompts device (torch.device): Device to run encoding on dtype (torch.dtype): Data type for embeddings Returns: torch.Tensor: Pooled text embeddings from CLIP MTr`rW)r`raadd_special_tokensr\r[ pooler_output)rhr8rUr9rp)rNrRrSrUr~ pooled_embeds r"_encode_prompt_clip)Kandinsky5T2IPipeline._encode_prompt_clips(14112,,22!! #  "  "V* **4V4_E u%%r$rnum_images_per_promptcU=(d URnU=(d URRn[U[5(dU/n[ U5nUVs/sHn[ U5PM nnURUUUUS9upURUUUS9n URSUS5nURXb-SURS5nU RSUS5n U RXb-S5n U R5n U RU5n [R"[R "S/U[R"S9U R%S5/5n XU 4$s snf)a0 Encodes a single prompt (positive or negative) into text encoder hidden states. This method combines embeddings from both Qwen2.5-VL and CLIP text encoders to create comprehensive text representations for image generation. Args: prompt (`str` or `list[str]`): Prompt to be encoded. num_images_per_prompt (`int`, *optional*, defaults to 1): Number of images to generate per prompt. max_sequence_length (`int`, *optional*, defaults to 512): Maximum sequence length for text encoding. Must be less than 1024 device (`torch.device`, *optional*): Torch device. dtype (`torch.dtype`, *optional*): Torch dtype. Returns: tuple[torch.Tensor, torch.Tensor, torch.Tensor]: - Qwen text embeddings of shape (batch_size * num_images_per_prompt, sequence_length, embedding_dim) - CLIP pooled embeddings of shape (batch_size * num_images_per_prompt, clip_embedding_dim) - Cumulative sequence lengths (`cu_seqlens`) for Qwen embeddings of shape (batch_size * num_images_per_prompt + 1,) )rRrSrTrU)rRrSrUrr^rrSrU)rhr6rU isinstancelistrmr+rrrepeatviewrjdiffrepeat_interleaverqcattensorrvrr)rNrRrrTrSrU batch_sizerwr0prompt_cu_seqlensr1original_lengthsrepeated_lengthsrepeated_cu_seqlenss r" encode_prompt#Kandinsky5T2IPipeline.encode_prompt!sB14110**00&$''XF[ +126a,q/62150H0H 3 1I1 -"556 066 $a 044  .4F4L4LR4P  066 $a 044Z5WY[\-113+== ! $ii \\1#fEKK @BRBYBYZ[B\ ] "7JJJa3sE#c r^U bU S:a [S5eXC4TR;aVSRTRV Vs/sHupSU SUS3PM snn 5n[R SUSUS US 35 U bX[ U4S jU 55(d>[S TR S U Vs/sHnUTR ;dMUPM sn35eUcUcU bUbUbU c [S5eUcUcU bUbUbU c [S5eUcUc [S5eUbA[U[5(d,[U[5(d[S[U535eUbC[U[5(d-[U[5(d[S[U535egggs snn fs snf)a Validate input parameters for the pipeline. Args: prompt: Input prompt negative_prompt: Negative prompt for guidance height: Image height width: Image width prompt_embeds_qwen: Pre-computed Qwen prompt embeddings prompt_embeds_clip: Pre-computed CLIP prompt embeddings negative_prompt_embeds_qwen: Pre-computed Qwen negative prompt embeddings negative_prompt_embeds_clip: Pre-computed CLIP negative prompt embeddings prompt_cu_seqlens: Pre-computed cumulative sequence lengths for Qwen positive prompt negative_prompt_cu_seqlens: Pre-computed cumulative sequence lengths for Qwen negative prompt callback_on_step_end_tensor_inputs: Callback tensor inputs Raises: ValueError: If inputs are invalid Nr?z*max_sequence_length must be less than 1024,()z'`height` and `width` have to be one of z , but are z and z(. Dimensions will be resized accordinglyc3@># UHoTR;v M g7f)N)_callback_tensor_inputs).0krNs r" 5Kandinsky5T2IPipeline.check_inputs..sF 7Y!-- -7Ysz2`callback_on_step_end_tensor_inputs` has to be in z , but found zuIf any of `prompt_embeds_qwen`, `prompt_embeds_clip`, or `prompt_cu_seqlens` is provided, all three must be provided.zIf any of `negative_prompt_embeds_qwen`, `negative_prompt_embeds_clip`, or `negative_prompt_cu_seqlens` is provided, all three must be provided.zProvide either `prompt` or `prompt_embeds_qwen` (and corresponding `prompt_embeds_clip` and `prompt_cu_seqlens`). Cannot leave all undefined.z2`prompt` has to be of type `str` or `list` but is z;`negative_prompt` has to be of type `str` or `list` but is ) ValueErrorrMjoinrnroallrrstrrtype)rNrRnegative_promptheightwidthr0r1r2r3rnegative_prompt_cu_seqlens"callback_on_step_end_tensor_inputsrTwhresolutions_strrs` r" check_inputs"Kandinsky5T2IPipeline.check_inputs|s\F  */BT/IIJ J ?$"2"2 2!hhAQAQ'RAQ!A3as! AQ'RSO NN9/9J*U[T\\abgahiQR  . 9#F 7YF C C DTEaEaDbbn|^pH|^vwbckokGkGbGpq|^pHoIJ   )-?-KO`Ol!)-?-GK\Kd 2 ( 3*6)5,3.6-5 2 >08`   z&#'>'>zRXZ^G_G_QRVW]R^Q_`a a  &?C00OUY9Z9ZZ[_`o[pZqrs s:[0 '[(SpHsF. 1F4 F4r?rnum_channels_latentsrr generatorr/returnc "UbURXeS9$US[U5UR-[U5UR-U4n [U[5(a*[ U5U:wa[ S[ U5SUS35e[XXeS9nU$)aG Prepare initial latent variables for text-to-image generation. This method creates random noise latents Args: batch_size (int): Number of images to generate num_channels_latents (int): Number of channels in latent space height (int): Height of generated image width (int): Width of generated image dtype (torch.dtype): Data type for latents device (torch.device): Device to create latents on generator (torch.Generator): Random number generator latents (torch.Tensor): Pre-existing latents to use Returns: torch.Tensor: Prepared latent tensor rrz/You have passed a list of generators of length z+, but requested an effective batch size of z@. Make sure the batch size matches the length of the generators.)rrSrU)rpintrKrrrmrr) rNrrrrrUrSrr/rjs r"prepare_latents%Kandinsky5T2IPipeline.prepare_latentss:  ::V:9 9  K488 8 J$77 7   i & &3y>Z+GA#i.AQR&<'gi  u&Vr$cUR$)z%Get the current guidance scale value.)_guidance_scalerNs r"guidance_scale$Kandinsky5T2IPipeline.guidance_scales###r$cUR$)z&Get the number of denoising timesteps.)_num_timestepsrs r" num_timesteps#Kandinsky5T2IPipeline.num_timesteps s"""r$cUR$)z)Check if generation has been interrupted.) _interruptrs r" interruptKandinsky5T2IPipeline.interruptsr$2g @pilTrnum_inference_stepsrr0r1r2r3rr output_typerbcallback_on_step_endrc[U[[45(a URnUR UUUUU U U U UUUUS9 XC4UR ;aWUR [ R"UR Vs/sHn[USUS- XC- - 5PM sn5upCX`l SUl URnURRnUb[U[5(aSnU/nO3Ub![U[5(a [!U5nOU R"SnU cUR%UUUUUS9upnUR&S:aUcSn[U[5(aUbU/[!U5-OU/nO<[!U5[!U5:wa$[)S [!U5S [!U5S 35eU cUR%UUUUUS9upnUR*R-UUS 9 UR*R.nURR0R2nUR5UU-UUUUUUU S 9n [6R8"SUS 9[6R8"X0R:-S-US 9[6R8"X@R:-S-US 9/n8"UR=5R?5RA5US 9nUb?[6R8"UR=5R?5RA5US 9OSn/SQnSn[!U5XPR*RB-- n [!U5Ul"URGUS9n![IU5GH.unn"URJ(aMU"RMS5ROUU-5n#URU RQU5U RQU5U RQU5U#RQU5UUUUSS9 RRn$UR&S:akU bhURU RQU5U RQU5U RQU5U#RQU5UUUUSS9 RRn%U%UU$U%- --n$UR*RUU$SS2SS24U"U SS9Sn Ub0n&UHn'[W5U'U&U''M U"UUU"U&5n(U(RYSU 5n U(RYSU 5n U(RYSU 5n U(RYSU 5n U(RYSU 5n U[!U5S- :Xd)US-U :a0US-UR*RB-S:XaU!R[5 [\(dGM[^R`"5 GM1 SSS5 U SS2SS2SS2SS2SU24n US:waU RQURbR5n U ReUUSX0R:-X@R:-U5n U RgSSSSSS5n U ReUU-UX0R:-X@R:-5n XRbR0Rh- n URbRkU 5RRn)URlRoU)US9n)OU n)URq5 U(dU)4$[sU)S9$s snf!,(df  GNM=f)a The call function to the pipeline for text-to-image generation. Args: prompt (`str` or `list[str]`, *optional*): The prompt or prompts to guide the image generation. If not defined, pass `prompt_embeds` instead. negative_prompt (`str` or `list[str]`, *optional*): The prompt or prompts to avoid during image generation. If not defined, pass `negative_prompt_embeds` instead. Ignored when not using guidance (`guidance_scale` < `1`). height (`int`, defaults to `1024`): The height in pixels of the generated image. width (`int`, defaults to `1024`): The width in pixels of the generated image. num_inference_steps (`int`, defaults to `50`): The number of denoising steps. guidance_scale (`float`, defaults to `5.0`): Guidance scale as defined in classifier-free guidance. num_images_per_prompt (`int`, *optional*, defaults to 1): The number of images to generate per prompt. generator (`torch.Generator` or `list[torch.Generator]`, *optional*): A torch generator to make generation deterministic. latents (`torch.Tensor`, *optional*): Pre-generated noisy latents. prompt_embeds_qwen (`torch.Tensor`, *optional*): Pre-generated Qwen text embeddings. prompt_embeds_clip (`torch.Tensor`, *optional*): Pre-generated CLIP text embeddings. negative_prompt_embeds_qwen (`torch.Tensor`, *optional*): Pre-generated Qwen negative text embeddings. negative_prompt_embeds_clip (`torch.Tensor`, *optional*): Pre-generated CLIP negative text embeddings. prompt_cu_seqlens (`torch.Tensor`, *optional*): Pre-generated cumulative sequence lengths for Qwen positive prompt. negative_prompt_cu_seqlens (`torch.Tensor`, *optional*): Pre-generated cumulative sequence lengths for Qwen negative prompt. output_type (`str`, *optional*, defaults to `"pil"`): The output format of the generated image. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`KandinskyImagePipelineOutput`]. callback_on_step_end (`Callable`, `PipelineCallback`, `MultiPipelineCallbacks`, *optional*): A function that is called at the end of each denoising step. callback_on_step_end_tensor_inputs (`List`, *optional*): The list of tensor inputs for the `callback_on_step_end` function. max_sequence_length (`int`, defaults to `512`): The maximum sequence length for text encoding. Examples: Returns: [`~KandinskyImagePipelineOutput`] or `tuple`: If `return_dict` is `True`, [`KandinskyImagePipelineOutput`] is returned, otherwise a `tuple` is returned where the first element is a list with the generated images. ) rRrrrr0r1r2r3rrrrTrrFN)rRrrTrSrU?z9`negative_prompt` must have same length as `prompt`. Got z vs .)rS)rrrrrUrSrr/r)rrr)totalT) rdencoder_hidden_statespooled_projectionstimestepvisual_rope_pos text_rope_pos scale_factor sparse_paramsrb)rbr/r0r1r2r3latentr )r)image):rr r tensor_inputsrrMnpargminabsrrrhr4rUrrrmrjrrrr: set_timesteps timestepsconfig in_visual_dimrrqarangerKrmaxitemorderr progress_barrkr unsqueezerrpsamplesteplocalspopupdate XLA_AVAILABLExm mark_stepr5reshapepermutescaling_factorrlrL postprocessmaybe_free_model_hooksr)*rNrRrrrrrrrr/r0r1r2r3rrrrbrrrTr{rSrUrrrrrnegative_text_rope_posrrnum_warmup_stepsrtr pred_velocityuncond_pred_velocitycallback_kwargsrcallback_outputsrs* r"__call__Kandinsky5T2IPipeline.__call__s@\ *-=?U,V W W1E1S1S . +11(C(C/'A/Q 3   ?$"2"2 2 ,, $JZJZ[JZQ3!qt ?@JZ[\ME .''  &&  *VS"9"9JXF  Jvt$<$<VJ+11!4J  %HLHZHZ&;$7 I[I E 4E    $&"$/3//EKEW?"3c&k"A^m]n_%V4 OPSTcPdOeeijmntjuivvwx+2&&..C,?%# 'e+Jd $$%8$HNN,,  $//66DD&&!$99!5'   LL6 * LL#@#@@AEf U LL"?"??1DV T   %6%;%;%=%A%A%C%H%H%JSYZ *5 LL388:>>@EEGPV W '  y>,?..BVBV,VV!)n   %8  9\!),1>>;;q>00>S1ST!% 0 0")**U"3*<*?*?*F'9'<']]qMq;r$rM..--mAqD.A1g[`-abcd'3&(O?-3Xa[*@';D!Q'X$.229gFG)9)=)=>RTf)g&)9)=)=>RTf)g&2B2F2F57R3/3C2F2F57R3/I**A9I/IqSTuX\XfXfXlXlNlpqNq '') =LLNq-:x!Q1&;';&;;< ( "jj0Goo%777666$ GooaAq!Q7Goo22$777666 G > >>GHHOOG,33E((44U 4TEE ##%8O+%88u\v: 9s<#[) )H [.[.. [=)rrrrLrIrJrMrK)NrQN)NN)rrQNN)NNNNNNNN)r?r?NNNN)+__name__ __module__ __qualname____firstlineno____doc__model_cpu_offload_seqrrrrrrrrrGrrrqrSrrUrrrr GeneratorTensorrpropertyrrrno_gradrEXAMPLE_DOC_STRINGfloatboolrr r r__static_attributes__ __classcell__)rOs@r"r-r-xsM4Mx1xx9 x $ x & x#x3x@'+#&$( B,S B, t#B,! B, {{T! B,N'+$( "&d3i"& t#"&{{T! "&N&'#&&*$( YKd3iYK #YK! YK  t# YK {{T! YKB $($(#'+/ Wtx%'$(&*DH'+00"0 0  0 {{T! 0 t#0??T%//%::TA0$0 0d$$## ]]_12#'26#% #,-DH'+2626;?;?15:>"' mq9B #&+X9d3iX9tCy4/X9 X9  X9 ! X9X9 #TzX9??T%//%::TAX9$X9"LL4/X9"LL4/X9&+\\D%8X9&+\\D%8X9!<<$.X9 %*LL4$7!X9"4Z#X9$%X9&'S$'79ILb9b'bcfjj'X9(-1I)X9*!+X93X9r$r-)6rtypingrnumpyrregexr'rqtorch.nnrrt transformersrrrr callbacksr r rLr loadersr modelsrmodels.transformersr schedulersrutilsrrrrutils.torch_utilsrpipeline_utilsrpipeline_outputrtorch_xla.core.xla_modelcore xla_modelrr get_loggerrrnrrr#r)r+r-rr$r"r%s  $kkA0/#?9 ..9))MM   H %   H %: x 9-/Gx 9r$