3jWSSKrSSKJr SSKrSSKrSSKJr SSK J r J r J r J r SSKJrJr SSKJr SSKJr SS KJr SS KJr SS KJrJrJrJr SS KJr SS K J!r! SSK"J#r# SSK$J%r% \"5(a SSK&J's J(r) Sr*OSr*\RV"\,5r-\"5(aSSK.r.\RV"\,5r-Sr/Sr0Sr1Sr2"SS\#\5r3g)N)Callable) functional) CLIPTextModel CLIPTokenizer"Qwen2_5_VLForConditionalGenerationQwen2VLProcessor)MultiPipelineCallbacksPipelineCallback)KandinskyLoraLoaderMixin)AutoencoderKLHunyuanVideo)Kandinsky5Transformer3DModel)FlowMatchEulerDiscreteScheduler)is_ftfy_availableis_torch_xla_availableloggingreplace_example_docstring) randn_tensor)VideoProcessor)DiffusionPipeline)KandinskyPipelineOutputTFa Examples: ```python >>> import torch >>> from diffusers import Kandinsky5T2VPipeline >>> from diffusers.utils import export_to_video >>> # Available models: >>> # kandinskylab/Kandinsky-5.0-T2V-Pro-sft-5s-Diffusers >>> # kandinskylab/Kandinsky-5.0-T2V-Lite-sft-5s-Diffusers >>> # kandinskylab/Kandinsky-5.0-T2V-Lite-nocfg-5s-Diffusers >>> # kandinskylab/Kandinsky-5.0-T2V-Lite-distilled16steps-5s-Diffusers >>> # kandinskylab/Kandinsky-5.0-T2V-Lite-pretrain-5s-Diffusers >>> # kandinskylab/Kandinsky-5.0-T2V-Lite-sft-10s-Diffusers >>> # kandinskylab/Kandinsky-5.0-T2V-Lite-nocfg-10s-Diffusers >>> # kandinskylab/Kandinsky-5.0-T2V-Lite-distilled16steps-10s-Diffusers >>> # kandinskylab/Kandinsky-5.0-T2V-Lite-pretrain-10s-Diffusers >>> model_id = "kandinskylab/Kandinsky-5.0-T2V-Lite-sft-5s-Diffusers" >>> pipe = Kandinsky5T2VPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) >>> pipe = pipe.to("cuda") >>> prompt = "A cat and a dog baking a cake together in a kitchen." >>> negative_prompt = "Static, 2D cartoon, cartoon, 2d animation, paintings, images, worst quality, low quality, ugly, deformed, walking backwards" >>> output = pipe( ... prompt=prompt, ... negative_prompt=negative_prompt, ... height=512, ... width=768, ... num_frames=121, ... num_inference_steps=50, ... guidance_scale=5.0, ... ).frames[0] >>> export_to_video(output, "output.mp4", fps=24, quality=9) ``` c[5(a[R"U5n[R"[R"U55nUR 5$)z Copied from https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/wan/pipeline_wan.py Clean text using ftfy if available and unescape HTML entities. )rftfyfix_texthtmlunescapestriptexts k/home/wildlama/miniconda3/lib/python3.13/site-packages/diffusers/pipelines/kandinsky5/pipeline_kandinsky.py basic_cleanr#bs> }}T" ==t, -D ::<cV[R"SSU5nUR5nU$)z Copied from https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/wan/pipeline_wan.py Normalize whitespace in text by replacing multiple spaces with single space. z\s+ )resubrr s r"whitespace_cleanr)ns& 66&#t $D ::\\R>-S-S)\R(S-S\R(4S*jjr \!S+5r"\!S,5r#\!S-5r$\RJ"5\&"\'5SSS!S.S/S0S1S SSSSSSSSS2S3SS)/S!4S\\\-S4\\\-S-S \S\S'\S5\S6\(S"\S-S(\R>\\R>-S-S)\R(S-S7\R(S-S8\R(S-S9\R(S-S:\R(S-S;\R(S-S<\R(S-S=\S-S>\)S?\*\\/S4\+-\,-S-S@\\S\4*SAjj55r-SBr.U=r/$)IKandinsky5T2VPipelineaK Pipeline for text-to-video generation using Kandinsky 5.0. This model inherits from [`DiffusionPipeline`]. Check the superclass documentation for the generic methods implemented for all pipelines (downloading, saving, running on a particular device, etc.). Args: transformer ([`Kandinsky5Transformer3DModel`]): Conditional Transformer to denoise the encoded video latents. vae ([`AutoencoderKLHunyuanVideo`]): Variational Auto-Encoder Model [hunyuanvideo-community/HunyuanVideo (vae)](https://huggingface.co/hunyuanvideo-community/HunyuanVideo) to encode and decode videos to and from latent representations. text_encoder ([`Qwen2_5_VLForConditionalGeneration`]): Frozen text-encoder [Qwen2.5-VL](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct). tokenizer ([`AutoProcessor`]): Tokenizer for Qwen2.5-VL. text_encoder_2 ([`CLIPTextModel`]): Frozen [CLIP](https://huggingface.co/docs/transformers/model_doc/clip#transformers.CLIPTextModel), specifically the [clip-vit-large-patch14](https://huggingface.co/openai/clip-vit-large-patch14) variant. tokenizer_2 ([`CLIPTokenizer`]): Tokenizer for CLIP. scheduler ([`FlowMatchEulerDiscreteScheduler`]): A scheduler to be used in combination with `transformer` to denoise the encoded video latents. z.text_encoder->text_encoder_2->transformer->vae)latentsprompt_embeds_qwenprompt_embeds_clipnegative_prompt_embeds_qwennegative_prompt_embeds_clip transformervae text_encoder tokenizertext_encoder_2 tokenizer_2 schedulerc >[TU]5 URUUUUUUUS9 SR/SQ5UlSUl[ USS5(a URRROSUl [ USS5(a URRROSUl [URS9Ulg) N)r4r5r6r7r8r9r: )zJ<|im_start|>system You are a promt engineer. Describe the video in detail.zjDescribe how the camera moves or shakes, describe the zoom and view angle, whether it follows the objects.zPDescribe the location of the video, main characters or objects and their action.z9Describe the dynamism of the video and presented actions.zName the visual style of the video: whether it is a professional footage, user generated content, some kind of animation, video game or scren content.z_Describe the visual effects, postprocessing and transitions if they are presented in the video.zGPay attention to the order of key actions shown in the scene.<|im_end|>z<|im_start|>user {}<|im_end|>r5)vae_scale_factor)super__init__register_modulesjoinprompt_template prompt_template_encode_start_idxgetattrr5configtemporal_compression_ratiovae_scale_factor_temporalspatial_compression_ratiovae_scale_factor_spatialrvideo_processor) selfr4r5r6r7r8r9r: __class__s r"rBKandinsky5T2VPipeline.__init__s  #%)#   $yy  14-;B$t:T:TDHHOO 6 6Z[ &V]]achjnUoUo(Q(Quv%-t?\?\]r$heightwidthreturnc@SnU"U5(aU"U5(agg)z Calculate the scale factor based on resolution. Args: height (int): Video height width (int): Video width Returns: tuple: Scale factor as (temporal_scale, height_scale, width_scale) c,SUs=:*=(a S:*$s $)NiV)xs r" between_480p=Kandinsky5T2VPipeline._get_scale_factor..between_480ps!??s? "? "r$)rrr)rHzG @r[rW)rNrQrRrYs r"_get_scale_factor'Kandinsky5T2VPipeline._get_scale_factors$ #   L$7$7"r$THWwTwHwWc$[R"XU/5R5n[R"SUS[RUS9nUR S5UR S5- R 5n U SU2SU24R5U SU2SU24R5U SU2SU24R5pn XS-:*n XS-:*n XS-:*n U R S5U R S5-RXX"5RSS5R5n U R S5U R S5-RXX-X-5RSS5nURX-U-X-U-5$)aQ Create a sparse temporal attention (STA) mask for efficient video generation. This method generates a mask that limits attention to nearby frames and spatial positions, reducing computational complexity for video generation. Args: T (int): Number of temporal frames H (int): Height in latent space W (int): Width in latent space wT (int): Temporal attention window size wH (int): Height attention window size wW (int): Width attention window size device (str): Device to create tensor on Returns: torch.Tensor: Sparse attention mask of shape (T*H*W, T*H*W) rrdtypedeviceNr) torchTensoramaxarangeint16 unsqueezeabsflattenreshape transpose)r^r_r`rarbrcrglrmatsta_tsta_hsta_wsta_hwstas r"fast_sta_nabla$Kandinsky5T2VPipeline.fast_sta_nablas{( LL! # ( ( * LLAq F C{{1~ A.335 BQBK   ! BQBK   ! BQBK   ! q q q //!$uq'99BB1NXXYZ\]^ffhq!F$4$4Q$77@@quaeT^^_`bcd{{1519aeai00r$c URRRSS:XdeURup4pVnX@RRRS-XPRRRS-X`RRRS-penURRRS:XGavUR UUS-US-URRR URRRURRRUS9nURS5RS5URRRSURRRURRR URRRURRRURRRXEU4URRRS. n U $S n U $) a Generate sparse attention parameters for the transformer based on sample dimensions. This method computes the sparse attention configuration needed for efficient video processing in the transformer model. Args: sample (torch.Tensor): Input sample tensor device (torch.device): Device to place tensors on Returns: Dict: Dictionary containing sparse attention parameters rrrnablar?rgT) sta_maskattention_type to_fractalPrarcrbadd_sta visual_shapemethodN) r4rH patch_sizeshaperrz attention_wT attention_wH attention_wW unsqueeze_ attention_Pattention_add_staattention_method) rNsamplergBr^r_r`_r sparse_paramss r"get_sparse_params'Kandinsky5T2VPipeline.get_sparse_paramss&&11!4999 aA !!((33A6 6 !!((33A6 6 !!((33A6 6    " " 1 1W <**QQ  ''44  ''44  ''44+H%//2==a@"&"2"2"9"9"H"H"%%,,88&&--::&&--::&&--::++22DD!"q **11BB M!Mr$Npromptrgmax_sequence_lengthrfc U=(d URnU=(d URRnUVs/sHoPRR U5PM nnUR U-nUR USSSSS9SnURSU:a[U5HsupXUR Sn UR RXS- S5n [U 5S :dMGU S[U 5*Xi'[RS US U 35 Mu UR USSUS SS S 9RU5n URU SS S S9SSSS2UR S24nU SSS2UR S24n[R"UR!S5S S9n["R$"USS S9R[R&S9nURU5U4$s snf)aL Encode prompt using Qwen2.5-VL text encoder. This method processes the input prompt through the Qwen2.5-VL model to generate text embeddings suitable for video generation. Args: prompt (str | list[str]): Input prompt or list of prompts device (torch.device): Device to run encoding on num_videos_per_prompt (int): Number of videos to generate per prompt max_sequence_length (int): Maximum sequence length for tokenization dtype (torch.dtype): Data type for embeddings Returns: tuple[torch.Tensor, torch.Tensor]: Text embeddings and cumulative sequence lengths Nptlongest)r!imagesvideosreturn_tensorspadding input_idsrrzXThe following part of your input was truncated because `max_sequence_length` is set to z tokens: T)r!rr max_length truncationrr)r return_dictoutput_hidden_states hidden_statesattention_maskrdim)rr)value)rf)_execution_devicer6rfrEformatrFr7r enumeratedecodelenloggerwarningtorhcumsumsumFpadint32)rNrrgrrfp full_textsmax_allowed_lenuntruncated_idsir!tokens removed_textinputsembedsr cu_seqlenss r"_encode_prompt_qwen)Kandinsky5T2VPipeline._encode_prompt_qwen9s1.14110**00>DEf**11!4f E??BUU.. )     $ 6$Z0(+D,Q,QTVW#~~44VRSJMNN/0 ,I 1&   "V* ""[)!%#     !$"G"G"II K  01!T5Z5Z5\2\]\\."4"4Q"7Q? UU:vQ7:::M yy++UFs$G*c U=(d URnU=(d URRnURUSSSSSS9R U5nUR"S0UD6SnUR U5$)a Encode prompt using CLIP text encoder. This method processes the input prompt through the CLIP model to generate pooled embeddings that capture semantic information. Args: prompt (str | list[str]): Input prompt or list of prompts device (torch.device): Device to run encoding on num_videos_per_prompt (int): Number of videos to generate per prompt dtype (torch.dtype): Data type for embeddings Returns: torch.Tensor: Pooled text embeddings from CLIP MTrr)rradd_special_tokensrr pooler_outputrW)rr8rfr9r)rNrrgrfr pooled_embeds r"_encode_prompt_clip)Kandinsky5T2VPipeline._encode_prompt_clips*14112,,22!! #  "  "V* **4V4_E u%%r$rnum_videos_per_promptcU=(d URnU=(d URRn[U[5(dU/n[ U5nUVs/sHn[ U5PM nnURUUUUS9upURUUUS9n URSUS5nURXb-SURS5nU RSUS5n U RXb-S5n U R5n U RU5n [R"[R "S/U[R"S9U R%S5/5n XU 4$s snf)a Encodes a single prompt (positive or negative) into text encoder hidden states. This method combines embeddings from both Qwen2.5-VL and CLIP text encoders to create comprehensive text representations for video generation. Args: prompt (`str` or `list[str]`): Prompt to be encoded. num_videos_per_prompt (`int`, *optional*, defaults to 1): Number of videos to generate per prompt. max_sequence_length (`int`, *optional*, defaults to 512): Maximum sequence length for text encoding. device (`torch.device`, *optional*): Torch device. dtype (`torch.dtype`, *optional*): Torch dtype. Returns: tuple[torch.Tensor, torch.Tensor, torch.Tensor]: - Qwen text embeddings of shape (batch_size * num_videos_per_prompt, sequence_length, embedding_dim) - CLIP pooled embeddings of shape (batch_size * num_videos_per_prompt, clip_embedding_dim) - Cumulative sequence lengths (`cu_seqlens`) for Qwen embeddings of shape (batch_size * num_videos_per_prompt + 1,) )rrgrrf)rrgrfrrrrgrf)rr6rf isinstancelistrr+rrrepeatviewrdiffrepeat_interleaverhcattensorrr)rNrrrrgrf batch_sizerr0prompt_cu_seqlensr1original_lengthsrepeated_lengthsrepeated_cu_seqlenss r" encode_prompt#Kandinsky5T2VPipeline.encode_promptsB14110**00&$''XF[ +126a,q/62150H0H 3 1I1 -"556 066 $a 044  .4F4L4LR4P  066 $a 044Z5WY[\ -113+== ! $ii \\1#fEKK @BRBYBYZ[B\ ] "7JJJe3sE#c ^U bU S:a [S5eUS-S:wd US-S:wa[SUSUS35eU bW[U4S jU 55(d=[S TRS U V s/sHoTR;dMU PM sn 35eUcUcU bUbUbU c [S 5eUcUcU bUbUbU c [S 5eUcUc [S5eUbA[U[5(d,[U[ 5(d[S[ U535eUbC[U[5(d-[U[ 5(d[S[ U535egggs sn f)a Validate input parameters for the pipeline. Args: prompt: Input prompt negative_prompt: Negative prompt for guidance height: Video height width: Video width prompt_embeds_qwen: Pre-computed Qwen prompt embeddings prompt_embeds_clip: Pre-computed CLIP prompt embeddings negative_prompt_embeds_qwen: Pre-computed Qwen negative prompt embeddings negative_prompt_embeds_clip: Pre-computed CLIP negative prompt embeddings prompt_cu_seqlens: Pre-computed cumulative sequence lengths for Qwen positive prompt negative_prompt_cu_seqlens: Pre-computed cumulative sequence lengths for Qwen negative prompt callback_on_step_end_tensor_inputs: Callback tensor inputs Raises: ValueError: If inputs are invalid Niz*max_sequence_length must be less than 1024rz8`height` and `width` have to be divisible by 16 but are z and .c3@># UHoTR;v M g7f)N)_callback_tensor_inputs).0krNs r" 5Kandinsky5T2VPipeline.check_inputs..*sF 7Y!-- -7Ysz2`callback_on_step_end_tensor_inputs` has to be in z , but found zuIf any of `prompt_embeds_qwen`, `prompt_embeds_clip`, or `prompt_cu_seqlens` is provided, all three must be provided.zIf any of `negative_prompt_embeds_qwen`, `negative_prompt_embeds_clip`, or `negative_prompt_cu_seqlens` is provided, all three must be provided.zProvide either `prompt` or `prompt_embeds_qwen` (and corresponding `prompt_embeds_clip` and `prompt_cu_seqlens`). Cannot leave all undefined.z2`prompt` has to be of type `str` or `list` but is z;`negative_prompt` has to be of type `str` or `list` but is ) ValueErrorallrrstrrtype)rNrnegative_promptrQrRr0r1r2r3rnegative_prompt_cu_seqlens"callback_on_step_end_tensor_inputsrrs` r" check_inputs"Kandinsky5T2VPipeline.check_inputss F  */BT/IIJ J B;! urzQWX^W__dejdkklmn n - 9#F 7YF C C DTEaEaDbbn|^pH|^vwkokGkGbGpq|^pHoIJ   )-?-KO`Ol!)-?-GK\Kd 2 ( 3*6)5,3.6-5 2 >08`   z&#'>'>zRXZ^G_G_QRVW]R^Q_`a a  &?C00OUY9Z9ZZ[_`o[pZqrs s:[0 'IpHs .E*E*rnum_channels_latents num_frames generatorr/c U bU RXvS9$US- UR-S-n UU [U5UR-[U5UR-U4n [ U[ 5(a*[ U5U:wa[S[ U5SUS35e[XXvS9n URR(a[R"U 5n [R"UU [U5UR-[U5UR-S/U RU RS9n [R "XU /SS 9n U $) a Prepare initial latent variables for video generation. This method creates random noise latents or uses provided latents as starting point for the denoising process. Args: batch_size (int): Number of videos to generate num_channels_latents (int): Number of channels in latent space height (int): Height of generated video width (int): Width of generated video num_frames (int): Number of frames in video dtype (torch.dtype): Data type for latents device (torch.device): Device to create latents on generator (torch.Generator): Random number generator latents (torch.Tensor): Pre-existing latents to use Returns: torch.Tensor: Prepared latent tensor rrz/You have passed a list of generators of length z+, but requested an effective batch size of z@. Make sure the batch size matches the length of the generators.)rrgrfrerr)rrJintrLrrrrrr4 visual_condrh zeros_likezerosrfrgr)rNrrrQrRrrfrgrr/num_latent_framesrrvisual_cond_masks r"prepare_latents%Kandinsky5T2VPipeline.prepare_latentsWsL>  ::V:9 9'!^0N0NNQRR   K488 8 J$77 7   i & &3y>Z+GA#i.AQR&<'gi  u&V    ' '**73K${{%K4#@#@@J$"?"?? mm~~  ii7G HbQGr$cUR$)z%Get the current guidance scale value.)_guidance_scalerNs r"guidance_scale$Kandinsky5T2VPipeline.guidance_scales###r$cUR$)z&Get the number of denoising timesteps.)_num_timestepsrs r" num_timesteps#Kandinsky5T2VPipeline.num_timestepss"""r$cUR$)z)Check if generation has been interrupted.) _interruptrs r" interruptKandinsky5T2VPipeline.interruptsr$iy2g@pilTrnum_inference_stepsrr0r1r2r3rr output_typercallback_on_step_endrc[U[[45(a URnUR UUUUU U U UUUUUS9 XPR -S:waA[ RSUR S35 XPR -UR -S-n[US5nXpl SUl URnURRnUb[U[5(aSnU/nO3Ub![U[5(a [!U5nOU R"SnU cUR%UUUUS9upnUR&S :aUcS n[U[5(aUbU/[!U5-OU/nO<[!U5[!U5:wa$[)S [!U5S [!U5S 35eU cUR%UUUUS9upnUR*R-UUS9 UR*R.nURR0R2nUR5UU-UUUUUUU U 5 n US- UR -S-n8"UUS9[6R8"X0R:-S-US9[6R8"X@R:-S-US9/n[6R8"UR=5R5R?5US9nUb?[6R8"UR=5R5R?5US9OSnURAX45nURCU U5n [!U5X`R*RD-- n![!U5Ul#URIUS9n"[KU5GHIun#n$URL(aMU$ROS5RQUU-5n%URU RSU5U RSU5U RSU5U%RSU5UUUU SS9 RTn&UR&S :akU bhURU RSU5U RSU5URSU5U%RSU5UUUU SS9 RTn'U'UU&U'- --n&UR*RWU&U$U SS2SS2SS2SS2SU24SS9SU SS2SS2SS2SS2SU24'Ub0n(UHn)[Y5U)U(U)'M U"UU#U$U(5n*U*R[SU 5n U*R[SU 5n U*R[SU 5n U*R[SU 5n U*R[SU5nU#[!U5S- :Xd)U#S-U!:a0U#S-UR*RD-S:XaU"R]5 [^(dGM4[`Rb"5 GML SSS5 U SS2SS2SS2SS2SU24n US:wGa"U RSURdR5n U RgUUUS- UR -S-X0R:-X@R:-U5n+U+RiSSSSSS5n+U+RgUU-UUS- UR -S-X0R:-X@R:-5n+U+URdR0Rj- n+URdRmU+5RTn+URnRqU+US9n+OU n+URs5 U(dU+4$[uU+S9$!,(df  GNq=f)a The call function to the pipeline for generation. Args: prompt (`str` or `list[str]`, *optional*): The prompt or prompts to guide the video generation. If not defined, pass `prompt_embeds` instead. negative_prompt (`str` or `list[str]`, *optional*): The prompt or prompts to avoid during video generation. If not defined, pass `negative_prompt_embeds` instead. Ignored when not using guidance (`guidance_scale` < `1`). height (`int`, defaults to `512`): The height in pixels of the generated video. width (`int`, defaults to `768`): The width in pixels of the generated video. num_frames (`int`, defaults to `25`): The number of frames in the generated video. num_inference_steps (`int`, defaults to `50`): The number of denoising steps. guidance_scale (`float`, defaults to `5.0`): Guidance scale as defined in classifier-free guidance. num_videos_per_prompt (`int`, *optional*, defaults to 1): The number of videos to generate per prompt. generator (`torch.Generator` or `list[torch.Generator]`, *optional*): A torch generator to make generation deterministic. latents (`torch.Tensor`, *optional*): Pre-generated noisy latents. prompt_embeds (`torch.Tensor`, *optional*): Pre-generated text embeddings. negative_prompt_embeds (`torch.Tensor`, *optional*): Pre-generated negative text embeddings. output_type (`str`, *optional*, defaults to `"pil"`): The output format of the generated video. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`KandinskyPipelineOutput`]. callback_on_step_end (`Callable`, `PipelineCallback`, `MultiPipelineCallbacks`, *optional*): A function that is called at the end of each denoising step. callback_on_step_end_tensor_inputs (`list`, *optional*): The list of tensor inputs for the `callback_on_step_end` function. max_sequence_length (`int`, defaults to `512`): The maximum sequence length for text encoding. Examples: Returns: [`~KandinskyPipelineOutput`] or `tuple`: If `return_dict` is `True`, [`KandinskyPipelineOutput`] is returned, otherwise a `tuple` is returned where the first element is a list with the generated images. ) rrrQrRr0r1r2r3rrrrrz(`num_frames - 1` has to be divisible by z!. Rounding to the nearest number.FNr)rrrgrfg?z{Static, 2D cartoon, cartoon, 2d animation, paintings, images, worst quality, low quality, ugly, deformed, walking backwardsz9`negative_prompt` must have same length as `prompt`. Got z vs rr~r)totalT) rencoder_hidden_statespooled_projectionstimestepvisual_rope_pos text_rope_pos scale_factorrr)rr/r0r1r2r3latentr r>)r )frames);rr r tensor_inputsrrJrrmaxrrrr4rfrrrrrrrr: set_timesteps timestepsrH in_visual_dimrrhrkrLritemr\rorderr progress_barrrrmrrrsteplocalspopupdate XLA_AVAILABLExm mark_stepr5rppermutescaling_factorrrMpostprocess_videomaybe_free_model_hooksr),rNrrrQrRrr rrrr/r0r1r2r3rrr rr rrrgrfrrrrrrnegative_text_rope_posrrnum_warmup_stepsr rtr pred_velocityuncond_pred_velocitycallback_kwargsrcallback_outputsvideos, r"__call__Kandinsky5T2VPipeline.__call__sR *-=?U,V W W1E1S1S . +11(C(C/'A/Q 3   66 6! ; NN:4;Y;Y:ZZ{| $'E'EEHfHffijjJQ' -''  &&  *VS"9"9JXF  Jvt$<$<VJ+11!4J  %HLHZHZ$7 I[I E 4E    $&#`/3//EKEW?"3c&k"A^m]n_%V4 OPSTcPdOeeijmntjuivvwx+2&&.,?%# 'e+Jd $$%8$HNN,,  $//66DD&& . .         (!^0N0NNQRR LL*6 : LL#@#@@AEf U LL"?"??1DV T   %6%;%;%=%A%A%C%H%H%JSYZ *5 LL388:>>@EEGPV W --f< ..w? y>,?..BVBV,VV!)n   %8  9\!),1>>;;q>00>S1ST!% 0 0")**U"3*<*?*?*F'9'<']]qMq;r$rM=A^^=P=P!1gaAq:O;O:O.O&P^c>Q>>1a$9%9$99:(3&(O?-3Xa[*@';D!Q'X$.229gFG)9)=)=>RTf)g&)9)=)=>RTf)g&2B2F2F57R3/3C2F2F57R3/I**A9I/IqSTuX\XfXfXlXlNlpqNq '') =LLNu-:|!Q1&;';&;;< ( "jj0GOO%aD$B$BBQF777666$ EMM!Q1a3EMM22$aD$B$BBQF777666 EDHHOO:::EHHOOE*11E((::5k:ZEE ##%8O&e44E: 9sH;]] ]&)rrrrErFrLrJrM)r r r cuda)NN)NN)rrNN)NNNNNNNN)rrVi@QNNNN)0__name__ __module__ __qualname____firstlineno____doc__model_cpu_offload_seqrrr rrrrrrBrtupler\ staticmethodrhrirzrrrrgrfrrrr Generatorrpropertyrrrno_gradrEXAMPLE_DOC_STRINGfloatboolrr r r4__static_attributes__ __classcell__)rOs@r"r-r-s4M(^1(^'(^9 (^ $ (^ & (^#(^3(^T##C#E#( 1# 1# 1# 13 1 1S 1hmhtht 1 1D/h'+#&$( D,d3iD, t#D,! D, {{T! D,R'+$( #&d3i#& t##&{{T! #&P&'#&&*$( [Kd3i[K #[K! [K  t# [K {{T! [KF $($(#'+/ Ttr%'$(&*DH'+BB"B B  B  B{{T!B t#B??T%//%::TAB$B BH$$## ]]_12#'26#% #,-DH'+2626;?;?15:>"' nr9B #&-\5d3i\5tCy4/\5 \5  \5  \5!\5\5 #Tz\5??T%//%::TA\5$\5"LL4/\5"LL4/\5&+\\D%8\5&+\\D%8\5 !<<$.!\5"%*LL4$7#\5$4Z%\5&'\5('Sz4'78;KKNddgkk)\5*-1I+\5,!-\53\5r$r-)4rtypingrregexr'rhtorch.nnrr transformersrrrr callbacksr r loadersr modelsr models.transformersr schedulersrutilsrrrrutils.torch_utilsrrMrpipeline_utilsrpipeline_outputrtorch_xla.core.xla_modelcore xla_modelr&r% get_loggerr9rrrDr#r)r+r-rWr$r"rZs  $kkA//?9 .-.4))MM   H %   H %&R E 5-/GE 5r$