3j,NSSKrSSKJr SSKJr SSKJrJr SSKJ r SSK J r SS K J r SS KJr S S KJr \"\5rS rSS\R*S\R,S-S\4Sjjr"SS\5rg)N)PipelineImageInput)AutoencoderKLLTX2Video) get_loggerreplace_example_docstring) randn_tensor)VideoProcessor)LTXPipelineOutput)DiffusionPipeline)LTX2LatentUpsamplerModela Examples: ```py >>> import torch >>> from diffusers import LTX2ImageToVideoPipeline, LTX2LatentUpsamplePipeline >>> from diffusers.pipelines.ltx2.export_utils import encode_video >>> from diffusers.pipelines.ltx2.latent_upsampler import LTX2LatentUpsamplerModel >>> from diffusers.utils import load_image >>> pipe = LTX2ImageToVideoPipeline.from_pretrained("Lightricks/LTX-2", torch_dtype=torch.bfloat16) >>> pipe.enable_model_cpu_offload() >>> image = load_image( ... "https://huggingface.co/datasets/a-r-r-o-w/tiny-meme-dataset-captioned/resolve/main/images/8.png" ... ) >>> prompt = "A young girl stands calmly in the foreground, looking directly at the camera, as a house fire rages in the background." >>> negative_prompt = "worst quality, inconsistent motion, blurry, jittery, distorted" >>> frame_rate = 24.0 >>> video, audio = pipe( ... image=image, ... prompt=prompt, ... negative_prompt=negative_prompt, ... width=768, ... height=512, ... num_frames=121, ... frame_rate=frame_rate, ... num_inference_steps=40, ... guidance_scale=4.0, ... output_type="pil", ... return_dict=False, ... ) >>> latent_upsampler = LTX2LatentUpsamplerModel.from_pretrained( ... "Lightricks/LTX-2", subfolder="latent_upsampler", torch_dtype=torch.bfloat16 ... ) >>> upsample_pipe = LTX2LatentUpsamplePipeline(vae=pipe.vae, latent_upsampler=latent_upsampler) >>> upsample_pipe.vae.enable_tiling() >>> upsample_pipe.to(device="cuda", dtype=torch.bfloat16) >>> video = upsample_pipe( ... video=video, ... width=768, ... height=512, ... output_type="np", ... return_dict=False, ... )[0] >>> encode_video( ... video[0], ... fps=frame_rate, ... audio=audio[0].float().cpu(), ... audio_sample_rate=pipe.vocoder.config.output_sampling_rate, # should be 24000 ... output_path="video.mp4", ... ) ``` encoder_output generator sample_modec[US5(a!US:XaURRU5$[US5(a US:XaURR5$[US5(a UR$[ S5e)N latent_distsampleargmaxlatentsz3Could not access latents of provided encoder_output)hasattrrrmoderAttributeError)rrrs p/home/wildlama/miniconda3/lib/python3.13/site-packages/diffusers/pipelines/ltx2/pipeline_ltx2_latent_upsample.pyretrieve_latentsr}~}--+2I))00;;  / /K84K))..00  + +%%%RSSc!^\rSrSrSrS\S\SS4U4SjjrS*S\RS-S \ S \ S \ S \ S \ S\ S\RS-S\RS-S\RS-S\RS-S\R4SjjrS+S\RS\RS\4SjjrS\RS\S\R4Sjr\S+S\RS\RS\RS\S\R4 Sjj5r\S,S\RS \ S \ S \ S\ S\ S\R4Sjj5rS r\R,"5\"\5S-S\\S-S \ S \ S \ S \ S\ S\RS-S!\S"\\\-S#\\\-S-S$\S%\S\R\\R-S-S&\S-S'\4S(jj55rS)rU=r$).LTX2LatentUpsamplePipelinehzvae->latent_upsamplervaelatent_upsamplerreturnNc>[TU]5 URXS9 [USS5bURR OSUl[USS5bURROSUl[UR S9Ul g)N)r r!r )vae_scale_factor) super__init__register_modulesgetattrr spatial_compression_ratiovae_spatial_compression_ratiotemporal_compression_ratiovae_temporal_compression_ratior video_processor)selfr r! __class__s rr(#LTX2LatentUpsamplePipeline.__init__ks  #I3:$t2L2XDHH . .^` *4;43M3YDHH / /_` + .t?a?abrvideo batch_size num_framesheightwidthspatial_patch_sizetemporal_patch_sizedtypedevicerrc U bcU RS:XaDUS- UR-S-n X@R-n XPR-nURXXXg5n U R XS9$UR XR R S9n[U [5(a|[U 5U:wa[S[U 5SUS35e[U5Vs/sH;n[UR RXRS55X5PM= nnODUVs/sH7n[UR RURS55U 5PM9 nn[R "USS9R U5nU$s snfs snf) Nrr r;r:z/You have passed a list of generators of length z+, but requested an effective batch size of z@. Make sure the batch size matches the length of the generators.rdim)ndimr.r,_unpack_latentstor r: isinstancelistlen ValueErrorrangerencode unsqueezetorchcat)r0r3r4r5r6r7r8r9r:r;rrlatent_num_frames latent_height latent_widthi init_latentsvids rprepare_latents*LTX2LatentUpsamplePipeline.prepare_latents|s  ||q %/!^8[8[$[^_$_! &*L*L L $(J(JJ .. M_::V:9 9hhnn= i & &9~+ Ec)nEUV *|+km affp`q`q[\ 1C1CA1F!GV`q Lgllfk_b,TXX__S]]1=M-NPYZfkLlyy1588? ms AF>F reference_latentsfactorcpUR5n[URS55Hpn[URS55HOn[R"X%U4SS9upx[R"XEU4SS9upXEU4U - U - U-U-XEU4'MQ Mr [R "XU5nU$)a Applies Adaptive Instance Normalization (AdaIN) to a latent tensor based on statistics from a reference latent tensor. Args: latent (`torch.Tensor`): Input latents to normalize reference_latents (`torch.Tensor`): The reference latents providing style statistics. factor (`float`): Blending factor between original and transformed latent. Range: -10.0 to 10.0, Default: 1.0 Returns: torch.Tensor: The transformed latent tensor rr Nr>)clonerGsizerJstd_meanlerp) r0rrTrUresultrOcr_sdr_meani_sdi_means radain_filter_latent.LTX2LatentUpsamplePipeline.adain_filter_latents w||A'A7<<?+$~~.?1.E4P $~~fTlE !'1!6$ >$FO!t ,(GV4 r compressioncUS-n[R"U5n[R"SU-US- -5nSSU-U-- nX-nU$)a Applies a non-linear tone-mapping function to latent values to reduce their dynamic range in a perceptually smooth way using a sigmoid-based compression. This is useful for regularizing high-variance latents or for conditioning outputs during generation, especially when controlling dynamic behavior with a `compression` factor. Args: latents : torch.Tensor Input latent tensor with arbitrary shape. Expected to be roughly in [-1, 1] or [0, 1] range. compression : float Compression strength in the range [0, 1]. - 0.0: No tone-mapping (identity transform) - 1.0: Full compression effect Returns: torch.Tensor The tone-mapped latent tensor of the same shape as input. g?g@?g?)rJabssigmoid)r0rrc scale_factor abs_latents sigmoid_termscalesfiltereds rtone_map_latents+LTX2LatentUpsamplePipeline.tone_map_latentss\*#T) ii( }}S<%7;;L%MN s\)L88#r latents_mean latents_stdscaling_factorcURSSSSS5RURUR5nURSSSSS5RURUR5nX-U- U-nU$)Nr )viewrBr;r:)rrorprqs r_denormalize_latents/LTX2LatentUpsamplePipeline._denormalize_latentssu $((B1a8;;GNNGMMZ !&&q"aA699'..'--X '.8<Gr patch_size patch_size_tc URS5nURXaX#SXTU5nURSSSSSSSS 5RSS 5RSS5RSS5nU$) Nrrsr r r)rXreshapepermuteflatten)rr5r6r7rwrxr4s rrA*LTX2LatentUpsamplePipeline._unpack_latentsst\\!_ //*&\gqr//!Q1aAq9AA!QGOOPQSTU]]^_abcrcX R-S:wdX0R-S:wa[SUSUS35eUbUb [S5eUcUc [S5eSUs=::aS::d O [S5eg) Nrz8`height` and `width` have to be divisible by 32 but are z and .z1Only one of `video` or `latents` can be provided.z/One of `video` or `latents` has to be provided.r z8`tone_map_compression_ratio` must be in the range [0, 1])r,rF)r0r3r6r7rtone_map_compression_ratios r check_inputs'LTX2LatentUpsamplePipeline.check_inputss 66 6! ;uGiGi?imn?nWX^W__dejdkklmn n  !4PQ Q =W_NO O/414WX X5rlatents_normalizeddecode_timestepdecode_noise_scale adain_factorr output_type return_dictc8URUUUUU S9 UbSnOURSnURnUb[U5nX@R-S:waUX@R-UR-S-nUSUn[ R SURS[U5SUS35 URRXUS 9nURU[RS 9nUSLnURUUUUUUU[RUU US 9 nU(a`U(aYURXpRRURR URR"R$5nURUR&R(5nUR'U5nU S :aUR+UX{5nOUnU S :aUR-X|5nUS :XaUnGOURR"R.(dSnO[1URU UUR(S9n[3U [45(dU /U-n U cU n O [45(dU /U-n [R6"U UUR(S 9n[R6"U UUR(S 9SS2SSSS4n SU - U-U U--nURR9UUSS9SnURR;XS9nUR=5 U(dU4$[?US9$)aq Function invoked when calling the pipeline for generation. Args: video (`list[PipelineImageInput]`, *optional*) The video to be upsampled (such as a LTX 2.0 first stage output). If not supplied, `latents` should be supplied. height (`int`, *optional*, defaults to `512`): The height in pixels of the input video (not the generated video, which will have a larger resolution). width (`int`, *optional*, defaults to `768`): The width in pixels of the input video (not the generated video, which will have a larger resolution). num_frames (`int`, *optional*, defaults to `121`): The number of frames in the input video. spatial_patch_size (`int`, *optional*, defaults to `1`): The spatial patch size of the video latents. Used when `latents` is supplied if unpacking is necessary. temporal_patch_size (`int`, *optional*, defaults to `1`): The temporal patch size of the video latents. Used when `latents` is supplied if unpacking is necessary. latents (`torch.Tensor`, *optional*): Pre-generated video latents. This can be supplied in place of the `video` argument. Can either be a patch sequence of shape `(batch_size, seq_len, hidden_dim)` or a video latent of shape `(batch_size, latent_channels, latent_frames, latent_height, latent_width)`. latents_normalized (`bool`, *optional*, defaults to `False`) If `latents` are supplied, whether the `latents` are normalized using the VAE latent mean and std. If `True`, the `latents` will be denormalized before being supplied to the latent upsampler. decode_timestep (`float`, defaults to `0.0`): The timestep at which generated video is decoded. decode_noise_scale (`float`, defaults to `None`): The interpolation factor between random noise and denoised latents at the decode timestep. adain_factor (`float`, *optional*, defaults to `0.0`): Adaptive Instance Normalization (AdaIN) blending factor between the upsampled and original latents. Should be in [-10.0, 10.0]; supplying 0.0 (the default) means that AdaIN is not performed. tone_map_compression_ratio (`float`, *optional*, defaults to `0.0`): The compression strength for tone mapping, which will reduce the dynamic range of the latent values. This is useful for regularizing high-variance latents or for conditioning outputs during generation. Should be in [0, 1], where 0.0 (the default) means tone mapping is not applied and 1.0 corresponds to the full compression effect. generator (`torch.Generator` or `list[torch.Generator]`, *optional*): One or a list of [torch generator(s)](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation deterministic. output_type (`str`, *optional*, defaults to `"pil"`): The output format of the generate image. Choose between [PIL](https://pillow.readthedocs.io/en/stable/): `PIL.Image.Image` or `np.array`. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~pipelines.ltx.LTXPipelineOutput`] instead of a plain tuple. Examples: Returns: [`~pipelines.ltx.LTXPipelineOutput`] or `tuple`: If `return_dict` is `True`, [`~pipelines.ltx.LTXPipelineOutput`] is returned, otherwise a `tuple` is returned where the first element is the upsampled video. )r3r6r7rrNr rz-Video length expected to be of the form `k * z + 1` but is z. Truncating to z frames.)r6r7r=) r3r4r5r6r7r8r9r:r;rrlatent)rr;r:F)r)r)frames) rshape_execution_devicerEr.loggerwarningr/preprocess_videorBrJfloat32rRrur rorpconfigrqr!r:rarmtimestep_conditioningrrCrDtensordecodepostprocess_videomaybe_free_model_hooksr )r0r3r6r7r5r8r9rrrrrrrrrr4r;latents_suppliedlatents_upsampledtimestepnoises r__call__#LTX2LatentUpsamplePipeline.__call__sKT 'A    J q)J''  UJ???1D"E"EEHkHkknookz*CDDgDgChhuvyzwAvBBRS]R^^fg((99%V[9\EHHF%--H@E"$.&&!!1 3--'    2//..0D0DdhhooFdFdG**T22889 11': # ../@'XG'G % +++GPG ( "E88??88$W]]iPV^e^k^kl!/488'6&7*&DO%-)8&#$6==*<)= )J& <<gmm\%*\\2DV[b[h[h%itT4-&"11Wrs{  1,:--3.6 H 8z`h TLL T-2__t-C TY\ TA/!2A/r