3j, SSKrSSKJr SSKJr SSKrSSKJr SSKJ r J r SSK J r J r SSKJrJrJrJr SS KJrJr SS KJrJrJr SS KJr SS KJr SS KJrJ r SSK!J"r" SSK#J$r$ \RJ"\&5r'S\RPS\)\RP\RP4S\RP4Sjr*S\RPS\)\RP\RP4S\RP4Sjr+\"SS\55r,"SS\RZ5r."SS5r/"SS5r0"SS\RRZ\5r1"SS \RZ5r2"S!S"\RZ5r3"S#S$\"\ \\ \ \5r4g)%N) dataclass)Any) ConfigMixinregister_to_config)FromOriginalModelMixinPeftAdapterMixin) BaseOutputapply_lora_scaleis_torch_versionlogging)ContextParallelInputContextParallelOutput)AttentionMixinAttentionModuleMixin FeedForward)dispatch_attention_fn) CacheMixin))PixArtAlphaCombinedTimestepSizeEmbeddingsPixArtAlphaTextProjection) ModelMixin)RMSNormxfreqsreturnc"Uup#URSS5RS5upE[R"U*U/SS9R S5nUR 5U-UR 5U--R UR5nU$)Nr)rrdim) unflattenunbindtorchstackflattenfloattodtype)rrcossinx_realx_imag x_rotatedouts h/home/wildlama/miniconda3/lib/python3.13/site-packages/diffusers/models/transformers/transformer_ltx2.pyapply_interleaved_rotary_embr0&s|HC[[G,33B7NF fWf-26>>qAI 779s?Y__.4 4 8 8 AC Jc6Uup#URnSnURS:waDURS:Xa4URupgpURXhUS5R SS5nSnURSn U S-S:wa[ SU S 35eU S-n UR"/URSSQSPU P76R 5n U S SS2SS24n U S SS2SS24nURS 5nURS 5nX-nUS SS2SS24nUS SS2SS24nURU*U5 URUU 5 UR"/URSS QU P76nU(a#UR SS5RWWS5nURUS 9nU$) NFrrTrz6Expected x.shape[-1] to be even for split rotary, got ..r() r(ndimshapereshapeswapaxes ValueErrorr& unsqueezeaddcmul_r')rrr)r*x_dtype needs_reshapebht_lastrsplit_xfirst_xsecond_xcos_usin_ur. first_out second_outs r/apply_split_rotary_embrN.sHCggGMvv{sxx1}YY a IIaAr " + +Aq 1  772;D ax1}QRVQWWXYZZ  Aii,",q,!,224Gc2A2qj!GsABz"H MM" E MM" E /CC!QJIS!"aZJ vx(w' ++ ,syy"~ ,t ,Cll1a ((Ar2 &&w& C Jr1c.\rSrSr%SrS\S'S\S'Srg)AudioVisualModelOutputWa Holds the output of an audiovisual model which produces both visual (e.g. video) and audio outputs. Args: sample (`torch.Tensor` of shape `(batch_size, num_channels, num_frames, height, width)`): The hidden states output conditioned on the `encoder_hidden_states` input, representing the visual output of the model. This is typically a video (spatiotemporal) output. audio_sample (`torch.Tensor` of shape `(batch_size, TODO)`): The audio output of the audiovisual model. z torch.Tensorsample audio_sampleN)__name__ __module__ __qualname____firstlineno____doc____annotations____static_attributes__rTr1r/rPrPWs    r1rPc4^\rSrSrSrSS\S\S\4U4SjjjrSS\RS \ \ \R4S-S \S-S \RS-S \ \R\R\R\R\R44 S jjrSrU=r$)LTX2AdaLayerNormSingleha Norm layer adaptive layer norm single (adaLN-single). As proposed in PixArt-Alpha (see: https://huggingface.co/papers/2310.00426; Section 2.3) and adapted by the LTX-2.0 model. In particular, the number of modulation parameters to be calculated is now configurable. Parameters: embedding_dim (`int`): The size of each embedding vector. num_mod_params (`int`, *optional*, defaults to `6`): The number of modulation parameters which will be calculated in the first return argument. The default of 6 is standard, but sometimes we may want to have a different (usually smaller) number of modulation parameters. use_additional_conditions (`bool`, *optional*, defaults to `False`): Whether to use additional conditions for normalization or not. embedding_dimnum_mod_paramsuse_additional_conditionsc>[TU]5 X l[XS-US9Ul[ R "5Ul[ R"XRU-SS9Ul g)Nr) size_emb_dimraTbias) super__init__r`rembnnSiLUsiluLinearlinear)selfr_r`ra __class__s r/rgLTX2AdaLayerNormSingle.__init__ysX ,< (:Vo GGI ii /B/B]/RY]^ r1Ntimestepadded_cond_kwargs batch_size hidden_dtypercU=(d SSS.nUR"U40UDX4S.D6nURURU55U4$)N) resolution aspect_ratiorsrt)rhrmrk)rnrqrrrsrtembedded_timesteps r/forwardLTX2AdaLayerNormSingle.forwardsN.[VZ1[ HHXu1Buzu{{499%678:KKKr1)rhrmr`rk)F)NNN)rUrVrWrXrYintboolrgr#Tensordictstrr(tuplerzr[ __classcell__ros@r/r]r]hs _c _3 __c _ _=A!%+/ L,, L U\\ 12T9 L$J L kkD( L u||U\\5<<u||S T L Lr1r]c$\rSrSrSrSrSrSrSSSS\RS\RS-S \RS-S \ \R\R4S-S \ \R\R4S-S \R4S jjr Sr g)LTX2AudioVideoAttnProcessora- Processor for implementing attention (SDPA is used by default if you're using PyTorch 2.0) for the LTX-2.0 model. Compared to the LTX-1.0 model, we allow the RoPE embeddings for the queries and keys to be separate so that we can support audio-to-video (a2v) and video-to-audio (v2a) cross attention. Nc<[SS5(a [S5egN>]28QE6/I^O_C^^w.25KE0/I^O_COOA B'78E--DJJ#34COOA B'78E1(// $ 5 5 M*11!Q7M),,U[[9M , a+ % 5AR S    *)33A B7GHM%-- 44E)EOOB,??M)11!Q7M A}5  A}5 r1rT)NNNNNN)rUrVrWrXrYrrrgr#rrr~rr[rTr1r/rrs6:.2EICG15%)RR||R %||d2 R  t+ R   ell :;dB RellELL89D@R!<<$.Rd{R RRr1rc^\rSrSrSr\r\\/rSS\ S\ S\ S\ S\ S \ S \ S-S \ S \ S \ S\ S\ S\ 4U4Sjjjr SS\RS\RS-S\RS-S\\R\R4S-S\\R\R4S-S\R4 SjjrSrU=r$)riJz Attention class for all LTX-2.0 attention layers. Compared to LTX-1.0, this supports specifying the query and key RoPE embeddings separately for audio-to-video (a2v) and video-to-audio (v2a) cross-attention. N query_dimrkv_headsdim_headdropoutrecross_attention_dimout_biasqk_normnorm_epsnorm_elementwise_affinerapply_gated_attentionc>[TU]5 U S:wa [S5eX@lXB-UlUc UROXC-UlXlUbUOUUlX`lXPl Xl X l Xl [RRXB-XS9Ul[RRXC-XS9Ul[RR%XRUS9Ul[RR%URUR US9Ul[RR%URUR US9Ul[RR-/5UlUR.R1[RR%URURUS95 UR.R1[RR3U55 U (a$[RR%XSS9UlOSUlUcUR75nUR9U5 g)Nrms_norm_across_headszIOnly 'rms_norm_across_heads' is supported as a valid value for `qk_norm`.epselementwise_affinerdT)rfrgNotImplementedErrorhead_dim inner_dim inner_kv_dimrruse_biasrout_dimrrr#rirrrrlrrr ModuleListrappendDropoutr_default_processor_cls set_processor)rnrrrrrrerrrrrrr processorros r/rgLTX2Attention.__init__Ss"  - -%&qr r !).6.>DNNHDW":M:Y#6_h     "hh&&x'7X&r hh&&x':&u HHOOI~~DOI HHOOD$<$O>OVZO[ HHOOD$<$O>OVZO[ hh))"-  588??4>>4<J>41Q5I$!$>J !6HX lr S Ks D.D D"D)rrrrrrrrrrrrrrrrr) r@rTNTrư>TrFNr)rUrVrWrXrYrrr_available_processorsr}r&r~rrgr#rrrzr[rrs@r/rrJsn 98:TU *..(,&&+1&1&1& 1&  1&  1&1&!4Z1&1&1&1&"&1&1& $1&1&l6:.2EICG || %||d2 t+    ell :;dB  ellELL89D@  r1rc7H^\rSrSrSrS9S\S\S\S\S\S\S \S \S \S \S \S\S\S\S\S\S\S\S\4&U4Sjjjr \ S\ RS\ RS\S\ \ RS44Sj5rS:S\ RS\ RS \ RS!\ RS\ RS"\ RS#\ RS$\ RS%\ RS&\ RS'\ RS-S(\ RS-S)\ \ R\ R4S-S*\ \ R\ R4S-S+\ \ R\ R4S-S,\ \ R\ R4S-S-\ RS-S.\ RS-S/\ RS-S0\ RS-S1\ RS-S2\ RS-S3\S4\S5\ RS-S6\S-S\ R46S7jjrS8rU=r$);LTX2VideoTransformerBlockiam Transformer block used in [LTX-2.0](https://huggingface.co/Lightricks/LTX-Video). Args: dim (`int`): The number of channels in the input and output. num_attention_heads (`int`): The number of heads to use for multi-head attention. attention_head_dim (`int`): The number of channels in each head. qk_norm (`str`, defaults to `"rms_norm"`): The normalization layer to use. activation_fn (`str`, defaults to `"gelu-approximate"`): Activation function to use in feed-forward. eps (`float`, defaults to `1e-6`): Epsilon value for normalization layers. r num_attention_headsattention_head_dimr audio_dimaudio_num_attention_headsaudio_cross_attention_dimvideo_gated_attnvideo_cross_attn_adalnaudio_gated_attnaudio_cross_attn_adalnr activation_fnattention_biasattention_out_biasrrrperturbed_attnc>[TU]5 UUlU(a[nO[n[ UUUS9Ul[UUUUUSUU UU U"5S9 Ul[ UUUS9Ul [UUUUUSUU UU U"5S9 Ul [ UUUS9Ul [UUUUUUUU UU U"5S9 Ul [ UUUS9Ul [UUUUUUUU UU U"5S9 Ul[ UUUS9Ul[UUUUUUUU UU U"5S9 Ul[ UUUS9Ul[UUUUUUUU UU U"5S9 Ul[ UUUS9Ul[)XS9Ul[ UUUS9Ul[)X^S9UlXlXlUR0(aSOSnUR2(aSOSn[4R6"[8R:"UU5US-- 5Ul[4R6"[8R:"UU5US-- 5UlU =(d U Ul UR@(a`[4R6"[8R:"SU55Ul![4R6"[8R:"SU55Ul"[4R6"[8R:"S U55Ul#[4R6"[8R:"S U55Ul$g) Nr) rrrrrerrrrrr) rrrrrrerrrrr)r r|?r)%rfrgrrrrnorm1rattn1 audio_norm1 audio_attn1norm2attn2 audio_norm2 audio_attn2audio_to_video_normaudio_to_video_attnvideo_to_audio_normvideo_to_audio_attnnorm3rff audio_norm3audio_ffrrri Parameterr#randnscale_shift_tableaudio_scale_shift_tablecross_attn_adalnprompt_scale_shift_tableaudio_prompt_scale_shift_table&video_a2v_cross_attn_scale_shift_table&audio_a2v_cross_attn_scale_shift_table)rnr rrrrraudio_attention_head_dimrrrrrrrrrrrrrattn_processor_clsvideo_mod_param_numaudio_mod_param_numros r/rg"LTX2VideoTransformerBlock.__init__s . , !; !< Sc>PQ "%(' $'"2(*   #9#J\](+.- $'"2(*  Sc>PQ " 3%(''"2(*   #9#J\]( 9+.-'"2(*   $+3CL^#_ #0 )+.-'"2(* $  $+9#Rd#e #0 #+.-'"2(* $  Sc>PQ c?"9#J\]#IK '=#&<##'#>#>aA#'#>#>aA!#ekk:Ms.SVY[^V^.^!_')||EKK@SU^4_bkmpbp4p'q$!7 P:P  ,.LLQ9L,MD )24,,u{{1i?X2YD /79ll5;;qRUCV6W368ll5;;qR[C\6]3r1rtembrsr.cURSnUSRUR5URX!RSUS5-nUR SS9nU$)NrNNr4rrr)r9r'devicer:r")rr%rsnum_ada_params ada_values ada_paramss r/get_mod_params(LTX2VideoTransformerBlock.get_mod_paramsHsh+003&z255dkkBT\\  1 ~rF   &&1&- r1Nraudio_hidden_statesraudio_encoder_hidden_states temb_audiotemb_ca_scale_shifttemb_ca_audio_scale_shift temb_ca_gatetemb_ca_audio_gate temb_prompttemb_prompt_audiovideo_rotary_embaudio_rotary_embca_video_rotary_embca_audio_rotary_embencoder_attention_maskaudio_encoder_attention_maskself_attention_maskaudio_self_attention_maska2v_cross_attention_maskv2a_cross_attention_maskuse_a2v_cross_attentionuse_v2a_cross_attentionrrc URS5nURURUU5nUSSunnnn n!n"UR(a USSun#n$n%UR U5n&U&SU--U-n&U&SU US.n'UR (a UU'S'UU'S'UR "S 0U'D6n(UU(U--nURURUU5n)U)SSun*n+n,n-n.n/UR(a U)SSun0n1n2URU5n3U3SU+--U*-n3U3SUUS.n4UR (a UU4S'UU4S'UR"S 0U4D6n5UU5U,--nUR(aDURURU U5n6U6un7n8URURU U5n9U9un:n;URU5n&UR(a U&SW$--W#-n&UR(a USW8--W7-nURU&USUS9n(UR(aU(W%-n(UU(-nUR!U5n3UR(a U3SW1--W0-n3UR(a USW;--W:-nUR#U3USUS9n5UR(aU5W2-n5UU5-nU(dU(GaUR%U5n&UR'U5n3UR(SS 2SS24nURU=U U5n?U>un@nAnBnCU?SR+S 5nDUR,SS 2SS24nEUR,S S2SS24nFURUEUU5nGURUFU U5nHUGunInJnKnLUHSR+S 5nMU(amU&SW@R+S 5--WAR+S 5-nNU3SWIR+S 5--WJR+S 5-nOUR/UNUOUUUS 9nPUWDUP--nU(amU&SWBR+S 5--WCR+S 5-nNU3SWKR+S 5--WLR+S 5-nOUR1UOUNUUUS 9nQUWMUQ--nUR3U5SU!--U -n&UR5U&5nRUURU"--nUR7U5SU.--U--n3UR9U35nSUUSU/--nX4$) Nrr|rr4)rrrrrr)rrrr3r)rrrrrT)sizer,rrrrrrrr r rrrr r r rrrrsqueezerrrrrrr)Trnrr.rr/r%r0r1r2r3r4r5r6r7r8r9r:r;r<r=r>r?r@rArBrrrsvideo_ada_params shift_msa scale_msagate_msa shift_mlp scale_mlpgate_mlp shift_text_q scale_text_q gate_text_qnorm_hidden_statesvideo_self_attn_argsattn_hidden_statesaudio_ada_paramsaudio_shift_msaaudio_scale_msaaudio_gate_msaaudio_shift_mlpaudio_scale_mlpaudio_gate_mlpaudio_shift_text_qaudio_scale_text_qaudio_gate_text_qnorm_audio_hidden_statesaudio_self_attn_argsattn_audio_hidden_statesvideo_prompt_ada_params shift_text_kv scale_text_kvaudio_prompt_ada_paramsaudio_shift_text_kvaudio_scale_text_kvvideo_per_layer_ca_scale_shiftvideo_per_layer_ca_gatevideo_ca_ada_paramsvideo_ca_gate_paramvideo_a2v_ca_scalevideo_a2v_ca_shiftvideo_v2a_ca_scalevideo_v2a_ca_shifta2v_gateaudio_per_layer_ca_scale_shiftaudio_per_layer_ca_gateaudio_ca_ada_paramsaudio_ca_gate_paramaudio_a2v_ca_scaleaudio_a2v_ca_shiftaudio_v2a_ca_scaleaudio_v2a_ca_shiftv2a_gatemod_norm_hidden_statesmod_norm_audio_hidden_statesa2v_attn_hidden_statesv2a_attn_hidden_states ff_outputaudio_ff_outputsT r/rz!LTX2VideoTransformerBlock.forwardSsx:#''*  ..t/E/EtZXIYZ\[\I]F 9h 9h  & &6Fq6K 3L, !ZZ 6/1y=AIM0%) 01     8I !4 54A  1!ZZ?*>?%(:X(EE  ..t/K/KZYcd Ra  k./?\j  & &HXYZ[\H] E  24E#'#3#34G#H #;q??R#SVe#e 6%) 07     8I !4 54A  1#'#3#3#K6J#K 14L~4]]  &*&9&9$:W:WYdfp&q #+B (M=&*&9&9335F ' #8O 4 !4"ZZ 6  & &!3q<7G!HE!E2"#5<<#=>E#E2$!& t 3%E2&',llT&9'E2(#\\D0)E2*$)<<$#6+E2,#(,,"5-E2.#(,,"5/E20"&1E22"&3E24!<<$.5E26d{7E28 9E2E2r1rc!^\rSrSrSrS#S\S\S\S\S\S\S \S \S \\S 4S \S\S\S\ S\S\SS4 U4Sjjjr S$S\S\S\S\S\ RS\S\ R4SjjrS%S\S\S\ RS\S\ R4 SjjrSrS&S \ RS\\ R-S-S\\ R\ R44S!jjrS"rU=r$)'LTX2AudioVideoRotaryPosEmbedia` Video and audio rotary positional embeddings (RoPE) for the LTX-2.0 model. Args: causal_offset (`int`, *optional*, defaults to `1`): Offset in the temporal axis for causal VAE modeling. This is typically 1 (for causal modeling where the VAE treats the very first frame differently), but could also be 0 (for non-causal modeling). r patch_size patch_size_tbase_num_frames base_height base_width sampling_rate hop_length scale_factors.theta causal_offsetmodalitydouble_precisionrrrNc>[TU]5 XlX lX0lUS;a[ SU<S35eXlX@lXlXPl X`l Xpl Xl [U5[U5- [U S5- UlXlXlXlXlUR$S;a[ SU S35eXlg)N)rrz rope_type=z9 not supported. Choose between 'interleaved' and 'split'.r)videoaudioz Modality z@ is not supported. Supported modalities are `video` and `audio`.)rfrgr rrr<rrrrrrrr&audio_latents_per_secondrrrrr)rnr rrrrrrrrrrrrrrros r/rg%LTX2AudioVideoRotaryPosEmbed.__init__%s$ $( 4 4 |+def f".#6 '$+$(-m(%/d6G6Gu}}ek  "//24(+=+==@RRXX]^X_%7$:L:LL !2!226HH $'9'99"UU/SS9n[R>"UU/SS9nURSnURSnURAUUURBS5nURAUUURBS5n[RD"USS5n [RD"USS5nW W4$s snf)Nr4r3rrrrrrrg?)rrstepsr(r(r7r6rrr)axis)#r(r9r8chunkrErrrrr#r$ranger'rfloat64rpowrlinspacer pir= transposer%rr)repeat_interleaver* ones_like zeros_likecat concatenater:rr;)rnrr( num_pos_dims coords_start coords_end max_positionsirnum_rope_elems freqs_dtype pow_indicesr cos_freqs sin_freqs cos_padding sin_paddingexpected_freqs current_freqspad_sizecos_freqsin_freqrArCs r/rz$LTX2AudioVideoRotaryPosEmbed.forwards(6==||A  ;;! '-||A2|'> $L"/36F^^B'F ==G #!1143C3CT__UM ]]g %!113M{{U>] * 55aR5@I 55aR5@Ixx.(A-#ooi1>Y>@Y>Y8Y.Z[ #..yA?ZNAZ?Z9Z/[\ !II{&>BG !II{I&>BG 4##1^^w &!XX]N!KKOM% 5Hyy{Hyy{H1}#oohq!YhY.GH #..x1ixi/HI  ,,k8-D2N ,,k8-D2Nq!Aq!A''1d.F.FKH''1d.F.FKHxA6IxA6I)##mZs P&)rrrrrr rrrrrrrrrr)r4r4r>r r@r4rTrr)8@)r)N)rUrVrWrXrYr}rr&rr~rgr#r(rrrrrzr[rrs@r/rrs!")4!%&#%!/1 /1/1 /1  /1  /1/1/1/1S#X/1/1/1/1/1/1 !!/1" #/1/1pDDD D  D  DD DV 444 4  4  4l>IMK$llK$,/%,,,>,EK$ u||U\\) *K$K$r1rcV^\rSrSrSrSrS/rS/r\"SSSS 9\"SSSS 9\"SS SS 9S .\"SSSS 9\"SSSS 9S .\ "SSS 9S.r \ STS\ S\ S-S\ S\ S\ S\ S\ S\ \ \ \ 4S\ S\ S\ S\S\S\ S\ S-S\ S \ S!\ S"\ S#\ S$\ S%\ S&\ S'\ S(\S)\S*\ S+\S,\S-\S.\S/\ S0\S1\S2\S3\S4\ S5\ S6\ S7\S8\S9S4TU4S:jjj5r\"S;5SUS<\R(S=\R(S>\R(S?\R(S@\R*SA\R*S-SB\R(S-SC\R(S-SD\R(S-SE\R(S-SF\ S-SG\ S-SH\ S-SI\SJ\ S-SK\R(S-SL\R(S-SM\SN\\ S-SO\R(S-SP\S;\\\4S-SQ\S9\R(40SRjj5rSSrU=r$)VLTX2VideoTransformer3DModeli&a A Transformer model for video-like data used in [LTX](https://huggingface.co/Lightricks/LTX-Video). Args: in_channels (`int`, defaults to `128`): The number of channels in the input. out_channels (`int`, defaults to `128`): The number of channels in the output. patch_size (`int`, defaults to `1`): The size of the spatial patches to use in the patch embedding layer. patch_size_t (`int`, defaults to `1`): The size of the tmeporal patches to use in the patch embedding layer. num_attention_heads (`int`, defaults to `32`): The number of heads to use for multi-head attention. attention_head_dim (`int`, defaults to `64`): The number of channels in each head. cross_attention_dim (`int`, defaults to `2048 `): The number of channels for cross attention heads. num_layers (`int`, defaults to `28`): The number of layers of Transformer blocks to use. activation_fn (`str`, defaults to `"gelu-approximate"`): Activation function to use in feed-forward. qk_norm (`str`, defaults to `"rms_norm_across_heads"`): The normalization layer to use. Tnormrr4rF) split_dim expected_dims split_outputr)rrr;)rr4) gather_dimr)ropeproj_out in_channels out_channelsNrrrrrvae_scale_factorspos_embed_max_posrr gated_attncross_attn_modaudio_in_channelsaudio_out_channelsaudio_patch_sizeaudio_patch_size_trr rraudio_pos_embed_max_posaudio_sampling_rateaudio_hop_lengthraudio_cross_attn_mod num_layersrrrrcaption_channelsrr rope_thetarope_double_precisionrtimestep_scale_multiplier$cross_attn_timestep_scale_multiplierrrrc+*>[T1U]5 U=(d UnU=(d UnXV-n+UU-n,[R"UU+5Ul[R"UU,5UlU)(a[ U U+S9Ul[ U U,S9UlU =(d UUl U (aSOSn-U(aSOSn.[U+U-SS9Ul [U,U.SS9Ul [U+SSS9Ul [U,SSS9Ul[U+SSS9Ul[U,SSS9Ul[R""[$R&"SU+5U+S -- 5Ul[R""[$R&"SU,5U,S -- 5UlUR(a [U+SSS9Ul[U,SSS9Ul[1U+UUU U U UU#U%S U$U(US 9 Ul[1U,UUUUUU/U#U%S U$U(US 9 Ul[7U U5n/[1UUUU/U U U#U%S U$U(US9 Ul[1UUUU/UUU#U%S U$U(US9 Ul[R<"[?U5V0s/sHHn0[AS&0SU+_SU_SU_SU_SU,_SU_SU_SU_SU _SU _SU_SU_SU_SU_SU!_SU"_S U_S!U_S"U(_S#U*_6PMJ sn05Ul![RD"U+S$SS%9Ul#[R"U+U5Ul$[RD"U,S$SS%9Ul%[R"U,U5Ul&SUl'gs sn0f)'N) in_features hidden_sizerr|F)r`rar3r4rrr) r rrrrrrrrrrrrr) r rrrrrrrrrrrr) r rrrrrrrrrrr) r rrrrrrrrrrrr rrrrrr rrrrrrrrrrrrrrrrT)(rfrgrirlproj_in audio_proj_inrcaption_projectionaudio_caption_projectionprompt_modulationr] time_embedaudio_time_embedav_cross_attn_video_scale_shiftav_cross_attn_audio_scale_shiftav_cross_attn_video_a2v_gateav_cross_attn_audio_v2a_gaterr#rrr prompt_adalnaudio_prompt_adalnrr audio_ropemaxcross_attn_ropecross_attn_audio_roperrrtransformer_blocks LayerNormnorm_outraudio_norm_outaudio_proj_outgradient_checkpointing)2rnrrrrrrrrrrrrrrrrrrr rrrrrrrrrrrrrrrrr rr r ruse_prompt_embeddingsrraudio_inner_dimvideo_time_emb_mod_paramsaudio_time_emb_mod_paramscross_attn_pos_embed_max_posrDros2 r/rg$LTX2VideoTransformer3DModel.__init__Ss\ #2{ /D3D'< 36NNyyi8 YY'8/J !&?L\js&tD #,E,/-D ) "0!G3G*8AQ!)=A1!0 &?[` !7 ,Eaf! 0F a50 ,0F A0 , -C a5- ) -C A- ) "$ekk!Y.G)UX..X!Y')||EKK?4SVegjVj4j'k$  ! ! 6yQRns tD &<U'D # 1!%-#!+'2 3  7'+3-'-.'2 9 "(++<>U'V$;)!%8#!'2 3 &B)'+8-''2 9 & " #%--0z*/ .+A-*!(;(:)<  .  /H .F/H&0,:&6,@$#0$2 (:!"!#$(?%&('($2),+/ # : YDUS  )\:  ll?Y^_ ii9KL&+#E s?ALattention_kwargsrr.rr/rqaudio_timestepsigma audio_sigmar;r<rrrraudio_num_frames video_coordsrisolate_modalitiesspatio_temporal_guidance_blocksruse_cross_timestep return_dictcUbUOUnUbUOUnU bBU RS:Xa2SU RUR5- S-n U RS5n U bBU RS:Xa2SU RUR5- S-n U RS5n UR S5nUc&UR R UXXRUS9nUc&URRUXR5nUR UURS9nURUURS9nURUSS2SS2SS24URS9nURUSS2SS2SS24URS9nURU5nURU5nURRURR - nUR#UR%5UURS9unnUR'US UR S 55nUR'US UR S 55nUR)UR%5UURS9un n!U R'US U R S 55n U!R'US U!R S 55n!UR*(aUR-UR%5UURS9un"n#UR/UR%5UURS9un$n#U"R'US U"R S 55n"U$R'US U$R S 55n$OS=n"n$U(aUR%5OUR%5n%UR1U%UURS9un&n#UR3U%U-UURS9un'n#U&R'US U&R4S 5n&U'R'US U'R4S 5n'U(aUR%5OUR%5n(UR7U(UURS9un)n#UR9U(U-UURS9un*n#U)R'US U)R4S 5n)U*R'US U*R4S 5n*URR:(afUR=U5nUR'US UR S 55nUR?U5nUR'US UR S 55nU=(d /n[AU5S:aUc[BRD"U45nUbURS:Xa USS2SS4nUb[BRF"US:H5OS n+[IU5n,[KURL5Hun-n.U-U,;aUOSn/U-U,;aU+OS n0[BRN"5(aJURP(a9URSU.UUUUUU U&U)U'U*U"U$UUUUU U SSSSU(+U(+U/U05upM~U."S'0S U_S U_S U_SU_SU_SU _SU&_SU)_SU'_SU*_SU"_SU$_SU_SU_SU_SU_SU _SU _SS_SS_SS_S S_S!U(+_S"U(+_S#U/_S$U0_6upM URTS%USS2SS2S4-n1U1SS2SS2S4U1SS2SS2S4n3n2URWU5nUSU3--U2-nURYU5n4URZS%U!SS2SS2S4-n5U5SS2SS2S4U5SS2SS2S4n7n6UR]U5nUSU7--U6-nUR_U5n8U(dU4U84$[aU4U8S&9$)(a Forward pass for LTX-2.0 audiovisual video transformer. Args: hidden_states (`torch.Tensor`): Input patchified video latents of shape `(batch_size, num_video_tokens, in_channels)`. audio_hidden_states (`torch.Tensor`): Input patchified audio latents of shape `(batch_size, num_audio_tokens, audio_in_channels)`. encoder_hidden_states (`torch.Tensor`): Input video text embeddings of shape `(batch_size, text_seq_len, self.config.caption_channels)`. audio_encoder_hidden_states (`torch.Tensor`): Input audio text embeddings of shape `(batch_size, text_seq_len, self.config.caption_channels)`. timestep (`torch.Tensor`): Input timestep of shape `(batch_size, num_video_tokens)`. These should already be scaled by `self.config.timestep_scale_multiplier`. audio_timestep (`torch.Tensor`, *optional*): Input timestep of shape `(batch_size,)` or `(batch_size, num_audio_tokens)` for audio modulation params. This is only used by certain pipelines such as the I2V pipeline. sigma (`torch.Tensor`, *optional*): Input scaled timestep of shape (batch_size,). Used for video prompt cross attention modulation in models such as LTX-2.3. audio_sigma (`torch.Tensor`, *optional*): Input scaled timestep of shape (batch_size,). Used for audio prompt cross attention modulation in models such as LTX-2.3. If `sigma` is supplied but `audio_sigma` is not, `audio_sigma` will be set to the provided `sigma` value. encoder_attention_mask (`torch.Tensor`, *optional*): Optional multiplicative text attention mask of shape `(batch_size, text_seq_len)`. audio_encoder_attention_mask (`torch.Tensor`, *optional*): Optional multiplicative text attention mask of shape `(batch_size, text_seq_len)` for audio modeling. num_frames (`int`, *optional*): The number of latent video frames. Used if calculating the video coordinates for RoPE. height (`int`, *optional*): The latent video height. Used if calculating the video coordinates for RoPE. width (`int`, *optional*): The latent video width. Used if calculating the video coordinates for RoPE. fps: (`float`, *optional*, defaults to `24.0`): The desired frames per second of the generated video. Used if calculating the video coordinates for RoPE. audio_num_frames: (`int`, *optional*): The number of latent audio frames. Used if calculating the audio coordinates for RoPE. video_coords (`torch.Tensor`, *optional*): The video coordinates to be used when calculating the rotary positional embeddings (RoPE) of shape `(batch_size, 3, num_video_tokens, 2)`. If not supplied, this will be calculated inside `forward`. audio_coords (`torch.Tensor`, *optional*): The audio coordinates to be used when calculating the rotary positional embeddings (RoPE) of shape `(batch_size, 1, num_audio_tokens, 2)`. If not supplied, this will be calculated inside `forward`. isolate_modalities (`bool`, *optional*, defaults to `False`): Whether to isolate each modality by turning off cross-modality (audio-to-video and video-to-audio) cross attention (for all blocks). Use for modality guidance in LTX-2.3. spatio_temporal_guidance_blocks (`list[int]`, *optional*, defaults to `None`): The transformer block indices at which to apply spatio-temporal guidance (STG), which shortcuts the self-attention operations by simply using the values rather than the full scaled dot-product attention (SDPA) operation. If `None` or empty, STG will not be applied to any block. perturbation_mask (`torch.Tensor`, *optional*): Perturbation mask for STG of shape `(batch_size,)` or `(batch_size, 1, 1)`. Should be 0 at batch elements where STG should be applied and 1 elsewhere. If STG is being used but `peturbation_mask` is not supplied, will default to applying STG (perturbing) all batch elements. use_cross_timestep (`bool` *optional*, defaults to `False`): Whether to use the cross modality (audio is the cross modality of video, and vice versa) sigma when calculating the cross attention modulation parameters. `True` is the newer (e.g. LTX-2.3) behavior; `False` is the legacy LTX-2.0 behavior. attention_kwargs (`dict[str, Any]`, *optional*): Optional dict of keyword args to be passed to the attention processor. return_dict (`bool`, *optional*, defaults to `True`): Whether to return a dict-like structured output of type `AudioVisualModelOutput` or a tuple. Returns: `AudioVisualModelOutput` or `tuple`: If `return_dict` is `True`, returns a structured output of type `AudioVisualModelOutput`, otherwise a `tuple` is returned where the first element is the denoised video latent patch sequence and the second element is the denoised audio latent patch sequence. Nrr4gr)rrrxrFrr.rr/r%r0r1r2r3r4r5r6r7r8r9r:r;r<r=r>r?r@rArBrrr')rRrSrT)1r8r'r(r=rDrrr(rrrrrrconfigr r rr%rrrrrrrr9rrr&rrrr#zerosrr enumerater is_grad_enabledr%_gradient_checkpointing_funcrr"rrr#r$rP)9rnrr.rr/rqr-r.r/r;r<rrrrr0r1rr2r3rr4r,r5rsr7r8video_cross_attn_rotary_embaudio_cross_attn_rotary_emb%timestep_cross_attn_gate_scale_factorr%ryr0audio_embedded_timestepr5rDr6video_ca_timestepvideo_cross_attn_scale_shiftvideo_cross_attn_a2v_gateaudio_ca_timestepaudio_cross_attn_scale_shiftaudio_cross_attn_v2a_gater stg_blocks block_idxblockblock_perturbation_maskblock_all_perturbedscale_shift_valuesrscaleoutputaudio_scale_shift_values audio_shift audio_scale audio_outputs9 r/rz#LTX2VideoTransformer3DModel.forward)s H,:+E8%0%">z2OfOkOklnOo"p  ! !!.. J]EXEX/NK$(#:#:##%*K^KdKd$;$ q&**:r;;K;KB;OPK 1 6 6z2GXG]G]^`Ga b .2 2K+6HK//1XM]M]M_*.*N*N !&,,+O+ '$a (,'H'H  E E!&,,(I( $!1 (D'H'H 8>>rB( $%>$B$B:rSlSrSrsuSv$w!/AEMMO~G]G]G_*.*N*N !,22+O+ '$a (,'H'H  E E!,22(I( $!1 (D'H'H 8>>rB( $%>$B$B:rSlSrSrsuSv$w! ;; , ,$($;$;$>z2}OaOabdOe$f !*.*G*GHc*d '*E*J*JB 3 8 8 <+ ' +J*OR' . /! 38I8Q % ZM :   (->-C-Cq-H 1!T4- @ =N=Z "3q"89`e 89 )$*A*A B Iu;D ;R&7X\ #3< 3J-PU $$&&4+F+F595V5V!')/00--%$$//*0****+'762 2<6;6"/6(;6+@61L 6  6 * 6)E6/K6";6(A6!,6'86&66&66)D6 )D!6",B#6$2N%6&)-'6(/3)6*.2+6,.2-6.1C,B/601C,B162'>364#6562 2G!CB"33J?BSTUWXZ^T^B__)!Q'24Fq!Qw4Ou m4 %U3e; }-#'#?#? #KNefgijlpfpNq#q #;Aq!G#DF^_`bcef_fFg[ "112EF1Q_E S**+>? L) )%V,OOr1)rr#rr$rrrrrrrrrrrr%r"rrrrrrrr )*rSr4r4rrSirrrrFFrSrSr4r4rrrr3rrrFF0rrFriTTrTr4rUrTF)NNNNNNNNrNNNFNNFNT)rUrVrWrXrY _supports_gradient_checkpointing _skip_layerwise_casting_patterns_repeated_blocksrr_cp_planrr}rr~rr&rgr r#r LongTensorlistrrrzr[rrs@r/rr&s4(,$(.x$342AQ]bc%9AUVej%k&:QVWfk&l $aqtT#aqtT *QaH H#&#%"%#'2=!# $!$), !"#)+(*)-"#')#( #!&%*/.(- $##'#&*)-48&"$WS,S,DjS, S,  S, ! S, S,!S,!c3/S,S,S,S,S,S,S,  $J!S,"#S,$ %S,&$''S,(#&)S,*$'+S,, -S,."%/S,0!1S,23S,45S,6#7S,89S,:;S,<=S,>"&?S,@AS,BCS,DES,F!GS,HIS,J $KS,LMS,N$'OS,P/2QS,RSS,VWS,X YS,S,j()37%)+/6:<@!%! '+,0,0#(<@15#(26 1rP||rP#\\rP %|| rP &+\\ rP "" rP((4/rP||d"rP\\D(rP!& t 3rP',llT&9rP$JrPd rPTzrPrP *!rP"llT)#rP$llT)%rP&!'rP(*.cT)9)rP*!<<$.+rP,!-rP.sCx.4//rP01rP2 3rP*rPr1r)5r dataclassesrtypingrr#torch.nnriconfiguration_utilsrrloadersrr utilsr r r r _modeling_parallelrr attentionrrrattention_dispatchr cache_utilsr embeddingsrrmodeling_utilsr normalizationr get_loggerrUrrrr0rNrPModuler]rrrrrrrTr1r/rksa ! B?LLLII6$]'#   H %ELLu||U\\?Y9Z_d_k_k&ell&5u||9S3T&Y^YeYe&R  !Z !  ! &LRYY&LRSSl``FOEHHOO%9Od|2 |2~ H$299H$VvP ^-CEUWavPr1