3j2SSKrSSKJr SSKJrJr SSKJr SSKJ r J r SSK J r J r SSKJrJr SS KJr SS KJr SS KJrJr \R0"\5r"S S \R65r"SS\\ \5rg)N) ConfigMixinregister_to_config)logging)AttentionMixin FeedForward) AttentionCogVideoXAttnProcessor2_0)&CogView3CombinedTimestepSizeEmbeddingsCogView3PlusPatchEmbed)Transformer2DModelOutput) ModelMixin)AdaLayerNormContinuous%CogView3PlusAdaLayerNormZeroTextImagec ^\rSrSrSrSS\S\S\S\4U4SjjjrS\RS \RS \RS \ \R\R44S jr S r U=r $)CogView3PlusTransformerBlock a Transformer block used in [CogView](https://github.com/THUDM/CogView3) model. Args: dim (`int`): The number of channels in the input and output. num_attention_heads (`int`): The number of heads to use for multi-head attention. attention_head_dim (`int`): The number of channels in each head. time_embed_dim (`int`): The number of channels in timestep embedding. dimnum_attention_headsattention_head_dimtime_embed_dimc >[TU]5 [XAS9Ul[ UUUUSSSS[ 5S9 Ul[R"USSS9Ul [R"USSS9Ul [XS S 9Ul g) N) embedding_dimrT layer_normFư>) query_dimheadsdim_headout_dimbiasqk_normelementwise_affineeps processorgh㈵>)r#r$zgelu-approximate)rdim_out activation_fn) super__init__rnorm1r r attn1nn LayerNormnorm2 norm2_contextr ff)selfrrrr __class__s p/home/wildlama/miniconda3/lib/python3.13/site-packages/diffusers/models/transformers/transformer_cogview3plus.pyr)%CogView3PlusTransformerBlock.__init__/s :a %' $/1   \\#%TJ \\#%TR#BTU hidden_statesencoder_hidden_statesembreturnc URS5nURXU5u nnnnn n n n n nURXZS9unnXRS5U--nX+RS5U--nUR U5nUSUSS2S4--USS2S4-nUR U5n U SU SS2S4--U SS2S4-n [ R"X/SS9nURU5nXRS5USS2US24--nX.RS5USS2SU24--nUR[ R:XaURSS5nUR[ R:XaURSS5nX4$)N)r6r7)ri i) sizer*r+ unsqueezer.r/torchcatr0dtypefloat16clip)r1r6r7r8text_seq_lengthnorm_hidden_statesgate_msa shift_mlp scale_mlpgate_mlpnorm_encoder_hidden_states c_gate_msa c_shift_mlp c_scale_mlp c_gate_mlpattn_hidden_statesattn_encoder_hidden_states ff_outputs r3forward$CogView3PlusTransformerBlock.forwardKs 044Q7 JJ}S A      &    :>,:D: 66&(:(:1(=@R(RR 58L8LQ8ORl8l l"ZZ 6/1yD7I3IJYWXZ^W^M__%)%7%78M%N"%?1{STVZSZG[C[%\_jklnrkr_s%s"#YY(B'W]^_GG./ %(:(:1(= !_M]J]@^(^^ 58L8LQ8OR[\]_o`o_o\oRp8p p   %-- /)..vu=M & &%-- 7$9$>$>vu$M !33r5)r+r0r*r.r/)i @() __name__ __module__ __qualname____firstlineno____doc__intr)r>TensortuplerQ__static_attributes__ __classcell__r2s@r3rr s  #%"$! V V!V V  VV804||04 %||04\\ 04 u||U\\) * 0404r5rc\^\rSrSrSrSrSS/rSS/r\SS\ S \ S \ S \ S \ S \ S\ S\ S\ S\ S\ 4U4Sjjj5r SS\ RS\ RS\ RS\ RS\ RS\ RS\S\\ R\-4SjjrSrU=r$) CogView3PlusTransformer2DModel~a! The Transformer model introduced in [CogView3: Finer and Faster Text-to-Image Generation via Relay Diffusion](https://huggingface.co/papers/2403.05121). Args: patch_size (`int`, defaults to `2`): The size of the patches to use in the patch embedding layer. in_channels (`int`, defaults to `16`): The number of channels in the input. num_layers (`int`, defaults to `30`): The number of layers of Transformer blocks to use. attention_head_dim (`int`, defaults to `40`): The number of channels in each head. num_attention_heads (`int`, defaults to `64`): The number of heads to use for multi-head attention. out_channels (`int`, defaults to `16`): The number of channels in the output. text_embed_dim (`int`, defaults to `4096`): Input dimension of text embeddings from the text encoder. time_embed_dim (`int`, defaults to `512`): Output dimension of timestep embeddings. condition_dim (`int`, defaults to `256`): The embedding dimension of the input SDXL-style resolution conditions (original_size, target_size, crop_coords). pos_embed_max_size (`int`, defaults to `128`): The maximum resolution of the positional embeddings, from which slices of shape `H x W` are taken and added to input patched latents, where `H` and `W` are the latent height and width respectively. A value of 128 means that the maximum supported height and width for image generation is `128 * vae_scale_factor * patch_size => 128 * 8 * 2 => 2048`. sample_size (`int`, defaults to `128`): The base resolution of input latents. If height/width is not provided during generation, this value is used to determine the resolution as `sample_size * vae_scale_factor => 128 * 8 => 1024` T patch_embednormrr patch_size in_channels num_layersrr out_channelstext_embed_dimr condition_dimpos_embed_max_size sample_sizec >[T U]5 X`lXT-UlSU -Ul[ UURUUU S9Ul[UU URURS9Ul[R"[U5V s/sHn [URUUUS9PM sn 5Ul [URUSSS9Ul[R "URX-UR-SS 9UlSUlgs sn f) N)rg hidden_sizerftext_hidden_sizerl)rrkpooled_projection_dim timesteps_dim)rrrrFr)rconditioning_embedding_dimr#r$T)r!)r(r)ri inner_dimrrr rdr time_condition_embedr, ModuleListrangertransformer_blocksrnorm_outLinearproj_outgradient_checkpointing)r1rfrgrhrrrirjrrkrlrm_r2s r3r)'CogView3PlusTransformer2DModel.__init__s (,A&+]%:"1#!+1  %K('"&"<"<.. % !#%--z* +A -(;'9#1  +  # /..'5$    $..*2IDL]L]2]dhi &+#' s D r6r7timestep original_size target_size crop_coords return_dictr9c$URSSupURSn URX5nURX4XVUR5n USS2SU 24nUSS2U S24n[ UR 5HRup[ R"5(a)UR(aURU UUU 5upMHU "UUU S9upMT URX5nURU5nURRnX-nX-n URURSXURX4S9n[ R "SU5nURURSURX-X-4S9nU(dU4$[#US9$) a The [`CogView3PlusTransformer2DModel`] forward method. Args: hidden_states (`torch.Tensor`): Input `hidden_states` of shape `(batch size, channel, height, width)`. encoder_hidden_states (`torch.Tensor`): Conditional embeddings (embeddings computed from the input conditions such as prompts) of shape `(batch_size, sequence_len, text_embed_dim)` timestep (`torch.LongTensor`): Used to indicate denoising step. original_size (`torch.Tensor`): CogView3 uses SDXL-like micro-conditioning for original image size as explained in section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952). target_size (`torch.Tensor`): CogView3 uses SDXL-like micro-conditioning for target image size as explained in section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952). crop_coords (`torch.Tensor`): CogView3 uses SDXL-like micro-conditioning for crop coordinates as explained in section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952). return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~models.transformer_2d.Transformer2DModelOutput`] instead of a plain tuple. Returns: `torch.Tensor` or [`~models.transformer_2d.Transformer2DModelOutput`]: The denoised latents using provided inputs as conditioning. Nr;)r6r7r8r)shapeznhwcpq->nchpwq)sample)rrdrvr@ enumerateryr>is_grad_enabledr}_gradient_checkpointing_funcrzr|configrfreshaperieinsumr)r1r6r7rrrrrheightwidthrCr8 index_blockblockrfoutputs r3rQ&CogView3PlusTransformer2DModel.forwardsL&++BC0 /55a8((  ''[n[no -a1A/1A.A B%a)9&9: "+D,C,C"D K$$&&4+F+F7;7X7X!) 84 48="/*?84 4#E m9  m4 [[++ %#%-- &&q)6$:K:KZd.  %5}E && &&q)4+<+QSXSef' 9 'v66r5) r}rurzrirdrrr|rvry) rrTrSrirUr)T)rVrWrXrYrZ _supports_gradient_checkpointing _skip_layerwise_casting_patterns_no_split_modulesrr[r)r>r\ LongTensorboolr]rrQr^r_r`s@r3rbrb~sZ D(,$(5v'>$79QR"$#%"! "%9,9,9, 9,  9, ! 9,9,9,9,9, 9,9,9,F!S7||S7 %||S7"" S7 || S7 \\ S7\\S7S7 u|| 7 7S7S7r5rb)r>torch.nnr,configuration_utilsrrutilsr attentionrr attention_processorr r embeddingsr r modeling_outputsrmodeling_utilsr normalizationrr get_loggerrVloggerModulerrbr5r3rs\  B3FW7'Y   H %[4299[4|v7Zv7r5