3jW0SSKJr SSKJr SSKJr SSKJr SSKJ r J r J r SSK J r SSKJr SS KJr SS KJr SS KJrJrJr SS KJr SS KJr SSKJr SSKJ r J!r!J"r"J#r#J$r$ SSK%J&r& SSK'J(r( \#"5(aSSKr"SS\ RR5r*\!"SS\55r+\"SS\55r,"SS\ RR5r-Sr.Sr/\!"SS 9"S!S"\+55r0\!"S#S 9\"S$S%\555r1\!"S&S 9"S'S(\+\55r2/S)Qr3g)*)Callable) dataclass)pi)Optional)Tensorbroadcast_tensorsnn)initialization)ACT2FN)Cache)GenerationMixin)BaseModelOutputWithPastBaseModelOutputWithPooling ModelOutput)ROPE_INIT_FUNCTIONS)PreTrainedModel)Unpack)TransformersKwargsauto_docstringcan_return_tupleis_torch_availabletorch_compilable_check) AutoModel)MusicFlamingoConfigNc ^\rSrSr%Sr\R \S'SS\4U4Sjjjr \ SS\S-S\ SS \ S-S \ S \44S jj5r\R "5S \S \ S \ \\44Sj5rSrSrU=r$)MusicFlamingoRotaryEmbedding.aRotary time embedding module used by MusicFlamingo checkpoints. This is a checkpoint-faithful integration, not a direct implementation of the RoTE formulation described in (Goel et al., 2024): https://arxiv.org/abs/2410.12109. It applies axial rotary embeddings over the window index within each audio sample and the encoder time index within each window, then modulates both axes with absolute timestamps in seconds. inv_freqNconfigc>[TU]5 URUlURUlXlUR R SUlURnURS:wa[URnU"UR U5uo@l URSUSS9 URSUR5SS9 URUR5nURSUSS9 g)N rope_typedefaultr!F) persistentoriginal_inv_freqposition_angles)super__init__max_position_embeddingsmax_seq_len_cachedoriginal_max_seq_lenr"rope_parametersr$compute_default_rope_parametersrattention_scalingregister_bufferclone_compute_position_anglesr!)selfr"device rope_init_fnr!r( __class__s r/home/wildlama/miniconda3/lib/python3.13/site-packages/transformers/models/musicflamingo/modeling_musicflamingo.pyr*%MusicFlamingoRotaryEmbedding.__init__9s "("@"@$*$B$B! 44[A!%!E!E >>Y &.t~~>L+7 V+L(( ZeD 0(..2BuU77 F .ERr5z torch.deviceseq_lenreturnz torch.Tensorc jURSnURRSS5n[USS5=(d URUR-n[ XT-5nSnSU[ R"SUS[ RS9RU[ RS 9U- -- nX4$) aH Computes the inverse frequencies according to the original RoPE implementation Args: config ([`~transformers.PreTrainedConfig`]): The model configuration. device (`torch.device`): The device to use for initialization of the inverse frequencies. seq_len (`int`, *optional*): The current sequence length. Unused for this type of RoPE. Returns: Tuple of (`torch.Tensor`, `float`), containing the inverse frequencies for the RoPE embeddings and the post-processing scaling factor applied to the computed cos/sin (unused in this type of RoPE). rope_thetapartial_rotary_factorg?head_dimNrrdtyper5rB) r.getgetattr hidden_sizenum_attention_headsinttorcharangeint64tofloat) r"r5r;baser?r@dimattention_factorr!s r8r/ ]]_(&(3(5;P((&88r:rc~^\rSrSr%\\S'SrSrSrSr S/r Sr Sr Sr \R"5U4Sj5rS rU=r$) MusicFlamingoPreTrainedModelr"model)audiotextTNpast_key_valuesc>[TU]U5 [U[5(a=UR UR 5n[ R"URU5 ggrj) r) _init_weights isinstancerr3r!initcopy_r()r4module buffer_valuer7s r8r*MusicFlamingoPreTrainedModel._init_weightssK f% f: ; ;!::6??KL JJv--| <