3jjSSKJr SSKJr SSKJr SSKJr SSKJ r SSK J r J r J r Jr SS KJr SS KJr \ "5(a:SS Kr\R(R+\ 5r \R(R+\ 5r \R,"\5rS"S#S jjrS$SjrS%SjrS%SjrSrSr\ "5(aS\R>RAS\SSS9 \R>RCS\5 \R>RES\\S9 S&Sjr#S%Sjr$S"S'Sjjr%S$Sjr&"SS\ 5r'\'"5r(S(Sjr)S)\(SS S SS .S*S!jjjr*g )+) annotations)Callable)wraps)logging)GeneralInterface)is_torch_availableis_torch_greater_or_equalis_torch_less_or_equalis_torchdynamo_compiling)deepgemm_bf16_experts_forward)sonicmoe_experts_forwardNFcU(a6[R"URS5U5RS5nO4[R"XRS55RS5nUbUR U5 U$)aBatched linear layer supporting optional bias and transposed weights. Args: input (`torch.Tensor`): Input tensor of shape (batch_size, input_dim). weight (`torch.Tensor`): Weight tensor of shape (batch_size, output_dim, input_dim) if transposed is `False`, else of shape (batch_size, input_dim, output_dim). bias (`torch.Tensor`, *optional*): Bias tensor of shape (batch_size, output_dim). Default is `None`. is_transposed (`bool`, *optional*, defaults to `False`): Whether the weight tensor is transposed. Returns: `torch.Tensor`: Output tensor of shape (batch_size, output_dim). r )torchbmm unsqueezesqueezeadd_)inputweightbias is_transposedouts W/home/wildlama/miniconda3/lib/python3.13/site-packages/transformers/integrations/moe.py_batched_linearrTse*ii*F3;;A>ii 34< )v\$))A,Geflfqfqrsfteu+vv ) ::a=FKKN * (A}V[[QR^DTU * ::%++u{{3 3h7[\`\f\f[g5hh 3 ;;uzz!}fkk!nU\\QVQ\Q\ ]]rcHURUSUS5 USUlg)zjSaves input and weight for backward; offs is stored directly as it is a non-differentiable integer tensor.rr rN)save_for_backwardrN)ctxinputsrOs r"_grouped_mm_fallback_setup_contextras%&)VAY/ayCHrcURup#[R"U5n[R"U5nSn[URR 55HZupxXh:XaM [R "XUX7RXFUS9 [R "X&URXUXWS9 UnM\ XES4$)zuBackward pass for `_grouped_mm_fallback`. Computes grad_input and grad_weight per expert group; offs has no gradient.rrIN) saved_tensorsr zeros_likerKrNrLrMT) r_ grad_outputrr grad_input grad_weightrPrQrRs r_grouped_mm_fallback_backwardris%%ME!!%(J""6*K ECHHOO-. <  3'*3:OP S!##[s%;P / D ((rz!transformers::grouped_mm_fallbackz4(Tensor input, Tensor weight, Tensor offs) -> Tensor) mutates_argsschema) setup_contextcZ[5(aUR[R:wdWURR S:Xa>[ SSS9(a/UR5S-S:wdUR5S-S:wagURR S:Xa[[RRS 5(a,[RRUR5S :$[[S 5(ag[S SS9(a,[RRUR5S :$[RRUR5S :$g[[RRS 5=(d [[S 5$)a Check if torch.nn.functional.grouped_mm or torch._grouped_mm can be used based on availability and compatibility with torch.compile. Args: input (`torch.Tensor`): Input tensor of shape (S, input_dim). weight (`torch.Tensor`): Weight tensor of shape (num_experts, input_dim, output_dim). offs (`torch.Tensor`): Offsets tensor indicating the boundaries of each group in the input tensor. Returns: `bool`: True if grouped_mm can be used, False otherwise. cpuz2.10.0T) accept_devrFcuda grouped_mm)r _grouped_mmz2.9) r)r r5rbfloat16rHtyper data_ptrhasattrnn functionalrrget_device_capabilityr r[s r_can_use_grouped_mmr~ s& !""v||u~~'E e# "8 = __  #q (ENN,r? expert_ids_gpermselected_hidden_states_gsample_weights_g histc_inputtokens_per_expertoffsets sentinel_maskr@rArBrCinv_permrDs rgrouped_mm_experts_forwardr|s  ! !F  $I##A&J##B'J#**2.N$$R(JJ/L,Y->?%+ +1++*G,$$&\M]M]M_K K6F6FASWScScfgSghll,!5;;GG""%5%55@@DMD,,q01 }},,BF--$00>UY<<=A]]$++L9PT))-= "2G/aeasasH  }}##H-;;x(~~;?==d)),7dO"G/QUQcQcH .88<HTN)L '++J:NRRWXRY  ! !-"5"5 66rc>^\rSrSrSr\\\\S.r SU4Sjjr Sr U=r $)ExpertsInterfaceiz;Interface for registering custom experts forward functions.)deepgemm batched_mmrssonicmoec>Uc[RS5 OUS:waX;a[SUS35e[TU]X5$)zfReturn the requested `experts_implementation`. Also strictly check its validity, and raise if invalid.a You tried to access the `ExpertsInterface` with a `config._experts_implementation` set to `None`. This is expected if you use an Expert Module as a standalone Module. If this is not the case, something went wrong with the dispatch of `config._experts_implementation`eager`zL` is not a valid experts implementation registered in the `ExpertsInterface`)logger warning_onceKeyErrorsuperget)r6experts_implementationdefault __class__s r get_interfaceExpertsInterface.get_interfaces[ ! )   N  $w .3I3U*++wx w{1;;rrj)rstrrrreturnr) __name__ __module__ __qualname____firstlineno____doc__rrErr_global_mappingr__static_attributes__ __classcell__)rs@rrrs%E200, O < ^^^^^SUUUUU4SjjnUbU"U5$U$)a\Decorator to modify experts class to support different experts implementations. Args: experts_class (`type[torch.nn.Module]`, *optional*): The experts class to modify. If not provided, returns a decorator that can be applied to the class. experts_interface (`ExpertsInterface`, *optional*, defaults to `ALL_EXPERTS_FUNCTIONS`): The experts interface to use for dispatching the forward method. is_concatenated (`bool`, *optional*, defaults to `True`): Whether the expert weights are stored in concatenated layout [gate;up] or interleaved layout [gate0, up0, gate1, up1, ...]. is_transposed (`bool`, *optional*, defaults to `False`): Whether the expert weights are stored in transposed format. has_bias (`bool`, *optional*, defaults to `False`): Whether the expert layers include bias terms or not. has_gate (`bool`, *optional*, defaults to `True`): Whether the experts use a gating mechanism or not. Whether it has gate_up_proj weights or just up_proj weights. Returns: `type[torch.nn.Module]`: The modified experts class. c>^^URmURm[T5UUUU U4Sj5n[T5UU4Sj5n[US5(d [UlXlX lU$)Ncb>T"X/UQ70UD6 XlTUlTUlTUlTUlgN)configr(r*rr) r6rargskwargsr*r(rr original_inits r__init__=use_experts_implementation..wrapper..__init__1s8 $ 8 8 8 K$DM$DM!.D #2D rch>TRURRT5nU"U/UQ70UD6$r)rr_experts_implementation)r6rrexperts_forwardroriginal_forwards rforward.wrapper..forward:s5/==dkk>a>acstO"49$9&9 9rr.)rrrrzrr.) experts_classrrrrrr*r(rrs @@rwrapper+use_experts_implementation..wrapper-sy%.. (00 }  3 3  3   : ! :}m44(;M %!) 'r)rtype[torch.nn.Module]rrrj)rrrrr*r(rs ````` ruse_experts_implementationrs%>2 }%% Nr)NF) r torch.Tensorrrrtorch.Tensor | Nonerboolrr) r6ztorch.nn.Moduler7rr8rr9rrr)rrrrrNrrr)rrrrrNrrr) rrrrrNrrrrrrr)rrrrr)rztype[torch.nn.Module] | Nonerrrrrrr*rr(rrr)+ __future__rcollections.abcr functoolsrutilsr utils.genericrutils.import_utilsr r r r rrrrr_dynamoassume_constant_result get_loggerrrrrErSr\rarilibrary custom_op register_fakeregister_autogradr~rurrrALL_EXPERTS_FUNCTIONSrrrjrrrs#$, 4. !& D DE^ _"]]AABXY   H %\!%       D=7 =7=7=7 =7  =7F4 ^ )& MM+E   MM CE^_ MM##+%8$*WZP P P P PF!% # # # #  #  #  #Le7 e7e7e7 e7  e7P<'<2)* "37;+@ ;/;(; ;  ;  ;;;r