3ji8%SSKrSSKrSSKrSSKrSSKJr SSKJr SSK J r SSK J r SSK Jr SSKJr SS KJrJr SS KJr SS KJrJr SS KJrJr S SKJr \(aSSKJ r SSK!J"r" SSK#J$r$ \"5(a SSK%r%SSK&J'r' \RP"\)5r*SSK+J,r,J-r-J.r.J/r/J0r0J1r1 SSK+J2r3 SSK+J4r5 SSK+J6r6 SSK+J8r9 Sr:\RvRySS5R{5r>Sr?\>\;r@S\A4Sjr4S\A4Sjr8S\-"S S!S"90S\-"S#S$S"90\/R\-"S%S&S"90\/R\-"S%S&S"90\/R\-"S'S(S"90\/R\-"S)S(S"90\/R\-"S%S&S"90S*.S\-"S+S,S"90\/R\-"S-S.S"9\/R\-"S-S.S"90\/R\-"S/S.S"90\/R\-"S-S.S"90\/R\-"S-S0S"90S1.S\/R\/R-\-"S2S3S S4900S\/R\/R-\-"S2S5S S4900S\/R\/R-\-"S2S6S S4900S\/R\/R-\-"S2S7S S4900S\/R\/R-\-"S2S8S S4900S\/R\/R-\-"S2S9S S4900S:. rE\F\A\F\,\A-\-\F\/\-4-44\GS;'\6b;\/R\6"S90\/R\6"S9\/R\6"S90S?.\ES@'SArHSlSBjrISLS SM.SNS SM.SNS SM.SOSSM.SPSSM.SQSRSS.ST.rJ\F\A\F\A\A44\GSU'0rK\F\A\ S-4\GSV'SW\AS-SX\L4SYjrMSmSW\ASZ\S-S[\LSX\ S-4S\jjrN\K4S]\AS^\F\A\ S-44S_jjrOSnS]\AS`\AS-Sa\P\A-S-S[\LSX\ 4 Sbjjr2Sc\Q\\-4SdjrRSqS\Se5rTSf\USg\Q\ASh\USX\U4SijrVSoSjjrW/SkQrXg!\7a Sr6GNf=f!\7a Sr:GNf=f!\7a+ Sr?Sr@SCr4SDr8"SESF5r-"SGSH5r.SIr1SJr0SKrIGNf=f)pN)Callable)contextmanager)Path) ModuleType) TYPE_CHECKING)version)!get_checkpoint_conversion_mapping®ister_checkpoint_conversion_mapping)register_patch_mapping)ENV_VARS_TRUE_VALUESlogging)is_kernels_availableis_torch_available)flash_attention_forward)PretrainedConfig)PreTrainedModel) KernelConfig)DeviceLayerRepositoryLocalLayerRepositoryModeregister_kernel_mappingreplace_kernel_forward_from_hub) get_kernel)use_kernel_forward_from_hub)FuncRepository)use_kernel_func_from_hubTFUSE_HUB_KERNELSYES layer_namecl[(a [U5$[RS[35 S$)NFkernels hub usage is disabled through the environment USE_HUB_KERNELS=cU$Nclss _/home/wildlama/miniconda3/lib/python3.13/site-packages/transformers/integrations/hub_kernels.py-use_kernel_forward_from_hub..Vss)_kernels_enabled$_kernels_use_kernel_forward_from_hublogger warning_once_TRANSFORMERS_USE_HUB_KERNELS)r"s r*rrOs5  7 C C   XYvXwx # "r- func_namec[(a[(a [U5$[(d[R S5 S$[R S[ 35 S$)Nzwuse_kernel_func_from_hub is not available in the installed kernels version. Please upgrade kernels to use this feature.r$cU$r&r'funcs r*r+*use_kernel_func_from_hub..esr-)r._has_use_kernel_func_from_hub!_kernels_use_kernel_func_from_hubr0r1r2)r3s r*rrXs_   = =4Y? ?00##B% $##\]z\{|% $r-cudaz!kernels-community/deformable-detrMultiScaleDeformableAttentionrepo_idr"zkernels-community/moe Llama4TextMoezkernels-community/liger_kernels LigerRMSNormzkernels-community/rmsnormRMSNormzkernels-community/mlx_rmsnorm)r;rocmxpumpsnpuzmedmekk/triton-llama-mlpTritonLlamaMLPzkernels-community/megablocksMegaBlocksMoeMLPzahadnagy/megablocksCPUMegaBlocksMoeMLP)r;rBrCcpuzkernels-community/activationFastGELU)r>r"r QuickGELUNewGELUSiluGeluGeluTanh) r<r?rAMLPrGrJrKrLSiLUGeLUrO_KERNEL_MAPPINGzkernels-community/rotaryapply_rotary_transformers)r>r3)rCr;rotary_pos_embcb^TU;=(d# [U4SjUR555$)Nc3j># UH(n[U[5=(a [UT5v M* g7fr&) isinstancedicthas_key).0vkeys r* has_key..s([PZ1z!T2Fwq#FPZs03)anyvalues)dr]s `r*rZrZs'ax[3[PQPXPXPZ[[[r-cUc[n[US5(a[SS9(d [S5e[ U5 g)NrCz0.10.2) MIN_VERSIONzfkernels uses an incompatible version. Please install the latest version with `pip install -U kernels`.)rSrZr ImportErrorr)mappings r*$register_kernel_mapping_transformersrgs; ?%G 7E " "+?H+Ux  (r-c SnU$)NcU$r&r'r(s r* decorator.use_kernel_forward_from_hub..decoratorsJr-r'argskwargsrjs r*rrs r-c SnU$)NcU$r&r'r6s r*rj+use_kernel_func_from_hub..decoratorsKr-r'rls r*rrs r-c\rSrSrSrSrg)ric[S5e)NzNLayerRepository requires `kernels` to be installed. Run `pip install kernels`. RuntimeErrorselfrmrns r*__init__LayerRepository.__init__sop pr-r'N__name__ __module__ __qualname____firstlineno__rx__static_attributes__r'r-r*rrs qr-rc\rSrSrSrSrg)ri c[S5e)NzSLocalLayerRepository requires `kernels` to be installed. Run `pip install kernels`.rtrvs r*rxLocalLayerRepository.__init__!stu ur-r'Nrzr'r-r*rr s vr-rc[S5e)Nz^replace_kernel_forward_from_hub requires `kernels` to be installed. Run `pip install kernels`.rtrmrns r*rr$s l  r-c[S5e)NzVregister_kernel_mapping requires `kernels` to be installed. Run `pip install kernels`.rtrs r*rr)ssttr-c[S5e)Nzcregister_kernel_mapping_transformers requires `kernels` to be installed. Run `pip install kernels`.rtrs r*rgrg,s q  r-zkernels-community/causal-conv1d)r>rzkernels-community/mamba-ssmz!kernels-community/finegrained-fp8zkernels-community/deep-gemmzkernels-community/sonic-moez ep-support)r>revision)z causal-conv1dz mamba-ssmzfalcon_mamba-ssmzfinegrained-fp8z deep-gemmz sonic-moe_HUB_KERNEL_MAPPING_KERNEL_MODULE_MAPPINGattn_implementationreturncJUSL=(a [R"SU5SL$)zJCheck whether `attn_implementation` matches a kernel pattern from the hub.Nz'^[^/:]+/[^/:]+(?:@[^/:]+)?(?::[^/:]+)?$)research)rs r* is_kernelr>s- 4' c II@BU V^b br-attention_wrapperallow_all_kernelscSSKJn SSKJn SU;aUR S5SOUn[ U5(dg[ (d [S5eSU;a$UR S5upgUR5nOUnSnUR5nURS 5uphn UR5nU (aU R5OSn [XiUS 9n Sn [U S5(a Uc[nUn O/[U S5(aSSKJn UbUOUn Sn OUb [#X5n UR%UW 5 UR%XU 5 U $![an [S US U S 35eSn A ff=f)a Load and register the kernel associated to `attn_implementation`. Args: attn_implementation: A string, usually a kernel repo like "kernels-community/flash-mla". attn_wrapper: a callable for the wrapper around the attention implementation. In `transformers` we have a wrapper around the `flash_attn_var_len` call, and the same goes for `sdpa` and `eager`. They just prepare the arguments properly. This is mostly used for continious batching, where we want the `paged` wrapper, which calls the paged cache. allow_all_kernels (`bool`, optional): Whether to load kernels from unverified hub repos, if it is a custom kernel outside of the `kernels-community` hub repository. r )ALL_MASK_ATTENTION_FUNCTIONS)ALL_ATTENTION_FUNCTIONS|rN`kernels` is either not installed or uses an incompatible version. Please install the latest version with `pip install -U kernels`.:@)rrz-An error occurred while trying to load from 'z': .flash_attention_2flash_attn_varlen_funcsparse_atten_func)msa_attention_forwardsdpa) masking_utilsrmodeling_utilsrsplitr_kernels_availablerestrip partitionr Exception ValueErrorhasattrr msa_attentionrgetattrregister)rrrrractual_attn_namer> kernel_name_revkernelemask_implementationkernel_functionrs r*load_and_register_attn_kernelrFs =8rr)rrrzFailed to load kernel z: r-ris_ _availablez..utils.import_utils)rXrrr0r1rgetrALLOW_ALL_KERNELSFileNotFoundErrorAssertionError importlibreplace import_module __package__rrcallable) rrfr>rrrrrnew_kernel_namer3 utils_modis_kernel_availablemodules r*lazy_load_kernelrs*W-A:"N"N##--gk]2STU# ()+6yAG*;7;;JMH)+6::9dKGG_pqF#)K @  - %--c37/**5  '!//0F TI"))"E  ' ( (-@-B-B ,"00O3DF'- $ $(G  ?! M#'G   "8 Rs K L L:  9 (#'G 4  9 ( '"&  ' ,'+$    ,sCAD(#E3F( E02EE0/E03 FFFFrrc^SSKJn [(d [S5eUR S5SnUS:waU(d [ S5eSX@S .n[ RRS 5n[R"U5[R"S 5:a [XX&S 9$[XUS 9$)Nr ) __version__r/rzkernels-communityziYou need to specify `allow_all_kernels=True` to use kernels outside of the `kernels-community` repository transformers) frameworkrr>kernelsz0.10.4)rr user_agent)rr) rrrerrrmetadatar pkg_versionparseget_kernel_hub)rrrrr repo_parentrkernels_versions r*rrs    -  ##C(+K))2C w   .+^J((00;O)[->->x-HHkgeekgNNr- module_namescD^[T[5(aT/mU4SjnU$)a3 This decorator attaches the target function within the module as a plain attribute (not as a submodule). Keep in mind that this registration is only meant for `kernelize` to recognize its target modules (i.e. function exchanged for a weightless `nn.Module` with the same forward) to then exchange to the kernel variation (in-place) if the conditions are met. We cache each of these function-based registrations: After proper registration and exchange it is removed from the module's `_modules` dict as it does not really act as `nn.Module` but a base function. c<>^URmUU4SjnXlU$)Nc>T"U/UQ70UD6 URRS05nTHNn[USS5=(d! [USS5=(d [USS5nUc[SU<35eXCU'MP g)N_hidden_kernelsr{kernel_layer_namer3z)Could not infer kernel function name for )__dict__ setdefaultrr)rwrmrnhidden_kernelsfnnamer orig_inits r*new_init8use_kernelized_func..decorator..new_inits d ,T ,V ,"]]556GLN"B D16r#6=6r;5 <$'PQSPV%WXX(*t$#r-)rx)r)rrrs @r*rj&use_kernelized_func..decoratorsLL  *"  r-)rXr)rrjs` r*use_kernelized_funcrs&,))$~ . r-c#,# SqSv Sqg!Sqf=f7f)z Context manager used to adjust the value of the global `ALLOW_HUB_KERNELS`. This is needed, as this argument cannot be forwarded directly to the `__init__` of the models, where we set the attention implementation. TNF)rr'r-r*allow_all_hub_kernelsrs!"  "Es  parent_cls child_names kernel_clsc^^^URmUUU4Sjn[SUR3U4SU05nSUR3UlU$)a" Create a new class that inherits from `parent_cls` and fuses the child modules specified in `child_names with the provided `kernel_cls`. The first child in `child_names` will be replaced with the `kernel_cls`, and the rest will be replaced with `nn.Identity()` to keep the same interface. c>T"U/UQ70UD6 TVs/sHn[X5PM nnT"U6n[UTSU5 TSSH"n[X[R"55 M$ gs snf)Nrr)rsetattrnnIdentity) rwrmrnrchildrenkernel_instancerr original_inits r* patched_init9make_parent_class_for_kernel_fusion..patched_init#snd,T,V,4?@KDGD'K@$h/k!no6OD D .$AsA*Fusedrx)rxtyper{r})rrrr patched_clsrs `` @r*#make_parent_class_for_kernel_fusionrsW''M/z2234zmjR^E_`K!&z'>'>&?@K r-c P ^[US5(a[URS5(d[SURS35eURRn0n0nSnUR R 5GHupx[U[5(aN[UR55S:wa[SU35e[[UR555n O+[U[5(aUn O[SU<SU<35eU RS 5upn U (aU (d[S U <SU<35eUR(a8U R!S 5R#S 5S n [%['U 5U U S 9nO [)XS9nUR+5nUc[SU<35e[,R.R1UR25nU(a[5UURS3S5OSn[U[5(a/UcXU'GMURUlUXG'XUR'GO[U[85(GaUc'[SUR<SURS35eURUlUVs/sHnUSPM nnUVs/sHnUR;SS5SPM nn[[=U55S:wa[SU35eUSR?SS5nUVs/sHnUR;SS5SPM nnUc'[@RB"S5 U"U5nSSS5 SnURE5Hunm[FRH"UU5(dM$[KU4SjU55(d [SU<SUSUR<35eSn[MT5n[OUUU5UUR'M U(d[S U<S!UR<S"35e[QUSS#9 [US$5(a3[SU5nURV5nUbUU-n[YUUSS#9 XUR'GM XRlgs snfs snfs snf!,(df  GNF=f)%N config_class model_typezModel z# has no config_class or model_type.rzLExpected exactly one kernel repo regardless of device/mode specificity, got zInvalid hub repo z for layer rzInvalid kernel repo string r) repo_path package_namer"r=z*Could not load kernel class from hub_repo=Layoutz Fused kernel z* requires a companion layout class named 'zLayout' in the same module.rrzGAll patterns for a fused kernel must share the same parent module, got *z\w+metaFc3<># UHn[TU5v M g7fr&)r)r[childrs r*r^;register_kernel_replacements_and_fusions..sK{e76511{szModule z* does not have the expected child modules z required for the fused kernel TzNo module matched pattern z for fused kernel z3. Provide the full dotted path from the model root.) overwriteconversion_mapping)-rrrr{rkernel_mappingitemsrXrYlenranextiterstrruse_local_kernelrstriprrrrloadsysmodulesrr|rrtuplersplitsetrtorchdevice named_modulesr fullmatchallrrr r listrr )r)config kernel_configr patch_mapping new_mapping meta_modelr"hub_reporepo_strr>rlayer_name_in_reporrepor kernel_mod layout_clsitem glob_patternspparent_patternsparent_patternr matched_anyr module_clsexisting transformsrs @r*(register_kernel_replacements_and_fusionsr-0s 3 ' 'ws7G7G/V/V6#,,/RSTT!!,,J%'MKJ - < < B B D h % %8??$%* bckblmD!234H # & &H0 K ~VW W)1););C)@&&0:8,kR\Q_`a a  ) )">>#.44S9"=L'w-)-D #7RDYY[  I(VW W[[__Z%:%:; R\WZJ,?,?+@)GNbf  j# & &!*2J',6+>+>J ((2M %08 ++ , E * *! #J$7$7#:;(1122MO ,6+>+>J (1;<T!WM<rGsQ $% ,j41I4602   H %B  * .Y(,%%'JJNN3De$L$R$R$T!48LL## %C %" O;:*  O/* =-!=-!7(!;(!=-!5  D O2+  :1 :1! 11!:1!:4!/ > !3!33_:)6  !3!33_:*6  !3!33_:(6  !3!33_:vWX6  !3!33_:vWX6  !3!33_:z[\6 wBeOT#tFSL/DI^D_2_$_``aBJ!6B]!  ~6B] 6B]!  - () \)f"CqQ:qI$AaP#FSTU:qI: U 2T#tCH~-.8:S*t"33493:$diDD19DD\`D$DNPf- #- S*t:K5K0L- d $# OODjO3Y O O  O:$d8nx&?$P " " c 4y/ y/ y/"y/ y/x Y  .(-%.D%   qqvv u E% sUO(9O OH=O( OO(OO(O%!O($O%%O((-PP