3j(%SSKrSSKrSSKrSSKrSSKr/SQrSrSSKr \ RRSr Sr\RS:Xa#\"\ S5r\(a\R""\5rO\"\ S5r\(a}SSKr\R&"\5HcrS\;dM S\;dM\R*"\ 5 \R,"\R.R1\\5\R2S 9 SSS5 Me Sr\R.R1\R.R7\5\RS:XaS OS 5r\R.R=\5(d\RS:XaS OSr\R.R7\5r \R&"\ 5HOr\RCS 5(dM\RE\5(dM3\R.R1\ \5rMQ \R.R=\5(a\RFRIS\5r%\%(a[\%RL(aJ\RFRO\%5r\\RPS'\%RLRS\5 Sr*Sr+OSr*S\3r+OSr*S\3r+SSK/J0r0 SSK1J2r2 \*=(a \3"\SS5r4Sq5\RlS-\7S'S\84Sjr9S\Rl4Sjr:S\Rl4Sjr;\Rx4S\RlS\RlS\RzS\Rl4S jjr>\R~4S\RlS\RlS\RzS\Rl4S!jjr@\Rx4S\RlS"\RlS\RzS\Rl4S#jjrAS\S\RlS$\RlS%\BS&\CS'\CS\D\Rl\Rl44 S(jjrE\R~S4S)\RlS$\RlS*\RlS\RzS'\CS\Rl4 S+jjrFS]S\RlS,\CS\D\Rl\Rl44S-jjrGS^S.\RlS/\RlS0\RlS1\RlS2\RlS3\RlS4\RlS5\RzS6\RlS\Rl4S7jjrHS\RlS8\RlS\Rl4S9jrIS:\RlS;\RlS8\RlS\D\Rl\Rl44S<jrJS\RlS8\RlS\Rl4S=jrKS:\RlS;\RlS8\RlS\D\Rl\Rl44S>jrLS?rMS@rN0rO\P\D\QSA4\D\Rl\Rl\Rl44\7SB'SC\RlS\C4SDjrRSC\RlS\84SEjrSSF\RlS\Rl4SGjrTS_S\RlSI\RlSJ\RlSK\8SL\CSM\RlS-SN\CS\D\Rl\Rl\Rl44SOjjrUS`SP\RlSC\RlSQ\RlSR\RlSS\RlSF\RlS4\RlS-SL\CS\Rl4STjjrVSHrWS\RlSU\RlSR\RlSV\RlSW\8S\Rl4 SXjrXSaS\RlSU\RlSR\RlSV\RlS4\RlS-SW\8S\Rl4SYjjrYS\P4SZjrZS[r[\["5 g!\ a \RSr GNf=f!,(df  GM=f!\ a GN(f=f!\,ar-Sr*\."\-5r+SrSr-C-GNSr-C-f\ ar-Sr*S\-3r+SrSr-C-GNSr-C-ff=f)bN) apply_rope apply_rope1apply_rope_split_halfapply_rope_split_half1dequantize_nvfp4dequantize_per_tensor_fp8gemv_awq_w4a16quantize_mxfp8quantize_nvfp4quantize_per_tensor_fp8quantize_svdquant_w4a4scaled_mm_nvfp4scaled_mm_svdquant_w4a4stochastic_rounding_fp8cl[R"U5Hup#nUHnX;dM Us s $ M gN)oswalk) start_dir lib_patternroot_dirsfilesfiles ^/home/wildlama/miniconda3/lib/python3.13/site-packages/comfy_kitchen/backends/cuda/__init__.py find_lib_dirr0s4"$'')"4 D&K#5win32 cublasLt64zlibcublasLt.socublasLtz.so)modez _C.abi3.pydz _C.abi3.soz.pydz_C.zcomfy_kitchen.backends.cuda._CTFz!Could not create module spec for zExtension file not found: zFailed to load extension: ) DTYPE_TO_CODE)roundup HAS_CUBLASLT_cublas_workspacereturnc[RR[RR55RS:agg)zAReturn 32 MiB if using hopper, 4 MiB for all other architectures. ii@)torchcudaget_device_propertiescurrent_devicemajorrrget_cublas_workspace_size_bytesr/ss2 zz'' (A(A(CDJJaO rct[c,[R"[5[RSS9q[$)zReturns workspace for cublas.r*dtypedevice)r%r)emptyr/uint8r.rrget_cublas_workspacer6zs/ !KK + -U[[  rtensorcbUR(aUR5nURSS9$)aExport tensor via DLPack without cross-stream sync. Works around PyTorch issue where __dlpack__(stream=None) syncs with the default stream, breaking CUDA graph capture on non-default streams. See: https://github.com/pytorch/pytorch/pull/163242 Detaches first so nn.Parameter (requires_grad=True) inputs like bias export without PyTorch's gradient-tracking refusal. Returns a PyCapsule containing the DLTensor that nanobind can import. )stream) requires_graddetach __dlpack__)r7s r_wrap_for_dlpackr>s,   B  ''rxscale output_typec [URn[UnUR5(dUR5n[R "UR [RURS9nUR5n[RRUR5Rn[R[U5[U5[U5UUUU5 UR!U5$)Nr1)r"r2 is_contiguous contiguousr)r4shaper5r3numelr*current_stream cuda_stream_Cr r>view)r?r@rAinput_dtype_codeoutput_dtype_code result_uint8rF stream_ptrs rr r s%QWW-%k2 ??   LLN;;qwwekk!((KL GGIE**1884@@J&    [ ))rc UR5S:XdS5e[URn[Un[R"UR X R S9nUR5n[RRUR 5Rn[R[UR[R55[U5[U5UUUU5 U$)NzScale must be a scalar tensorr1)rFr"r2r)r4rEr3r*rGrHrIrr>rJr5)r?r@rArKrLresultrFrNs rrrs ;;=A >>> $QWW-%k2 [[ HH EF GGIE**1884@@J   ,-   Mrrngc[UnUR5(dUR5nUR5(dUR5n[RR UR 5Rn[R[U5[U5UUR5U5 URU5$r) r"rCrDr)r*rGr3rHrIstochastic_round_fp8r>rFrJ)r?rRrArLrNs rrrs &k2 ??   LLN     nn**1884@@J    88K  rper_tensor_scaleepsilonpad_16xhi_firstc pUR5(dS5eURupVU(a[US5n[US5nO(XVpUS-S:Xd SU35eUS-S:Xd SU35e[R"XxS-4UR [R [RS9n [US5n [US-S 5n [R"X4UR [R S 9n UR5S:XaURS 5n[RRUR 5Rn [R[!U5[!U5[!U 5[!U 5UUUU 5 U R#[R$5nX4$) NInput tensor must be contiguousrz&num_rows must be divisible by 16, got z&num_cols must be divisible by 16, got r3r2 memory_formatr3r2rP)rCrEr#r)r4r3r5contiguous_formatzerosdimreshaper*rGrHrIr r>rJ float8_e4m3fn)r?rUrVrWrX orig_rows orig_colsnum_rowsnum_colsqx scale_rows scale_colssx_uint8rNsxs rr r s ??  ??? 77I9b)9b)&("}!V%KH:#VV!"}!V%KH:#VV! hA .qxxu{{bgbyby zB 3'JR+J{{J3AHHEKKXH"+33A6**1884@@J)*"  u** +B 6Mrrk block_scalesc $UR5(dS5eURupVUS-n[R"XW4URUS9nUR 5S:XaUR S5nUR[R5n [Un [RRUR5Rn [R[U5[U5[U 5[U5U UU 5 U$)NrZr\rarrP)rCrEr)r4r3rdrerJr5r"r*rGrHrIrr>) rkrUrprArXrinum_cols_packedrjoutputblock_scales_uint8rLrNs rrrs     @@@  "H"H [[(-bii{ SF"+33A6%**5;;7%k2**2995AAJ)*+,  Mrpad_32xcUR5(dS5eURup#U(a[US5n[US5nO(X#pTUS-S:Xd SU35eUS-S:Xd SU35e[R"XE4UR [R [RS9n[US5n[US-S5n[R"Xx4UR [RS 9n [RRUR 5Rn [R[U5[U5[U 5UU 5 U R![R"5n Xk4$) NrZ rz&num_rows must be divisible by 32, got z&num_cols must be divisible by 32, got r]r_r`ra)rCrEr#r)r4r3rfrbrcr5r*rGrHrIr r>rJfloat8_e8m0fnu) r?rurgrhrirjrkrlrmrnrNros rr r 9sO ??  ??? 77I9b)9b)&("}!V%KH:#VV!"}!V%KH:#VV! h)!((%BUBUeje|e| }B3'JR+J{{J3AHHEKKXH**1884@@J"  u++ ,B 6Mrabtensor_scale_atensor_scale_b block_scale_a block_scale_bbias out_dtypealphac  Sn Sn [U[RR5(a URn[U[RR5(a URnUcX#-nO5[U[RR5(a URnUR [R :waUR[R 5nUR5S:XaURS5n[U[RR5(a URn[U[RR5(a URn[U[RR5(a URn[U[RR5(a URnURupURupX:XdS5eSU -n[R"XXpRS9nU S-S:XdS 5eUR UR :XdS 5eUR [R:XaU S :XdS 5e[S 5eUR [R:XaXU S:XdS5eUcS5eUR [R :XdS5eUR!5S:XdS5eO[SUR 35e[#U S5n[#U S5n[#X-S5nUR5S:XdS5eUR%5UU4:XdS5eUR5S:XdS5eUR%5UU4:XdS5eUc[R&"5nO5UR [R([R*4;dS5eSunnUR-[R.5nUR-[R.5n[0Un[R2R5UR5R6nUbUR!5S:Xa/[R"SUR[R(S9nO5[U[RR5(a URn[8R;[=U5[=U5[=U5[=U5[=U5U[=U5[=[?55U [=U5U5 U$)NFr[rrPzMatrix dimensions do not matchr\r1z-B tensor must have 8 alignment in N dimensionzA and B scale dtype must matchrwz#MXFP4 only supports block length 32z2MXFP4 is not supported yet for cuBLAS in CUDA 12.9z#NVFP4 only supports block length 16z alpha must be provided for NVFP4zalpha must be float32zalpha must be a scalarzUnsupported scale dtype: r_r`zInvalid A scale shapezInvalid B scale shapez*Only fp16 and bfloat16 bias are supported.)TFra) isinstancer)nn Parameterdatar2float32tordrerEr4r3rx ValueErrorrfrFr#sizeTensorfloat16bfloat16rJr5r"r*rGrHrIcublas_gemm_blockwise_fp4r>r6)ryrzr{r|r}r~rrr accumulate block_lengthmk_ank_bkout roundup_m roundup_n roundup_sk_transa_transbblock_scale_b_uint8block_scale_a_uint8out_dtype_coderNs rrr]s~JL.%(("4"455',,.%(("4"455',, }/ E588-- . .  {{emm#' yy{a a !UXX''(( FF!UXX''(( FF-!3!344%** -!3!344%** WWFA WWFA :777: CA ++a)HH =C q5A:FFF:   -"5"5 5W7WW 5e222r!H#HH!MNN    3 3 3r!H#HH! D"DD {{emm+D-DD+{{}!;#;;!4]5H5H4IJKK3I3I*A.J    ! #<%<< #    Iz#: :Sr?rx_outrNs rrrs ??   LLN  " " $ $((*   Q E**1884@@JMM#    Lrxqxkc URUR:wa[X5[X54$UR5(dUR5nUR5(dUR5nUR5(dUR5n[R "U5n[R "U5n[R RUR5Rn[R[U5[U5[U5[U5[U5US5 X44$r) rErrCrDr)rr*rGr3rHrIrr>rrrxq_outxk_outrNs rrrs xx2882);r+EEE     ]]_     ]]_  " " $ $((*   b !F   b !F**2995AAJMM#    >rc UR5(dUR5nUR5(dUR5n[R"U5n[RR UR 5Rn[R[U5[U5[U5SSUS5 U$NTrrs rrrs ??   LLN  " " $ $((*   Q E**1884@@JMM#    Lrc URUR:wa[X5[X54$UR5(dUR5nUR5(dUR5nUR5(dUR5n[R "U5n[R "U5n[R RUR5Rn[R[U5[U5[U5[U5[U5US5 X44$r) rErrCrDr)rr*rGr3rHrIrr>rs rrr&s  xx288%b46LR6[[[     ]]_     ]]_  " " $ $((*   b !F   b !F**2995AAJMM#    >r@r_._SVDQUANT_WORKSPACE_CACHEwgtc(UR5S:H$)Nr`rdrs r_is_svdquant_tile_packed_weightrOs 779>rc[U5(a[URS5[-$[URS5$)Nr)rintrE_SVDQUANT_W4A4_BLOCK_Nrs r"_svdquant_out_features_from_weightrSs9&s++399Q< #999 syy| rlora_upcUR5S:waU$[USS5nUbiURUR:XaOURUR:Xa5URURS[ -URS4:XaU$UR SSS5RURS[ -URS5R5nX l U$)aReturn a natural (N, R) view/copy for tile-packed proj_up. Converter layout is (N/128, R, 128). cuBLAS addmm_ wants a natural row-major (N, R) tensor. Cache the one-time reorder on the source tensor so the runtime path keeps cuBLAS speed without paying a per-forward permute. _ck_natural_lora_upNrrPr\) rdgetattrr3r2rErpermutererDr)rcachednaturals r!_natural_lora_up_from_tile_packedrYs{{} W3T :F MMW^^ + LLGMM ) LLW]]1-0FF VWHXY Y ooaA&.. a117==3Cjl #* Nrsmooth lora_downpad_size act_unsignedlora_xreuse_workspacec RUR5S:XaUR5(dS5eURupxURSn [Xs5n [n X-S:Xd SUSU 35e[ R "SS5R5S ;a[RO URn [RRUR5Rn U=(a( [ R "S S 5R5S ;nU(GaURRbURRO[RR!5nU[#U 5URXX4n[$R'U5nUc[R("XS-[R*URS9[R("X-XRURS9[R("XXRS94nU[$U'UunnnO|[R("XS-[R*URS9n[R("X-XRURS9n[R("XXRS9n[,R/[1U5[1U5[1U5[1U5[1U5[1U5[3U5U 5 UbUOUnUS:a^US UnURUR:Xa,UR5(a[R4"UUUS9 OUR7UU-SS9 X:aUUS R95 UR;[R<5UU4$)u#SVDQuant W4A4 activation quantize + smooth + LoRA-down (CUDA). Kitchen-native layouts: x (M, K) bf16/fp16 main-path input (pre-shifted if unsigned) smooth (K,) bf16/fp16 lora_down (K, R) bf16/fp16 natural row-major lora_x (M, K) bf16/fp16 pre-shift x for LoRA (defaults to x) q_x (M_pad, K/2) int8 two int4 per byte ascales (K/G, M_pad) bf16/fp16 per-row per-group lora_act (M_pad, R) bf16/fp16 by default (same dtype as x) act_unsigned=True selects scale=max/15 + clamp [0,15] (u4 bit patterns for downstream u4.s4 MMA). Caller must ensure x is non-negative — shift is a model-topology concern kept out of the kernel API. Pass lora_x=raw_x when x was pre-shifted (LoRA operates on pre-quantization, pre-shift activations). r\zx must be 2D contiguousrPrK=z! must be divisible by group_size=$COMFY_KITCHEN_SVDQUANT_LORA_ACT_FP32>1onyestrue&COMFY_KITCHEN_SVDQUANT_REUSE_WORKSPACEr>0noofffalseNr1)rT) non_blocking)rdrCrEr#_SVDQUANT_W4A4_GROUP_SIZErgetenvlowerr)rr2r*rGr3rHindexr,rrgetr4r5rIsvdquant_quantize_w4a4r>boolmmcopy_zero_rJint8)r?rrrrrrrrrm_padglora_act_dtyperN device_indexkeyrq_xascaleslora_actlora_src lora_act_rowss rr r qs2 557a E6QXXN AFEJ EN88LF .4 %c *!'Whkk%au{{188L++afe77188L;;u~hhO #!" \ " +vH1u !   (.. 0]5P5P5R5R HHXym <   9 44  H y  88EJJ ( 22ractrwscales lora_act_inc"URSn[U5n [R"XURUR S9n [R"SURUR S9n [ R"SS5R5S;n [ R"S5n U c [U5nOU R5S;n[U5UR5S:H:Hn[ R"S 5nURU R:H=(a' U=(a USL=(d UR5S;n[RRUR 5Rn[R[!UR#[R$55[!UR#[R$55[!U5[!U5[!U5[!U5[!UbUOU 5[!U 5UU UUU5 U(dbURU R:XaUOUR'U R5n[)U5nU R+UUR-55 U $) aSVDQuant W4A4 int4 GEMM + LoRA-up + bias (CUDA). int4 MMA + per-group dequant + bias run in one kernel. By default, when lora_act_in already has the output dtype and proj_up's layout matches the weight layout, LoRA-up is fused into the CUDA epilogue with bf16/fp16 tensor-core MMA. Set COMFY_KITCHEN_SVDQUANT_FUSE_LORA_UP=0 to force the older Python/cuBLAS addmm_ epilogue for comparison. Kitchen-native layouts: act (M, K/2) int8 two int4 per byte (signed or unsigned) wgt (N, K/2) int8 signed int4 weight, natural row-major or (N/128, K/64, 32, 128) int8 kitchen_tile_packed_w4a4 ascales (K/G, M) bf16/fp16 per-row per-group wscales (K/G, N) bf16/fp16 per-col per-group or (N/128, K/G, 128) bf16/fp16 for tile-packed wgt lora_act_in (M, R) bf16/fp16 or fp32 lora_up (N, R) or (N/128, R, 128) bf16/fp16 bias (N,) or (N/128, 128) bf16/fp16 (optional) out (M, N) bf16/fp16 (= lora_up.dtype) act_unsigned: if True, A fragments go through u4.s4 MMA instead of s4.s4. Set COMFY_KITCHEN_SVDQUANT_FAST_ACCUM=1 to use the experimental packed half2/bfloat162 accumulator instead of the default fp32 accumulator. rr1!COMFY_KITCHEN_SVDQUANT_FAST_ACCUMr>rrrr#COMFY_KITCHEN_SVDQUANT_SHARED_SCALENr#COMFY_KITCHEN_SVDQUANT_FUSE_LORA_UP)rErr)r4r2r3rrrrrdr*rGrHrIsvdquant_scaled_mm_w4a4r>rJr5rraddmm_t)rrrrrrrrrrrr4 fast_accumshared_scale_env shared_scalelora_up_layout_matches_wgt fuse_lora_env fuse_lorarN lora_bf16 lora_up_mms rrrs D ! A*3/A ++a'-- CC KKszz BE>CIIKPJyy!FG6s; '--/3MM ',!1CDIICDMSYY& [ & [ d " Ym&9&9&;?Y&Y **3::6BBJ%++./%++./!!%!!1u= $/#4#4 #AK{~~VYV_V_G` 6w?  9jlln- Jrqweightwzeros group_sizecURupVUS-nUnURn UR[R5n U S-R[R 5n U S- S-R[R 5n [R "X/SS9RXW5RU 5n U RXWU-U5S- UR5RS5-UR5RS5-RXW5nURUR55$)u!Large-M fallback: dequantize qweight to bf16 then bf16 cuBLAS matmul. qweight (N, K/2) int8 packed uint4 → W (N, K) bf16 via group dequant, then `out = x @ W.T`. Same algebra as eager.gemv_awq_w4a16 — used for M values where the in-kitchen MMA kernel is slower than cuBLAS bf16 GEMM. r\r`r9rg @) rEr2rr)int32rstackrerJr unsqueezematmul)r?rrrrrk_halfrr compute_dtypex32lohinibblesws r_awq_w4a16_dequant_then_matmulr"s IA AAMMM **U[[ !C ) #B !8s   uzz *Bkk2(+33A9<<]KG aa #c )WYY[-B-B2-FF ((*  r " # d1j 88ACCE?rc URnURSUS5nURupURSn X-S:wa[SU SU35eURSS-U :wa[SURSSU 35eU[:a5[ UR 5R UR5XX55n O[R"XURURS 9n [RRUR5Rn [R[!UR 5R UR55[!UR#[R$55[!U5[!U5[!U 5UU 5 UbU R'U5 U R"/US SQU P76$) uAWQ W4A16 matmul: int4 weight @ fp activation (CUDA, kitchen-native). Tiered routing: M ≤ 8 naive 1-thread-per-output kernel (GEMV-style) 8 < M ≤ 512 fused int4 x bf16/fp16 MMA kernel — dequant into shmem, mma.m16n8k16.f32 along K, no intermediate bf16 W workspace M > 512 dequant + cuBLAS bf16 matmul fallback bias is applied externally (`out.add_`), mirroring scaled_mm_svdquant_w4a4's epilogue contract. Layouts (match comfy_kitchen.backends.eager.awq): x (M, K) bf16/fp16 row-major activation. Leading dims are flattened. qweight (N, K/2) int8 two unsigned int4 per byte wscales (K/G, N) bf16/fp16 per-group, per-output-col scale wzeros (K/G, N) bf16/fp16 per-group, per-output-col zero bias (N,) bf16/fp16 optional (= wscales.dtype) out (..., N) bf16/fp16 same dtype as wscales r9rrz not divisible by group_size=rPr\z qweight K//2=z inconsistent with x K=r1N)rErer_AWQ_W4A16_MMA_M_LIMITrrDrr2r)r4r3r*rGrHrI awq_w4a16r>rJr5add_) r?rrrrr orig_shapex2drrrout2drNs rr r ?s:J ))B 2 'C 99DA aA~2aS =j\JKK}}Q!q =q)9(::QRSQTUVV !!. NN    .&  A ahhGZZ..qxx8DD  S^^-00? @ W\\%++6 7 W % V $ U #     4 == -*Sb/ -1 --rcSSKJnJnJnJn [ S15nU"U"[ [ R[ R[ R15S9U"[ [ R15S9U"[ [ R[ R15S9S.US9U"U"[ [ R[ R15S9U"[ [ R15S9U"[ [ R[ R[ R15S9S.US9U"U"[ [ R[ R[ R15S9U"[ [ R15S9U"[ [ R[ R15S9S.US9U"U"[ [ R[ R[ R15U"S54S 9U"[ [ R15S9S .US9U"S U"[ [ R[ R[ R15U"S54S 90US9U"U"[ [ R15U"S5U"SS S 94S 9U"[ [ R15S9U"[ [ R15S9U"[ [ R[ R[ R15S9S.US9U"U"[ [ R[ R15U"S54S 9U"[ [ R[ R[ R15U"S54S 9S.US9U"U"[ [ R[ R15U"S54S 9U"[ [ R[ R15U"S54S 9U"[ [ R[ R[ R15U"S54S 9S.US9U"U"[ [ R[ R15U"S54S 9U"[ [ R[ R[ R15U"S54S 9S.US9U"U"[ [ R[ R15U"S54S 9U"[ [ R[ R15U"S54S 9U"[ [ R[ R[ R15U"S54S 9S.US9U"U"[ [ R[ R15U"S5U"SSS 94S 9U"[ [ R[ R15S9U"[ [ R[ R15U"S54S 9S.USS9U"U"[ [ R15U"S54S 9U"[ [ R15S9U"[ [ R[ R15S9U"[ [ R[ R15S9U"[ [ R[ R[ R15S9U"[ [ R[ R15S9S.USS9U"U"[ [ R[ R15S9U"[ [ R15U"S54S 9U"[ [ R[ R15U"S54S 9U"[ [ R[ R15U"S54S 9S.USS9S. n[(GaU"U"[ [ R15U"S5U"SS S 94S 9U"[ [ R15U"S5U"SS S 94S 9U"[ [ R15S9U"[ [ R15S9U"[ [ R15S9U"[ [ R15S9U"[ [ R[ R15S9S.USS9US'U$)Nr) DivisibleBy ExactDimsFunctionConstraintsParamConstraintr*)dtypes)r?r@rA)paramsdefault_devices)r?rRrAr\)r shape_rules)r?rUr?r[)rdfactor)rkrUrprAr`)r?r)rrrrPr)r?rr)rr)rr min_compute_capability)rrrrrr)r?rrr) r rrr r rrrrrr rr )ryrzr{r|r}r~r) rr)comfy_kitchen.constraintsrrrr frozensetr)rrrrf float8_e5m2r5r_CUBLASLT_AVAILABLE)rrrr cuda_devices constraintss r_build_constraintsr,|sfX&L$7$$emmU]]ENN%ST)$emm_5 /$e&9&95;L;L%MN  ) $ &9$$e&9&95;L;L%MN)$emm_5 /$emmU]]ENN%ST  ) & $7$$emmU]]ENN%ST'i .FG.$e&9&95;L;L%MN  ) $ .$$emmU]]ENN%ST!*1%4$emm_5% )  ._$emmU]]ENN%ST!*1 ) 0%$ekk]3!*1{q/L M%4$emm_5%!0$e&9&9%:;! /$emmU]]ENN%ST )! $+$$emmU^^%DE!*1-$emmU]]ENN%ST!*1 )  *%$emmU^^%DE!*1&$emmU^^%DE!*1-$emmU]]ENN%ST!*1 ) "#6$$emmU^^%DE!*1-$emmU]]ENN%ST!*1 ) # "5%$emmU^^%DE!*1&$emmU^^%DE!*1-$emmU]]ENN%ST!*1 )" "#6$$emmU^^%DE!*1{q/L M*$emmU^^%DE-$emmU^^%DE!*1 )#)# "$7&i .ET]^_T`Sbc&i .EF*)U]]ENN<[2\]*)U]]ENN<[2\].$emmU]]ENN%ST +)U]]ENN<[2\] )#) $ .$$emmU^^%DE+$ejj\2!*1+$emmU^^%DE!*1*$emmU^^%DE!*1")#)' W@KD)<$$ekk]3!*1{q/L M%$ekk]3!*1{q/L M#2$emm_5##2$emm_5#"1$e&9&9%:;""1$e&9&9%:;"-$emmU^^%DE+2)#*7* %&< rcSSKJn [(dURS[5 g[ R R5(dURSS5 gURS[[[S9[5S9 g)z/Register CUDA backend with the global registry.rregistryr*Nz!CUDA not available on this system)fromlist)namemodule capabilities) comfy_kitchen.registryr/_EXT_AVAILABLEmark_unavailable _EXT_ERRORr)r* is_availableregister __import____name____all__r,r.s r _registerr=jsj/ >!!&*5 :: " " $ $!!&*MN  (W5')r)gFT)F)NNN)rFNFr)Nr)\ contextlibimportlib.util importlibrsysr)r< _dll_handle nvidia.cu13nvidiacu13__path__nvidia_cu13_path Exceptionrplatformlib_diradd_dll_directoryctypeslistdirfilenamesuppressCDLLpathjoin RTLD_GLOBALrIdirname__file__ _module_pathexistsext directory startswithendswithutilspec_from_file_location_specloadermodule_from_specmodules exec_moduler5r7 ImportErrorestr)comfy_kitchen.backends.eager.quantizationr"comfy_kitchen.float_utilsr#rr)r%r__annotations__rr/r6r>rfr2r rrrfloatrtupler rr rrrrrrrrdictobjectrrrr rrrr r,r=r.rrrms  $  -!;;//2 ||w/> ..w7K/1AB  JJw/)ex.?#,,Y7 BGGLL($C&J\J\]870" B77<< 9CLL\cLc=iuvL 77>>, ' ' /fUGGOOH-  9-H""5))h.?.?.D.D!ww||Ix@ . ww~~l##66 ,l  U\\007B<>CKK8 9 LL $ $R (!NJ"N<\NKJ1,@ D-$K^U)K)-5<<$&-ell(U\\(&FKEXEX* ||*!LL*7<{{* \\*6FK^^ ||!LL7<{{ \\8 %22! ||! !! \\ !6 2 ||2ll22 2  2  5<< %& 2r %~~   ll ,,      \\ J! ||! ! 5<< %&!V!| ||| |||LL|LL | << | << | ,,|{{| <<| \\|~5<<ELLU\\, ,,38<< 5<< %&<ellu|| ,  || 5<< %& Hgi4fck 2E%,, V[VbVb:b4c cdi$ELLS u|| 8"&!R3 ||R3 LLR3||R3 R3  R3 LL4  R3R3 5<<u|| 34R3x!%O O O\\O\\ O  O \\ O ,, OO \\Or || \\\\ LL     \\ D!% :. ||:. \\:.\\:. LL :. ,,  :.  :. \\:.zkDk\& a" - >>!,-(87  @NQJ BN-aS1J Bs_.5`1-`"`*`<6` 2 `?C`*`*B=`*`*"`*.```` `  ``'&`'*a0 a a aa