3j : >SSKrSSKJr SSKJr SSKJrJrJrJ r J r J r J r J r JrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJr SSKJ r SSK!J"r" SS K#J$r$ SS K%J&r& SS K'J(r( SS K)J*r* SS K+J,r, SSK-J.r. SSK/J0r0 SSK1J2r2 SSK3J4r4 SSK5J6r6 SSK7J8r8 SSK9J:r: SSK;Jr> SSK?J@r@ SSKAJBrB SSKCJDrD SSKEJFrF SSKGJHrH SSKIJJrJ SSKKJLrL SSKMJNrN SSKOJPrP 0S \&_S!\*_S"\,_S#\<_S$\"_S%\F_S&\H_S'\6_S(\8_S)\0_S*\>_S+\@_S,\._S-\2_S.\N_S/\(_S0\P_\L\4\4\$\D\B\J\:S1.ErQ0S \_S!\ _S"\ _S)\ _S#\_S$\_S%\_S&\_S'\_S(\_S+\_S,\ _S-\ _S*\_S.\_S/\ _S0\_\\\\\\\\S1.ErR\\\\ \ \\\4rS\R"\U5rV"S2S35rW"S4S55rXS6\Y4S7jrZS8\Y4S9jr[S:r\g);N) AutoConfig)logging) AqlmConfigAutoRoundConfig AwqConfigBitNetQuantConfigBitsAndBytesConfigCompressedTensorsConfig EetqConfigFbgemmFp8ConfigFineGrainedFP8ConfigFourOverSixConfig FPQuantConfigGemmaQuantizationConfig GPTQConfig HiggsConfig HqqConfig MetalConfig Mxfp4ConfigQuantizationConfigMixinQuantizationMethod QuantoConfig QuarkConfig SinqConfig SpQRConfig TorchAoConfig VptqConfig) HfQuantizer)AqlmHfQuantizer)AutoRoundQuantizer) AwqQuantizer)BitNetHfQuantizer)Bnb4BitHfQuantizer)Bnb8BitHfQuantizer)CompressedTensorsHfQuantizer)EetqHfQuantizer)FbgemmFp8HfQuantizer)FineGrainedFP8HfQuantizer)FourOverSixHfQuantizer)FPQuantHfQuantizer)GemmaQuantizer)GptqHfQuantizer)HiggsHfQuantizer)HqqHfQuantizer)MetalHfQuantizer)Mxfp4HfQuantizer)QuantoHfQuantizer)QuarkHfQuantizer)SinqHfQuantizer)SpQRHfQuantizer)TorchAoHfQuantizer)VptqHfQuantizerawqbitsandbytes_4bitbitsandbytes_8bitgptqaqlmquantoquark fouroversixfp_quanteetqhiggshqqzcompressed-tensors fbgemm_fp8torchaobitnetvptq)spqrfp8mxfp8z auto-roundmxfp4metalsinqgemmac@\rSrSrSr\S\4Sj5r\S5rSr g)AutoQuantizationConfigz The Auto-HF quantization config class that takes care of automatically dispatching to the correct quantization config given a quantization config stored in a dictionary. quantization_config_dictc URS5nURSS5(dURSS5(a/URSS5(aSOSn[RU-nOUc [S5eU[;a,[SUS [ [ R5535e[UnURU5$) N quant_method load_in_8bitF load_in_4bit_4bit_8bitThe model's quantization config from the arguments has no `quant_method` attribute. Make sure that the model has been correctly quantizedUnknown quantization type, got  - supported types are: ) getrBITS_AND_BYTES ValueError AUTO_QUANTIZATION_CONFIG_MAPPINGlistAUTO_QUANTIZER_MAPPINGkeys from_dict)clsrSrUsuffix target_clss V/home/wildlama/miniconda3/lib/python3.13/site-packages/transformers/quantizers/auto.pyrd AutoQuantizationConfig.from_dicts/33NC # ' ' > >BZB^B^_motBuBu 8 < <^U S SWY`F-<>>( # #rjrQc~\rSrSrSr\S\\-4Sj5r\S5r \S\\-S\S-4Sj5r \ S 5r S r g) AutoHfQuantizerz The Auto-HF quantizer class that takes care of automatically instantiating to the correct `HfQuantizer` given the `QuantizationConfig`. rlc [U[5(a[RU5nURnU[ R :Xa<[U[5(d [S5eUR(aUS- nOUS- nU[;a,[SUS[[R5535e[UnU"U40UD6$)NzZFound `quant_method=bitsandbytes` but `quantization_config` is not a `BitsAndBytesConfig`.rYrXr[r\) isinstancer{rQrdrUrr^r TypeErrorrVrbr_rarc)rerlrrrUrgs rh from_configAutoHfQuantizer.from_configs )4 0 0"8"B"BCV"W *77  -<< <13EFFp#//' ' 5 51,@/44678:  ,L9 -888rjc R[R"U40UD6nURU5$)N)rQrnr)rerqrrrls rhrnAutoHfQuantizer.from_pretraineds*4DDEbmflm233rjquantization_config_from_argsNcJUbSnOSn[U[5(aA[U[5(a[R"U5nO[RU5nUbhUR R UR R :wa:[SUR R SUR R S35e[U[5(aq[U[5(a\UR5nUR5HupV[XU5 M U(a US[UR55S3- nUS:wa8[U[[[ 45(d["R$"U5 U$[&R)U5 U$)zt handles situations where both quantization_config from args and quantization_config from model config are present. zYou passed `quantization_config` or equivalent parameters to `from_pretrained` but the model you're loading already has a `quantization_config` attribute. The `quantization_config` from the model will be used.zThe model is quantized with z but you are passing a z| config. Please make sure to pass the same quantization config class to `from_pretrained` with different loading attributes.z"However, loading attributes (e.g. z]) will be overwritten with the one you passed to `from_pretrained`. The rest will be ignored.)rr{rrdrQ __class__rur_LOADING_ATTRIBUTES_CONFIG_TYPESget_loading_attributesitemssetattrrarcrrrwarningswarnloggerinfo)rerlr warning_msgloading_attr_dictattrvals rhmerge_quantization_configs*AutoHfQuantizer.merge_quantization_configss ) 4y  K )4 0 07II&5&?&?@S&T#&<&F&FGZ&[# * 5#--66:W:a:a:j:jj./B/L/L/U/U.VVmoLoVoVo_o_n``FF  )+J K KPZ )+JQ Q !> T T V .446 +377!!CDIZI_I_IaDbCcdA BB " Z0CkS^`tEu%v%v MM+ &#" KK $""rjc URSS5nURSS5(dURSS5(a/URSS5(aSOSn[RU-nOUc [S5eU[;a8[ R SUS [[R55S 35 gg ) NrUrVFrWrXrYrZr[r\z~. Hence, we will skip the quantization. To remove the warning, you can delete the quantization_config attribute in config.jsonT) r]rr^r_r`rwarningrarbrc)rSrUrfs rhsupports_quant_method%AutoHfQuantizer.supports_quant_methods/33NDI # ' ' > >BZB^B^_motBuBu 8 < <^U S SWY`F-<T[;a[STS35e[U[5(d [ S5eU[T'U$)NzConfig '' already registeredz*Config must extend QuantizationConfigMixin)r`r_ issubclassrr)rers rhregister_config_fn8register_quantization_config..register_config_fn-sK 5 5xx/CDE E#677HI I36(0 rjrm)rrs` rhregister_quantization_configr*s rjnamec^U4SjnU$)zRegister a custom quantizer.c>T[;a[STS35e[U[5(d [ S5eU[T'U$)Nz Quantizer 'rz!Quantizer must extend HfQuantizer)rbr_rr r)rers rhregister_quantizer_fn1register_quantizer..register_quantizer_fn=sJ ) ){4&0DEF F#{++?@ @'*t$ rjrm)rrs` rhregister_quantizerr:s ! rjcN[USS5SLnU(a&[RUR5(dSnU(dUbRU(a&[R URU5UlOXl[R URUS9nOSnUbUR UUS9 URU5nURU5nURU5n[URSS5(d&URRn[USU5US'X`U4$)NrlF) pre_quantized) device_map weights_only dequantizevaluequant) ror~rrlrrvalidate_environmentupdate_device_mapupdate_tp_planupdate_ep_planrU)configrlrr user_agentr hf_quantizerrUs rhget_hf_quantizerrJs&F$94@LM_BB6C]C]^^ +7 )8)S)S**,?*F &*= &&22  & &'3  ))!% * "33J? ,,V4,,V4|77uMM';;HHL"),"NJw   ++rj)]rmodels.auto.configuration_autorutilsrutils.quantization_configrrrr r r r r rrrrrrrrrrrrrrrrrbaser quantizer_aqlmr!quantizer_auto_roundr" quantizer_awqr#quantizer_bitnetr$quantizer_bnb_4bitr%quantizer_bnb_8bitr&quantizer_compressed_tensorsr'quantizer_eetqr(quantizer_fbgemm_fp8r)quantizer_finegrained_fp8r*quantizer_fouroversixr+quantizer_fp_quantr,quantizer_gemmar-quantizer_gptqr.quantizer_higgsr/ quantizer_hqqr0quantizer_metalr1quantizer_mxfp4r2quantizer_quantor3quantizer_quarkr4quantizer_sinqr5quantizer_spqrr6quantizer_torchaor7quantizer_vptqr8rbr`r get_loggerrurrQr~strrrrrmrjrhrs76+4'/22F+6@92++-)--/-++1+ <++ O   O     )" O  >6&!  !" O#$  $'$    9>$ 9$+$+$ J $  J $  J $ l$ [$$$ $ 9$1$/$ [$}$  !$" J#$$   !!    $3$ :  #   H %&#&#Rll^   !S ! !,rj