9j>SSKrSSKrSSKrSSKrSSKrSSKJrJr SSKrSSK J r SSK J r SSK JrJrJr SSKJrJr \"SS S 9rS r\(aSSKJs Jr SSKrS r\r\R<R?5r SKS jr!S \"4Sjr#Sr$Sr%Sr&Sr'Sr(Sr)Sr*Sr+Sr,Sr-Sr.Sr/\SLSj5r0Sr1Sr2Sr3SMSjr4S r5S!r6S"r7S#r8SKS$jr9S%r:S&r;S'rS*r?S+r@S,rAS-rBS.rCS/rDS0rES1rFS2rGS3rHS4rIS5rJS6rKS7rLS8rMS9rNS:rOSNS;jrP\SMS<j5rQ\SMS=j5rR\SMS>j5rS\SMS?j5rT\SMS@j5rUSArV\SMSBj5rW\SMSCj5rXSDrYSErZSFr[SGr\SHr]SIr^SOSJjr_g!\a GN&f=f)PN) lru_cachewraps)version)parse)parse_flag_from_envpatch_environment str_to_bool)compare_versionsis_torch_version USE_TORCH_XLAT)defaultFc[RRU5SLnU(a&[RRUcUOU5ngg![RRa gf=f)NTF) importlibutil find_specmetadataPackageNotFoundError)pkg_name metadata_namepackage_exists_s R/home/wildlama/miniconda3/lib/python3.13/site-packages/accelerate/utils/imports.py_is_package_availabler2si^^--h7tCN ""++ 8MHS`aA  !!66  s$AA10A1returnc[$N)_torch_distributed_availableris_torch_distributed_availabler!>s ''r cv[SS5(a([RRR 5$g)N>=z2.7.0F)r torch distributeddistributed_c10dis_xccl_availablerr rr'r'Bs-g&&  11CCEE r c[S5$)N import_timerrrr ris_import_timer_availabler+H  00r c>[S5=(d [SS5$)Npynvmlz nvidia-ml-pyr*rr ris_pynvml_availabler/Ls  * ].CHn.]]r c[S5$)Namdsmir*rr ris_amdsmi_availabler2P  **r c[RRSL=(a [RR 5$r)r$rhipcuda is_availablerr ris_rocm_availabler8Ts) ==  D ( FUZZ-D-D-FFr c[S5$)Npytestr*rr ris_pytest_availabler;Xr3r c[SS5$)Nmsampzms-ampr*rr ris_msamp_availabler>\s ( 33r c[S5$)N schedulefreer*rr ris_schedulefree_availablerA`r,r cP[5(a [SS5$[SS5$)Nintel_transformer_enginezintel-transformer-enginetransformer_enginetransformer-engine)is_hpu_availablerrr ris_transformer_engine_availablerGds)$%?A[\\$%9;OPPr cF[SS5(aSSKJn U"5S$g)NrDrErcheck_mxfp8_supportF)rtransformer_engine.pytorch.fp8rJrIs r%is_transformer_engine_mxfp8_availablerLks%13GHHF"$Q'' r c[S5$)N lomo_optimr*rr ris_lomo_availablerOs  ..r c[SS9 [RR5nSSS5 U$!,(df  W$=f)zw Checks if `cuda` is available via an `nvml-based` check which won't trigger the drivers and leave cuda uninitialized. 1)PYTORCH_NVML_BASED_CUDA_CHECKN)r r$r6r7) availables ris_cuda_availablerUws<  =JJ++-  >  > = s 3 AcU(aU(aS5e[(dgU(a![RR5S;$U(a![RR5S:H$g)z Check if `torch_xla` is available. To train a native pytorch job in an environment with torch xla installed, set the USE_TORCH_XLA to false. z6The check_is_tpu and check_is_gpu cannot both be true.F)GPUCUDATPUT)_torch_xla_available torch_xlaruntime device_type) check_is_tpu check_is_gpus ris_torch_xla_availabler`s[ h0hh .     ,,./AA   ,,.%77 r c[S5nU(a@[R"[RRS55n[ USS5$g)Ntorchaor#z0.6.1Frrrrrr )rtorchao_versions ris_torchao_availableres@*95N!-- (:(:(B(B9(MNw?? r c[S5$)N deepspeedr*rr ris_deepspeed_availablerh  --r c[SS5$Nr#z2.4.0r rr ris_pippy_availablerms D' **r c[SS9(aU(+$[5(a[RR 5$[ 5(a[R R 5$[5(a[RR 5$[5(a*[RRRSS5$g)z8Checks if bf16 is supported, optionally ignoring the TPUT)r^r) r`rUr$r6is_bf16_supportedis_mlu_availablemluis_xpu_availablexpuis_mps_availablebackendsmpsis_macos_or_newer) ignore_tpus ris_bf16_availablerzs40~zz++--yy**,,yy**,,~~!!33B:: r c$[5(agg)zChecks if fp16 is supportedFT)is_habana_gaudi1rr ris_fp16_availabler}s r cZ[5=(d [5=(d [5$)zChecks if fp8 is supported)r>rGrerr ris_fp8_availablers   ^#B#D ^H\H^^r c[S5nU(a@[R"[RRS55n[ USS5$g)N bitsandbytesr#z0.39.0Frcr bnb_versions ris_4bit_bnb_availabler@*>:NmmI$6$6$>$>~$NO  T8<< r c[S5nU(a@[R"[RRS55n[ USS5$g)Nrr#z0.37.2Frcrs ris_8bit_bnb_availablerrr c[S5nU(aCUb@[R"[RRS55n[ USU5$U$)Nrr#rc) min_versionrrs ris_bnb_availablersH*>:N+1mmI$6$6$>$>~$NO  T;??r cZ[5(dgSSKnS[US[55;$)NFr multi_backendfeatures)rrgetattrset)bnbs r'is_bitsandbytes_multi_backend_availablers'    gc:su= ==r c[S5$)N torchvisionr*rr ris_torchvision_availablers  //r c[[RRSS55S:Xa}[R R S5b\[[RRS55n[USS5(a [R R SS5$ggg![a#n[R"S U35 SnAg SnAff=f) NACCELERATE_USE_MEGATRON_LMFalsermegatronz megatron-corer#0.8.0z .trainingz)Parse Megatron version failed. Exception:F)r osenvirongetrrrrrrr Exceptionwarningswarn)megatron_versiones ris_megatron_lm_availablers2::>>">HIQN >> # #J / ; #(););)C)CO)T#U #$4dGDD$>>33KLLE <O   I!MN sAB,, C6CCc[S5$)N transformersr*rr ris_transformers_availablerr,r c[S5$)Ndatasetsr*rr ris_datasets_availabler  ,,r c[S5$)Npeftr*rr ris_peft_availabler  ((r c[S5$)Ntimmr*rr ris_timm_availablerrr cN[5(a [SS5$[S5$)Ntritonz triton-xpu)rsrrr ris_triton_availablers#$X|<<  **r c[S5nU(a@[R"[RRS55n[ USS5$g)Naim$>u$EF  S':: r c<[S5=(d [S5$)N tensorboard tensorboardXr*rr ris_tensorboard_availablers  / X3H3XXr c[S5$)Nwandbr*rr ris_wandb_availabler  ))r c[S5$)Ncomet_mlr*rr ris_comet_ml_availablerrr c[S5$)Nswanlabr*rr ris_swanlab_availabler  ++r cL[RS:=(a [S5$)N) trackio)sys version_inforrr ris_trackio_availablers   w & K+@+KKr c[S5$)Nboto3r*rr ris_boto3_availabler#rr c<[S5(a [SS5$g)NrichACCELERATE_ENABLE_RICHF)rrrr ris_rich_availabler'sV$$"#;UCC r c[S5$)N sagemakerr*rr ris_sagemaker_availabler-rir c[S5$)Ntqdmr*rr ris_tqdm_availabler1rr c[S5$)Nclearmlr*rr ris_clearml_availabler5rr c[S5$)Npandasr*rr ris_pandas_availabler9r3r c[S5$)N matplotlibr*rr ris_matplotlib_availabler=rPr c[S5(ag[RRS5b![RR S5ngg![RR a gf=f)NmlflowTz mlflow-skinnyF)rrrrrr)rs ris_mlflow_availablerAshX&&~~)5 ""++O>3E3E3R3R3T vY^YgYgYkYkYtYtYvvr c[RRS5cgSSKn[ SS9 [ R R5nSSS5 U$!,(df  W$=f)zv Checks if `mlu` is available via an `cndev-based` check which won't trigger the drivers and leave mlu uninitialized. torch_mluNFrrR)PYTORCH_CNDEV_BASED_MLU_CHECK)rrrrr r$rrr7) check_devicerrTs rrqrqUsW ~~ ,4  =II**,  >  > = s A A'c^[RRS5cgSSKnU(a=[R R 5n[R R5$[[S5=(a [R R5$![a gf=f)zSChecks if `torch_musa` is installed and potentially if a MUSA is in the environment torch_musaNFrmusa) rrrrr$r device_countr7 RuntimeErrorhasattr)rrrs ris_musa_availablerf~~ -5  '')A::**, , 5& ! ?ejj&=&=&??  ;B B,+B,c[RRS5cgSSKnU(a=[ R R5n[ R R5$[[ S5=(a [ R R5$![a gf=f![a gf=f)zQChecks if `torch_npu` is installed and potentially if a NPU is in the environment torch_npuNFrnpu) rrrrrr$rrr7rr)rrrs ris_npu_availablerxs~~ ,4  &&(A99))+ + 5% =UYY%;%;%==   s"B ;B0 B-,B-0 B=<B=c^[RRS5cgSSKnU(a=[R R 5n[R R5$[[S5=(a [R R5$![a gf=f)zSChecks if `torch_sdaa` is installed and potentially if a SDAA is in the environment torch_sdaaNFrsdaa) rrrrr$rrr7rr)rrrs ris_sdaa_availablerrrc[RRS5b [RRS5cgSSKnU(a SSKJs Js Jn [[ S5=(a [ RR5$)zQChecks if `torch.hpu` is installed and potentially if a HPU is in the environmenthabana_frameworksNzhabana_frameworks.torchFrhpu) rrrhabana_frameworks.torch(habana_frameworks.torch.distributed.hcclr$r%hcclrrr7) init_hcclrrs rrFrFs^   !45= >> # #$= > F"?? 5% =UYY%;%;%==r c[5(a5SSKJs Js Jn UR 5UR R:Xagg)NrTF)rF*habana_frameworks.torch.utils.experimentalr$utils experimental_get_device_type synDeviceTypesynDeviceGaudi)htexps rr|r|s4BB  ! ! #u':':'I'I I r c8[SS5(agU(a=[RR5n[RR 5$[ [S5=(a [RR 5$![ a gf=f)zr Checks if XPU acceleration is available via stock PyTorch (>=2.7) and potentially if a XPU is in the environment z<=z2.6Frt)r r$rtrr7rr)rrs rrsrssve$$  &&(A99))+ + 5% =UYY%;%;%==  s;B BBc^[RRS5cgU(aASSKn[R R 5n[R R5$[[S5=(a [R R5$![a gf=f)N torch_neuronxFrneuron) rrrrr$rrr7rr)rrrs ris_neuron_availablers~~08   ))+A<<,,. . 5( # C (A(A(CC  s?B B,+B,c[S5$)Ndvcliver*rr ris_dvclive_availablerrr c[S5$)N torchdatar*rr ris_torchdata_availablerrir c[S5nU(a@[R"[RRS55n[ USS5$g)Nrr#rFrc)rtorchdata_versions r*is_torchdata_stateful_dataloader_availablersB*;7N#MM)*<*<*D*D[*QR 14AA r c0^[T5U4Sj5nU$)z[ A decorator that ensures the decorated function is only called when torchao is available. cH>[5(d [S5eT"U0UD6$)Nze`torchao` is not available, please install it before calling this function via `pip install torchao`.)re ImportError)argskwargsfuncs rwrapper!torchao_required..wrappers.#%%w T$V$$r rr!r"s` rtorchao_requiredr&s"  4[%% Nr c0^[T5U4Sj5nU$)z[ A decorator that ensures the decorated function is only called when deepspeed is enabled. c>SSKJn SSKJn UR0:wa*U"5R UR :wa [S5eT"U0UD6$)Nr)AcceleratorState)DistributedTypez|DeepSpeed is not enabled, please make sure that an `Accelerator` is configured for `deepspeed` before calling this function.)accelerate.stater)accelerate.utils.dataclassesr* _shared_statedistributed_type DEEPSPEED ValueError)rr r)r*r!s rr"#deepspeed_required..wrapper sS5@  ) )R /4D4F4W4W[j[t[t4t0 T$V$$r r$r%s` rdeepspeed_requiredr2s"  4[ % % Nr c[SS5$rkrlrr ris_weights_only_availabler4s D' **r cl[[RRS55n[ USU5$)Nnumpyr#)rrrrr )r numpy_versions ris_numpy_availabler8s,),,44W=>M M4 ==r r)FF)F)z1.12)z1.25.0)`rimportlib.metadatarrr functoolsrrr$ packagingrpackaging.versionr environmentrr r versionsr r r rZtorch_xla.core.xla_modelcore xla_modelxmtorch_xla.runtimer[r_tpu_availabler%r7rrboolr!r'r+r/r2r8r;r>rArGrLrOrUr`rerhrmrzr}rrrrrrrrrrrrrrrrrrrrrrrrrrrurqrrrrFr|rsrrrrr&r2r4r8rr rrFsK & #LL8$OTB  -- # & %00==? (( 1^+G+41Q/  ".+ _ >0 1-))+ Y*-,L* .),+/ w      @ @" > >, @ @"  >  >  > >& D D",.  "(+ >m    sE<<FF