3jHSrSSKrSSKrSSKJr SSKJrJrJrJ r J r J r J r Sr SrSrSrS \S \\\44S jrS \ \\\4S \\\44S jr\"SS9"SS55rS\S\ \S\ \S \4SjrS\S \4SjrS\\\4S \\\44SjrS\ \\\\4\\\\4S \ \\\\\444SjrS\ \ \\\\4S\\\4S\S\ \S\S \ \ \\ \\4\4\\\444 S jrS\ \\\\\\44S \ \\ \44S!jrS\ \\\\4\\\4S \ \\ \ \\4\\\444S"jrS\ \ \\4S\\\4S\S\S \ \ \ \ \ \\4\\\444 S#jrS\ \ \\4S$\\\4S\S \ \ \ \4S%jr g)&a Internal multiprocessing helpers for DeepDiff. Phase 1 scope: parallelize the (added_hash x removed_hash) rough-distance loop in ``DeepDiff._get_most_in_common_pairs_in_iterables`` for ``ignore_order=True``. Determinism contract (see docs/multi_processing.md): - Pair selection happens in the parent only. - Workers compute distances. The parent submits jobs in a stable index order matching the serial nested loop and merges results by that index. - Worker completion order (``as_completed``) never affects the public output. Only module-level callables live here so the module is safe under the ``spawn`` start method (macOS/Windows). N) dataclass)AnyCallableDictListOptionalTuplecast@)z DIFF COUNTz PASSES COUNTzDISTANCE CACHE HIT COUNT)zMAX PASS LIMIT REACHEDzMAX DIFF LIMIT REACHED diff_instancereturnc[USS5=(d 0n0n[H)n[URUS5=(d S5X#'M+ [H n[ URUS55X#'M" U$)aSPull a small, picklable stats snapshot off a worker-local DeepDiff. Returns a dict with integer counters plus boolean limit flags. Missing keys are tolerated so this stays robust if ``_stats`` shrinks at the end of ``__init__`` (it currently deletes ``DISTANCE CACHE ENABLED`` and the ``PREVIOUS *`` bookkeeping keys before we get here). _statsNrF)getattr_WORKER_STATS_COUNTER_KEYSintget_WORKER_STATS_FLAG_KEYSbool)r statsdeltakeys S/home/wildlama/miniconda3/lib/python3.13/site-packages/deepdiff/_multiprocessing.py_extract_worker_statsr"sh M8T 2 8bEE)3*/a0 *&%))C/0 ' Ldeltasc J[Vs0sHoS_M nn[HnSX!'M UHpnU(dM [H1nX!==[URUS5=(d S5- ss'M3 [HnURU5(dMSX!'M! Mr U$s snf)z?Sum counter keys and OR-merge limit flags across worker deltas.rFT)rrrr)rroutrs r_aggregate_worker_statsr 3s-GH-Gc6-GCH&' -C HEIIc1-23 3H.*Cyy~~+  JIs B T)frozencJ\rSrSr%Sr\\S'\\S'\\S'S\S\4SjrS r g ) MPConfigCz2Normalized internal multiprocessing configuration.enabledworkers thresholdn_jobsrcpUR=(a$ URS:=(a XR:$)Nr%r&r')selfr(s rshould_parallelizeMPConfig.should_parallelizeJs&||M q 0MV~~5MMrN) __name__ __module__ __qualname____firstlineno____doc__r__annotations__rr-__static_attributes__r/rrr#r#Cs*< M LNNNNrr#multiprocessingmultiprocessing_workersmultiprocessing_thresholdc~US;a[SU<35e[U5nUc/[R"5=(d Sn[ [ U5nO,[ U[5(aUS:a[SU<35eUnUc[nO,[ U[5(aUS:a[SU<35eUn[X5US9$)zValidate and normalize the public multiprocessing parameters. ``multiprocessing`` accepts True/False. ``multiprocessing_workers`` accepts None or a positive int. ``multiprocessing_threshold`` accepts None or a non-negative int. )TFrr*z+multiprocessing must be True or False; got r*z@multiprocessing_workers must be None or a positive integer; got rzFmultiprocessing_threshold must be None or a non-negative integer; got r+) ValueErrorros cpu_countminDEFAULT_MAX_WORKERS isinstancerDEFAULT_THRESHOLDr#)r7r8r9r%cpur&r's rnormalize_mp_configrCNs11?N P  ?#G&lln!)3/1377;RUV;V*- * (% 3S99=VYZ=Z,/ . G JJrobjcR[R"U5 g![a gf=f)zReturn True if ``obj`` round-trips through ``pickle.dumps`` cleanly. Used to decide whether parallel execution is safe for a given input. A False result triggers serial fallback for that section. TF)pickledumps Exception)rDs r is_pickleablerIws(  S s  && parameterscb[U5nSUS'[SSSS9US'URSS5 URS S5 URS S5 URS S5 URS S5 URS S5 URSS5 URSS5 U$)aStrip parent-process-only state from a ``_parameters`` snapshot. The parent's ``_parameters`` may carry references that should not be reused inside a worker (mutable shared caches) or that would cause nested multiprocessing inside the worker. This produces a copy safe to ship. Fr7r*rr+ _mp_config_distance_cacheNhashes _numpy_pathsr group_by_keystree_iterable_opcodesis_root)dictr#pop)rJ sanitizeds r_sanitize_parameters_for_workerrWsZ I $)I &ua1MIl MM#T* MM(D! MM.$' MM(D! MM/4( MM&$ MM%t, MM)T" rjobc SSKJn SSKJn Uup4pVpxU"UUUUUUSS9n U[ [ U R 55[U 54$)aMCompute the rough distance between two items in a worker process. ``job`` layout matches what ``compute_distances_parallel`` ships: ``(job_index, sanitized_parameters, removed_item, added_item, original_type, iterable_compare_func)``. The worker constructs a fresh root ``DeepDiff`` (no shared parent state), requests the DELTA_VIEW so we hit the same code path as the serial call in ``_get_rough_distance_of_hashed_objs``, and returns the resulting float plus a ``_extract_worker_stats`` snapshot so the parent can aggregate diff/pass/cache-hit counts into its WORKER_* stats keys. rDeepDiff) DELTA_VIEW) _parametersview_original_typeiterable_compare_funccache_purge_level) deepdiff.diffr[deepdiff.helperr\r float_get_rough_distancer) rXr[r\ job_indexrJ removed_item added_item original_typer`diffs r_distance_workerrks]"'*\_YI<]  $3  D d5$":":"<=?TUY?Z ZZrjobsrir`configc U(d 0[/54$[U5n[U5(dgUb[U5(dg[US5(dgSSKJnJn /n[ U5H#upU Sn U Sn URXXX#45 M% 0n /nU"URS9nUVs/sHnUR[U5PM nnU"U5H-nUR5unnnUU U'URU5 M/ SSS5 0n[ U5HupXUU SU S4'M U[U54$s snf!,(df  NE=f![R[[4a gf=f)aRun ``_distance_worker`` over ``jobs`` and return distances by pair. ``jobs`` is a list of ``(added_hash, removed_hash, added_item, removed_item)`` tuples in the exact order the serial nested loop visits them. The parent is responsible for that ordering; this helper does not reorder anything. Returns: ``(distances_by_pair, aggregated_worker_stats)`` where the first item is a dict ``{(added_hash, removed_hash): distance}`` and the second is the aggregated ``_extract_worker_stats`` snapshot summed across all workers (counter keys summed, limit flags OR-merged). Returns ``None`` if the section is unsafe to parallelize (unpickleable inputs/parameters, worker import error, etc.). On ``None`` the caller MUST fall back to the serial path so correctness is preserved. Workers may finish out of order; we collect results into a dict keyed by the original job index, so callers see the same result regardless of completion order. NrProcessPoolExecutor as_completed max_workersr*)r rWrIconcurrent.futuresrprq enumerateappendr&submitrkresultrF PicklingErrorAttributeError TypeError)rlrJrir`rmsanitized_paramsrprqpayloadsirXrhrgresults_by_index stats_deltasexecutorpayloadfuturesfutureidxdistance stats_deltars rcompute_distances_parallelrs4 *2...6zB ) * *(?T1U1U a ! !EHD/V 1v  ,M a "*,)+L  V^^ <QYZQYgx'7AQYGZ&w/.4]]_*X{(0 %##K0 0=)+CD/ 0 3SVSV " ' 5 55![= <  .) < sBE*E / E;E  EE  EEEE<;E<cSSKJn SSKJn Uup4pVU"U4SUSS.UD6nXtnXLaUS4$X84$![a US4s$f=f)uIHash a single iterable item in a worker process. ``job`` layout: ``(job_index, item, parent_path, deephash_parameters)``. The worker constructs a fresh ``DeepHash`` (no shared parent state) and looks up the resulting top-level hash for ``item``. Returns ``(job_index, item_hash)`` where ``item_hash`` is None if the item could not be processed — the parent treats that exactly like the serial path's ``KeyError`` / ``unprocessed`` skip. UnicodeDecodeError and NotImplementedError propagate as in the serial path; other exceptions surface in the parent through ``future.result()``. r)DeepHash) unprocessedNT)rNparent apply_hash)deepdiff.deephashrrcrKeyError) rXrrrfitem parent_pathrJ deep_hash item_hashs r _hash_workerrs{++/2,I[    IO $   $s/AAc SSKJn SSKJn Uup4pVnU"XVUUUSS9n/n URR 5H(upU S:XaM U Hn U R X45 M M* X9[U54$)aiRun one paired-item subtree diff in a worker process. ``job`` layout: ``(job_index, sanitized_parameters, t1, t2, _original_type)``. The worker constructs a fresh root ``DeepDiff`` (no shared parent state), requests the TREE_VIEW so ``self.tree`` is populated and walks it once to flatten the leaves into ``[(report_type, leaf_difflevel), ...]``. The parent rebases each leaf's up-chain onto its own ``change_level`` so paths come out as if the diff had run inline. Returning bare DiffLevel objects is acceptable here because we already proved they pickle and re-attach cleanly (see tests/test_multiprocessing.py). rrZ) TREE_VIEW)r]r^r_ra deep_distance)rbr[rcrrQitemsrxr) rXr[rrfrJt1t2r_rjentries report_typelevelsleafs r_subtree_diff_workerr2s"')471I2>   % D&(G#yy0 / ) D NNK. / 1 4T: ::rc U(d /[/54$[U5n[U5(dg[US5(dgSSKJnJn [ U5VVV s/sH unupXtXU4PM n nnn 0n /n U"URS9n U Vs/sHoR[U5PM nnU"U5H-nUR5unnnUU U'U RU5 M/ SSS5 [![#U55Vs/sHo{UPM sn[U 54$s sn nnfs snf!,(df  NL=f![R[[4a gf=fs snf)uRun ``_subtree_diff_worker`` over ``jobs`` and return per-job entries. ``jobs`` is a list of ``(t1_item, t2_item)`` tuples in the exact order the serial paired-iteration code visits them. Returns ``(entries_by_job, aggregated_worker_stats)`` where ``entries_by_job`` is a list aligned to job order — each element is ``[(report_type, leaf_difflevel), ...]`` suitable for the parent to rebase and merge into its tree — and ``aggregated_worker_stats`` is the per-batch ``_stats`` deltas summed across workers (counters summed, limit flags OR-merged). Returns ``None`` when the section is unsafe to parallelize (unpickleable parameters/items, worker import error). On ``None`` the caller MUST run the same jobs serially so correctness is preserved. Workers may finish out of order; results are collected by their original job index so the merge order is identical regardless of completion order. Nrrort)r rWrIrvrprqrwr&ryrrzrxrFr{r|r}rangelen)rlrJrirmr~rprqrt1_itemt2_itemrrrrrrrrrrs rcompute_subtree_diffs_parallelrYsi, *2...6zB ) * * a ! !D&/t_%4 !A! g >%4  :<)+L V^^ <U]^U]'';WEU]G^&w/,2MMO)Wk(/ %##K00=',CI&67&6! &67 - #_= <  .) < 8sND"9D? D.D)-;D.(D?E$)D.. D<8D?<D??E! E!deephash_parametersc bU(d/$[U5(dg[US5(dgSSKJnJn [ U5VVVs/sH unupgXVXq4PM nnnn0n U"UR S9n UV s/sHoR [U 5PM n n U"U 5Hn U R5upXU'M SSS5 [[U55Vs/sHoYUPM sn$s snnnfs sn f!,(df  NA=f![R[[4a gf=fs snf)uRun ``_hash_worker`` over ``jobs`` and return per-item hashes. ``jobs`` is a list of ``(item, parent_path)`` tuples in the exact order the serial enumerate-loop visits them. Returns a list aligned to that order, with ``None`` for items the worker could not hash. Returns ``None`` when the section is unsafe to parallelize (unpickleable parameters/items, worker import error). On ``None`` the caller MUST fall back to the serial path. Workers may finish out of order; results are collected by their original index so callers see the same output regardless of completion order. Note: child object hashes computed inside each worker are NOT merged back into the parent's ``self.hashes`` — id-based keys for unhashable sub-objects would not match across process boundaries. Parent code that relies on the iterable-level hash being present must continue to compute it serially after the per-item parallel pass. Nrrort)rIrvrprqrwr&ryrrzrFr{r|r}rr)rlrrmrprqrrrrrrrrrrrs rcompute_hashes_parallelrs#,  , - - a ! !D'0o&5 "A" +3&5  24 V^^ <MUVX'|W=XGV&w/!'(1%0=*/s4y)9 :)9AQ )9 ::W= <  .) < ;sNC* D0C65C1'C6;DD,1C66 DDDD)(D))!r4r<rF dataclassesrtypingrrrrrr r r?rArrstrrr r#rrCrrIrWrdrkrrrrrr/rrrs'  !CCC XNc3h" Dc3h$8 T#s(^   $NNN&K&K%c]&K (}&K &KR s t S#X4S>2"[ sDcNCc36 7"[ 3tCH~ %&"[JK6 uS#sC'( )K6S#XK6K6$H- K6  K6 eDsCx%/0$sCx.@AB K6\ eCc4S>9: uS(SV-EW?X D$; sDcNCc1 2$; 3U38_%tCH~ 56$;N8 uS#X 8S#X88  8 eDeCHo./c3h?@A 8v1; uS#X 1;c3h1; 1;d8C=!" 1;r