3j:%SrSSKJr SSKJr SSKJr SSKJr SSK r SSK J r J r SSK Jr \"\5rSq\S-\S 'S q\\S 'S*S jrS*SjrS \4SjrS\S \S-4SjrS\S \4SjrS\S \4Sjr\"\5r\\\\4\S'Sq \!\S-\S'S \!\4Sjr"/r#\\$\\\4\S'Sq%\\S'Sq&\S-\S'\S\'\(\'\4-4Sj5r)S*Sjr*S\ RVRXS S4S jr-S \(\\\44S!jr.S*S"jr/S#q0\\S$'0r1\(\\4\S%'S\ RVRdS \4S&jr3S'\'S \4S(jr4\S\ RVRdS\'\(\'\4-4S)j5r5g!\a SrGNNf=f)+aAnnotate CUDA graph kernel nodes during capture. During CUDA graph capture, ``mark_kernels`` uses ``cudaGraphGetNodes`` to count nodes before and after the wrapped region. Nodes at indices ``[before, after)`` are the ones added within the scope. Each kernel or memcpy node found is annotated by its ``toolsId`` so it can later be matched to profiler trace events. The annotations can be pickled and later merged into a Chrome profiler trace using ``torch.cuda._annotate_cuda_graph_trace``. Requires ``cuda.bindings`` package and a CUDA driver that supports ``cudaGraphNodeGetToolsId`` (CUDA >= 13.1 or appropriate cuda-compat). When unavailable, ``mark_kernels`` silently becomes a no-op. Usage during capture:: from torch.cuda._graph_annotations import ( enable_annotations, mark_kernels, resolve_pending_annotations, remap_to_exec_graph, ) enable_annotations() with torch.cuda.graph(graph): with mark_kernels("phase_A"): y = workload_a(x) with mark_kernels("phase_B"): z = workload_b(y) resolve_pending_annotations() remap_to_exec_graph(graph) ) defaultdict)contextmanager) getLogger)AnyN)_check_cuda_bindings_HAS_CUDA_BINDINGS)runtime_tools_id_availableF_annotations_enabledreturncSqg)z#Enable kernel annotation recording.TNr W/home/wildlama/miniconda3/lib/python3.13/site-packages/torch/cuda/_graph_annotations.pyenable_annotationsrAs  rcSqg)z$Disable kernel annotation recording.FNrrrrdisable_annotationsrGs !rc\[(dg[SLag[[S5(dgg)zBReturn True if we already know cudaGraphNodeGetToolsId is missing.TFcudaGraphNodeGetToolsId)rr hasattr _cuda_runtimerrr_is_tools_id_unavailablerMs*  e# ="; < < rnodec[c$[[R"U55nSqU$[[R"U55$![a Sq[ R S5 gf=f)za [RAS5 N8f=f![R5 Sqf=f)zResolve pending scope index ranges into kernel annotations. Enumerates all graph nodes and annotates kernel/memcpy nodes whose indices fall within recorded scope ranges. Must be called while still inside the ``torch.cuda.graph()`` capture context. NrBz6resolve_pending_annotations: no graph handle availablerr. c36# UHunup#nX#XA4v M g7fr7r).0iannstartends r .resolve_pending_annotations..s& ,F(A(CS$,Fsc"USUS*US*4$)Nr)ss r-resolve_pending_annotations.. s1Q4!A$1.r)keyrYz:resolve_pending_annotations: toolsId unavailable, abortingnamez"resolve_pending_annotations failed)!r<r=rrErFrGrHrIr,rwarningclearr3rr0r r>r;sorted enumeraterangepoplenrJcudaGraphNodeGetTyper4reversedrDdictitems setdefaultr exception)r!r(r2nodes first_tid annotatable sorted_scopes scope_ptr active_stackrRrS _start_idxend_idx_idxr node_typermergedr1akavs rresolve_pending_annotationsr|s; ? ' '::,,.::F v &E } }OPZe$ !8  ! ! #!N h g*  + +   7%eAh/I:Ci2o ",.  ,5o,F /   .0 sA<#3A#6!#;  "<#3A#6!#;c-00]5Ma5PTU5U1>1I.; ''7Q c-00]5Ma5PTU5U  8D,22I +$T*HP %%'!%* )< A%#H-44\!_Q5GH*,&|4FAs!#t,,&)iikFB"--b"5'2))&#6 5 $H-44V? s@ 'L, B3L,?AL,A-L,B9L,,M M MMM)torch_cuda_graphc[(dg[R"UR5S9n[ [R "U55n[ n0n[R5HLupVUS- nUb Xs:waXdU'MUS-nUS-U-n X;aXIRU5 M?[U5XI'MN [R5 [RU5 g)aHRemap annotation keys from capture graph ID to exec graph ID. During capture, toolsId encodes the capture graph's ID in the upper 32 bits. After instantiation, the profiler uses the exec graph's ID. This function rewrites the keys so annotations match the trace. Must be called after the ``torch.cuda.graph()`` context exits. NrBrOl) r4rcudaGraphExec_traw_cuda_graph_execrcudaGraphExecGetIdr>rlextendlistrcupdate) r} exec_handle exec_graph_idcapture_graph_idremappedrann_listgraph_idnode_id new_tools_ids rremap_to_exec_graphrIs  //#779K)((  M.%'H1779r>  'H,H!)X  Z'%+w6  #  " ) )( 3%)(^H ":x(rc[$)zBReturn the current kernel annotation map (toolsId -> annotations).)r4rrrget_kernel_annotationsrts rcX[R5 [R5 Sqg)z9Clear all recorded kernel annotations and pending scopes.N)r4rcr<r=rrrclear_kernel_annotationsrys!Nr<_stream_id_counter_stream_id_mapclURnU[;a[[U'[S- q[U$)z;Return a small, stable stream ID for the given CUDA stream.rY)rIrr)r!r_s r_get_stream_idrs7   C . 0sa # rpg_keycTU[;a[[U'[S- q[U$)z:Return a unique stream ID for the given process group key.rY)rr)rs rget_stream_for_pgrs,^#!3va & !!rc## [(d-[RRU5 Sv SSS5 gUR[RR 5R:Xa[ U5 Sv SSS5 g[U[5(aSU0n[U[5(a[U5US'[RRU5 [ U5 Sv SSS5 SSS5 g!,(df  g=f!,(df  g=f!,(df  N9=f!,(df  g=f7f)u@Switch to stream, inject its ID into annotation, and mark kernels. If *stream* is already the current stream, no stream switch or stream ID injection happens — the kernels stay on whatever stream is active (which keeps the trace faithful when e.g. FSDP uses the current stream for copy-in instead of a separate one). NrAr!) r rFrGr!rIrHrMrDrArkr)r!r?s r mark_streamrs  ZZ  v & ' UZZ668DDD * % & % j# & &,J j$ ' '#1 Jx ZZ  v &j)*' &' & % %*)' &se+E DA E <DA$E % D:1D)6D:> E  DE  D&"E ) D7 3D:: EE )r N)6__doc__ collectionsr contextlibrloggingrtypingrrFtorch.cuda._utilsrr cuda.bindingsr r ImportError__name__rr bool__annotations__r rrrintr r,r3rr4r5setr;r<tupler=r>rArkrMr|rG CUDAGraphrrrrrStreamrrrrrrrs1"H$% F 8  $(TD['#d" ! $d 4 s s 334?t3D[d3i0D'+CHtO*C/1eCcM*+0&*d ))ES4S>1)E)EXrj()%***>*>()4()VS$s)^ 4 C!#S#X#5::,,"c"c" ))sT#s(^7KU MsE22E>=E>