3j1bSrSSKJr SSKrSSKJr SSKJr SSKJ r SSK J r SS K J r Jr \ (a SS KJr S S KJr 0r\\"S S9"SS555r"SS5r\"S5rSSSjjrSSjrSSSjjr\R4"5rS SjrSSS.Sjjrg)!z Contains the logic for automatic additional output capture with our forward decorators. This mostly describe the hooks used and the logic to make capture thread/context safe. ) annotationsN) ContextVar) dataclasswraps) TYPE_CHECKING)is_torchdynamo_compilingrequires)nnPreTrainedModel)torch)backendsc\\rSrSr%SrS\S'SrS\S'SrS \S 'SrS \S 'S r S \S'Sr g)OutputRecorder'ar Configuration for recording outputs from a model via hooks. Attributes: target_class (Type): The class (e.g., nn.Module) to which the hook will be attached. index (Optional[int]): If the output is a tuple/list, optionally record only at a specific index. layer_name (Optional[str]): Name of the submodule to target (if needed), e.g., "transformer.layer.3.attn". class_name (Optional[str]): Name of the class to which the hook will be attached. Could be the suffix of class name in some cases. capture_initial_hidden_state (bool): Whether to prepend the first module's input as the initial hidden state. ztype[nn.Module] target_classrintindexN str | None layer_name class_nameTboolcapture_initial_hidden_state) __name__ __module__ __qualname____firstlineno____doc____annotations__rrrr__static_attributes__r]/home/wildlama/miniconda3/lib/python3.13/site-packages/transformers/utils/output_capturing.pyrr's7 "!E3N!J !!J !)- $-r%rc0\rSrSrSrSrSrSrSrSr g) CompileableContextVar<a Convenience wrapper around a ContextVar for usage with `torch.compile`. This behaves exactly as a `ContextVar`, except when compilation is triggered in which case it behaves as a simple global variable. This is useful as `torch.compile` cannot trace the `get` method of `ContextVar`. This however means that the access to the underlying variable is not thread-safe when compilation is triggered. c>[USS9UlSUlSUlg)N)defaultF)r context_var global_var compiling)selfnames r&__init__CompileableContextVar.__init__Ds%dD9r%cpUR(a UR$URR5$N)r.r-r,get)r/s r&r5CompileableContextVar.getIs( >>?? "##'') )r%cr[5(aXlSUlgURR U5$)NT)r r-r.r,set)r/values r&r8CompileableContextVar.setPs1 # % %#O!DN##''. .r%cUR(dUcSUlSUlgURRU5 g)NF)r.r-r,reset)r/tokens r&r<CompileableContextVar.resetXs/ >>U]"DO"DN    " "5 )r%)r.r,r-N) rrr r!r"r1r5r8r<r$rr%r&r(r(<s */*r%r(output_collectorTc<^^^UUU4SjnURU5 g)zaInstall the forward hook needed to capture the output described by `key` and `index` in `module`.cZ>[R5nUbTUR5;agT(a/TS:Xa)[UT5S:XaUTR US5 [ U[ 5(dUTR U5 gUTbUTR UT5 gg)N hidden_statesr)_active_collectorr5keyslenappend isinstancetuple)moduleargsoutputcollected_outputsrrkeys r&output_capturing_hook;install_output_capuring_hook..output_capturing_hookis-113  $3D3I3I3K(K  'C?,BsK\]`KaGbfgGg c " ) )$q' 2&%(( c " ) )& 1 E] & c " ) )&- 8'r%N)register_forward_hook)rIrMrrrNs ``` r&install_output_capuring_hookrQds 9   !67r%cSSKJn UR5H6upE[XS5(d[ XQSU3U5 M([ XQSU3S9 M8 UHupgUR b[XR 5(d1URcM;URUR5(dM]URbURU;aM|[XURUR5 M g)a Recursively install all output capturing hooks on all submodules of `parent_module`. Note that we need to use this recursive approach instead of simply iterating over all modules, because we want to respect the `capture_tasks` of all individual submodels (`PreTrainedModel` instances) in the graph. That is, once we reach a submodel in the graph, its children should use this submodel's `capture_tasks`, but other parts of the graph should not. r r.)prefixN) modeling_utilsrnamed_childrenrGrecursively_install_hooks"install_all_output_capturing_hooksrrendswithrrQrr) parent_module module_name capture_tasksrr0rIrMspecss r&rWrWzs1&446 &22 %f Qtf.E} U /v Qtf>U V 7$     *z-I[I[/\/\    ([-A-A%BRBR-S-S+0@0@ 0S (U[[%JlJl m$r%c[R[UR55=(d 0n/nUR 5HupE[ U[ 5(dU/nUHpn[ U[5(dFSU;aSOSn[ U[5(dSOUn[ U[5(dUOSn [XUS9nURXF45 Mr M UbUOSn[XU5 [USS5 g) z Install the output recording hooks on all the modules in `model`. This will take care of correctly dispatching the `_can_record_outputs` property of each individual submodels in case of composite models. rBrr N)rrr!_output_capturing_hooks_installedT) _CAN_RECORD_REGISTRYr5str __class__itemsrGlistrrFrWsetattr) modelrT capture_flagsr\rM layer_specsr]rrrs r&rXrXs ),,S-ABHbMM)//1+t,,&-K Ee^44,3)3E3)?)?TU ,6uc,B,Bu &LZde  # . !2)VrFe]; E6=r%c[USS5(ag[ [USS5(a SSS5 g[U5 SSS5 g!,(df  g=f)z Check if the model already has output capturing hooks installed, and install them if it is not already the case. Note that this is thread-safe, in case 2 (or more) threads want to install them concurrently. r`FN)getattr_hook_installation_lockrX)rgs r&maybe_install_capturing_hooksrmsMu95AA  5=u E E  ! +51 ! sA  A  A)tie_last_hidden_statesc*^U4SjnUbU"U5$U$)a Decorator to intercept specific layer outputs through hooks. The hooks are installed only once and lazily, the first time output capture is requested with the `output_xxx` kwargs/config. The implementation is fully context/thread safe, except when using `torch.compile`, as dynamo is unable to trace through `ContextVar` methods. Args: tie_last_hidden_states (`bool`, *optional*, defaults to `True`): Whether to overwrite `out.hidden_states[-1]` with the `out.last_hidden_state`. This is true for all language models and should be toggled off only if `out.hidden_states[-1]` has to be the hidden state before last layer norm, which is needed for some vision models (e.g. CLIP, SigLIP) c4>^[T5UU4Sj5nU$)Nc >URS[URSS55n[R [ UR 55=(d 0nUVs0sH4nSU3UR SU3[URSU3S55_M6 nnSU;a*UR S[URSS55US'SU;a*UR S[URSS55US 'UR5VVs0sH!upWU(dMURSS 5/_M# nnn[U5S :a [U5 [RU5n T "U/UQ70UD6n [RU 5 UGHn U S :XaT (dOo[U S 5(a'XSSX'XRU R 5 O7[U S5(a&XSSX'XRU R"5 [%X5X'MU S:Xaj['XK[(5(aA[XK5S:Xa0[%XS SS25X'[%XSSS25U SU -'M[%X5X'GM[%X5X'GM USLaU R+5n U $s snfs snnf![RU 5 f=f)N return_dictToutput_Fcross_attentionsoutput_attentionsoutput_cross_attentionsmask_decoder_attentionsoutput_mask_decoder_attentionsr_rrBvision_hidden_stateslast_hidden_state attentionsr r cross_)poprkconfigrar5rbrcrdreplacerErmrCr8r<hasattrrFryr{rHrGreto_tuple)r/rJkwargsrrcapturable_flagskrecordable_keysvrL output_tokenoutputsrMfuncrns r&wrapper4capture_outputs..wrapped_fn..wrappers!**]GDKKX\4]^K 477DNN8KLRPR *)A! vzzGA3-PWXYWZm]b9cdd)  "%55=CZZ'>QSX)Y> 9:),<QSX)YE @AKZJ_J_Ja gJa$!ef!=9b!92!=Ja  g$%)-d3,001BCL 6t5d5f5"'' 5)/)1 *@AA1B1G1L).).55g6R6RS *=>>1B1G1L).).55g6O6OP#():)?#@GLL(!"2"7>>3GWG\C]abCb',->-CADqD-I'J 278I8NqtRSt8T2U3/',->-C'D #():)?#@GL)),e#!**,No!h"'' 5s;J9 J>J>) KKr)rrrns` r& wrapped_fn#capture_outputs..wrapped_fns! t=  = ~r%r)rrnrs ` r&capture_outputsrs#AF $ r%)T) rI nn.ModulerMrbrrrrreturnNone)rZrr[rbr\z list[tuple[str, OutputRecorder]]rrr4)rgrrTrrr)rgrrr)r" __future__r threading contextvarsr dataclassesr functoolsrtypingr import_utilsr r rr rUrrarr(rCrQrWrXLockrlrmrrr%r&rs #"! <0  :.. .&!*!*J**<=SW8 88(+8KO8 8,nn+.n?_n n@>:$..*2&TTr%