3jHSrSSKrSSKJr SSKJs Jr SS\RS\S\ S\S\ S\ S \ 4S jjr SS\RS\S\ S\S\ S\ 4 S jjr "S S \R5rSS\RS\S\ S\ 4Sjjr"SS\R5rg)a<DropBlock, DropPath PyTorch implementations of DropBlock and DropPath (Stochastic Depth) regularization layers. Papers: DropBlock: A regularization method for convolutional networks (https://arxiv.org/abs/1810.12890) Deep Networks with Stochastic Depth (https://arxiv.org/abs/1603.09382) Code: DropBlock impl inspired by two Tensorflow impl that I liked: - https://github.com/tensorflow/tpu/blob/master/models/official/resnet/resnet_model.py#L74 - https://github.com/clovaai/assembled-cnn/blob/master/nets/blocks.py Hacked together by / Copyright 2020 Ross Wightman Nx drop_prob block_size gamma_scale with_noiseinplace batchwisec|URupxpX-n [U[X55n UU-U -U S-- X- S-X- S--- n [R"[R"U 5R UR 5[R"U 5R UR 55upXS-:XU S- S-- :-XS-:XU S- S-- :--n[R"USSX45R URS9nU(a.[R"SXU 4URUR S9nO[R"U5nSU - U- U-S:R URS9n[R"U*U SU S-S9*nU(aU(a-[R"SXU 4URUR S9O[R"U5nU(a(URU5R!USU- -5 U$UU-USU- --nU$UR#5UR [R$S9R'5R)S5- R UR5nU(aURUU-5 U$UU-U-nU$)zDropBlock. See https://arxiv.org/pdf/1810.12890.pdf DropBlock with an experimental gaussian noise option. This layer has been tested on a few training runs with success, but needs further validation and possibly optimization for lower runtime impact. dtype)rdevice kernel_sizestridepaddinggHz>)shapemintorchmeshgridarangetorreshaperrand rand_likeF max_pool2drandn randn_likemul_add_numelfloat32sumadd)rrrrrrr _CHW total_sizeclipped_block_sizegammaw_ih_i valid_block uniform_noise block_mask normal_noisenormalize_scales S/home/wildlama/miniconda3/lib/python3.13/site-packages/spandrel/util/timm/__drop.py drop_block_2dr6sJA!JZQ3      a   NQ 1>A#5 6  8 ~~ Q188$ell1o&8&8&BHC a' 'C7IA7MRS6S2S,ST q( (S8JQ8NST7T3T-T UWK-- aA\:==AGG=LK AqQ)rrr empty_like bernoulli_rrrrnormal_r!r"r#r$r%r&)rrrrrrr'r)r*r+r,r-r2r3r4s r5drop_block_fast_2dr=]sJAqQJZQ3     a   NQ 1>A#5 6  8 !!!$//6J agg&"a' J''*224  FF3# $ ) ),*C D HS:%&)BBA H^    U]]!C!G!G!I!M!Md!S S "177"    FF:/ 0 H0A Hr7c^\rSrSrSrSS\S\S\S\S\S\S \4U4S jjjrS \ R4S jr S r U=r $) DropBlock2dz3DropBlock. See https://arxiv.org/pdf/1810.12890.pdfrrrrrr fastcv>[TU]5 XlX0lX lX@lXPlX`lXplgN) super__init__rrrrrr rA) selfrrrrrr rA __class__s r5rEDropBlock2d.__init__s4 "&$$ " r7rc UR(aUR(dU$UR(aB[UURURUR UR UR5$[UURURUR UR URUR5$rC) trainingrrAr=rrrrr6r rFrs r5forwardDropBlock2d.forwards}}DNNH 99%    !    r7)r rrrArrr)皙?r9FFFT)__name__ __module__ __qualname____firstlineno____doc__floatintboolrErTensorrL__static_attributes__ __classcell__rGs@r5r?r?s=       &r7r?rJ scale_by_keepcUS:XdU(dU$SU- nURS4SURS- --nURU5RU5nUS:aU(aUR U5 X-$)aDrop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). This is the same as the DropConnect impl I created for EfficientNet, etc networks, however, the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper... See discussion: https://github.com/tensorflow/tpu/issues/494#issuecomment-532968956 ... I've opted for changing the layer and argument names to 'drop path' rather than mix DropConnect as a layer name and use 'survival rate' as the argument. r r)r )rndim new_emptyr;div_)rrrJr\ keep_probr random_tensors r5 drop_pathrds}CxI I WWQZMD   EKK&11)[TU]5 XlX lgrC)rDrErr\)rFrr\rGs r5rEDropPath.__init__s "*r7rcX[XRURUR5$rC)rdrrJr\rKs r5rLDropPath.forwardsNNDMM4;M;MNNr7c6S[URS5S3$)Nz drop_prob=z0.3f)roundr)rFs r5 extra_reprDropPath.extra_reprsE$..3D9::r7)rr\)r^T)rPrQrRrSrTrUrWrErrXrLrorYrZr[s@r5rfrfs:c+%+d++ OO;;r7rf)rNrOr9FFF)rNrOr9FF)r^FT)rTrtorch.nnnntorch.nn.functional functionalrrXrUrVrWr6r=Moduler?rdrfr7r5rws,$  B  ||B B B  B  B  B B N /  ||/ / /  /  /  / d+"))+`  || 6 ;ryy ;r7