3jC+SSKrSSKrSSKJrJrJrJr SSKJr SSK J r SSK J r J r S\S\4Sjr"S S 5r"S S \5rg) N)AnyDictListOptional) EinopsError)_product)ParsedExpression _ellipsisaxesreport_messagecV[U5S:a[URU55eg)Nr)lenrformat)r r s O/home/wildlama/miniconda3/lib/python3.13/site-packages/einops/layers/_einmix.py _report_axesr s( 4y1}.//566c ^\rSrSrSS\S\S\\S\4U4SjjjrS\S\S\\S\4Sjr S\\S \\ S \\S \\ 4S jr S r Sr SrU=r$) _EinmixMixinpattern weight_shape bias_shape axes_lengthsc r>[TU]5 XlX lX0lX@lUR XX4S9 g)a EinMix - Einstein summation with automated tensor management and axis packing/unpacking. EinMix is a combination of einops and MLP, see tutorial: https://github.com/arogozhnikov/einops/blob/main/docs/3-einmix-layer.ipynb Imagine taking einsum with two arguments, one of each input, and one - tensor with weights >>> einsum('time batch channel_in, channel_in channel_out -> time batch channel_out', input, weight) This layer manages weights for you, syntax highlights a special role of weight matrix >>> EinMix('time batch channel_in -> time batch channel_out', weight_shape='channel_in channel_out') But otherwise it is the same einsum under the hood. Plus einops-rearrange. Simple linear layer with a bias term (you have one like that in your framework) >>> EinMix('t b cin -> t b cout', weight_shape='cin cout', bias_shape='cout', cin=10, cout=20) There is no restriction to mix the last axis. Let's mix along height >>> EinMix('h w c-> hout w c', weight_shape='h hout', bias_shape='hout', h=32, hout=32) Example of channel-wise multiplication (like one used in normalizations) >>> EinMix('t b c -> t b c', weight_shape='c', c=128) Multi-head linear layer (each head is own linear layer): >>> EinMix('t b (head cin) -> t b (head cout)', weight_shape='head cin cout', ...) ... and yes, you need to specify all dimensions of weight shape/bias shape in parameters. Use cases: - when channel dimension is not last, use EinMix, not transposition - patch/segment embeddings - when need only within-group connections to reduce number of weights and computations - next-gen MLPs (follow tutorial link above to learn more!) - in general, any time you want to combine linear layer and einops.rearrange Uniform He initialization is applied to weight tensor. This accounts for the number of elements mixed and produced. Parameters :param pattern: transformation pattern, left side - dimensions of input, right side - dimensions of output :param weight_shape: axes of weight. A tensor of this shape is created, stored, and optimized in a layer If bias_shape is not specified, bias is not created. :param bias_shape: axes of bias added to output. Weights of this shape are created and stored. If `None` (the default), no bias is added. :param axes_lengths: dimensions of weight tensor )rrrrN)super__init__rrrrinitialize_einmix)selfrrrr __class__s rr_EinmixMixin.__init__s?T  ($( :  rc ^URS5upV[U5n[U5n[U5n [[R UR 1UR kU R k5S5 U R (a [S5eUR (dUR (aOUR (aUR (d[SU35eUR(a[SU35e[SXxU 455(a [S5eSU;dS U;a[S U35eSn Sn Sn [S UR55(a{/n URHnX- n M U Vs/sHo[:waUOS PM n nS RU 5nUSU3n UR5VVs0sHunnX;dM UU_M n nn[SUR55(dUR(aN/n URHnX- n M U Vs/sHo[:waUOS PM n nS RU 5nUSU3n URXU 05 U R HnUU;dM [SUS35e [[R [U51UR kU R k5S5 [[R U R 1UR kUR k5S5 [U R 5S:Xa[ R""SSS9 U RVs/sH unUUPM nn[%U RVs/sHunUUR ;dMUUPM sn5nUGb(['U[(5(d [S5e[U5n[[R UR UR 5S5 [[R UR [U55S5 /nSnURHnU[:XaU(a [S5eM!UHZnU[:XaU(a [S5eM!UUR ;aUR+UU5 SnMIUR+S5 M\ M OSnS U- S!-nSU- S!-nUR-UUUU5 1UR kUR kU R kn[U;aUR/[5 [1U5n[3[4R6U5VVs0sH unnUU_M snnmS T['S"[4U4S#jjnU"U5S$U"U 5SU"U53Ulgs snfs snnfs snfs snfs snfs snnf)%Nz->z7Unrecognized identifiers on the right side of EinMix {}zKEllipsis is not supported in weight, as its shape should be fully specifiedz,Ellipsis in EinMix should be on both sides, z3Ellipsis on left side can't be in parenthesis, got c38# UHoRv M g7fN)has_non_unitary_anonymous_axes).0xs r 1_EinmixMixin.initialize_einmix..SsO9NA//9Nsz2Anonymous axes (numbers) are not allowed in EinMix()z,Parenthesis is not allowed in weight shape: c3># UHn[U5S:gv M g7fNrr%groups rr'r([s=,<5s5zQ,<... z-> c3># UHn[U5S:gv M g7fr,r.r/s rr'r(ds>,=5s5zQ,=r1z ->z Dimension z of weight should be specifiedzAxes {} are not used in patternzWeight axes {} are redundantrzCEinMix: weight has no dimensions (means multiplication by a number)) stacklevelzAbias shape should be string specifying which axes bias depends onz"Bias axes {} not present in outputz#Sizes not provided for bias axes {}Fz:all bias dimensions should go after ellipsis in the outputTr-g?r c >/nURH\n[U[5(a'URUVs/sHnTUPM sn5 M?U[:XdeUR S5 M^ SR U5$s snf)Nr2) composition isinstancelistextendr appendjoin)r result composed_axisaxismapping2letterss rwrite_flat_remapped;_EinmixMixin.initialize_einmix..write_flat_remappedsuF!%!1!1 mT22MM]"S]T?4#8]"ST(I555MM%( "2 776? " #TsB ,)splitr rset difference identifiers has_ellipsisrhas_ellipsis_parenthesizedanyr:r r?items_create_rearrange_layersrwarningswarnrr;strr>_create_parametersremovesortedzipstringascii_lowercaseeinsum_pattern) rrrrr left_pattern right_patternleftrightweightpre_reshape_patternpre_reshape_lengthspost_reshape_patternnamesr0namer:lengthrB _weight_shape_fan_inbias _bias_shapeused_non_trivial_sizer weight_bound bias_boundmapped_identifiersletterkrDrCs @rr_EinmixMixin.initialize_einmixCs&-mmD&9#  - /!,/ NN5,,.V0@0@.V6CUCU.V W E    kl l    2 2%%%*<*<!$PQXPY"Z[[..!$WX_W`"abb O$v9NO O ORS S , #"5 L\N[\ \""# =D,<,<= = =!E))*FKLedY.TE9eEL((5/K%1N#k]"C DPDVDVDX"jDXLD&\`\i<4E,=,=> > >%BbBbE**+FKLedY.TE9eEL((5/K&1]#m_#E  %%&9Pdfhi&&D<'!Jtf4R"STT'  NN3|,.V0@0@.V6CUCU.V W -   NN6--/V1A1A/VEDUDU/V WYw  v!! "a ' MM_lm n;A;M;MN;Md+;M Nf>P>Pr>P7DTX`e`q`qTq.L.>Prs  !j#..!"eff#J/D t//1B1BC4  t//\1BC5  K$) !))9$,)*fgg-!%9,4&12n&o o 5!T%5%55'..|D/AB481'..q1!% * KG + 'kc)   |[*UZt//Y%2C2CYfFXFXY * *  % %i 0#$6769&:P:PRd6ef6e1f96ef%* " #&6 ##4()+>v+F*GrJ]^cJdIe f qM#k M&OrZgs0V; WW4WW W W.Wr_r`rapost_reshape_lengthsc[S5e)N.Should be defined in framework implementationsNotImplementedErrorrr_r`rarps rrO%_EinmixMixin._create_rearrange_layerss""RSSrc[S5e)zShape and implementationsrrrsrrrjrrks rrS_EinmixMixin._create_parameterss!"RSSrc&[UR5nUSURS3- nURbUSURS3- nURR 5Hup#USUSU3- nM UR RSUS3$)Nz, ''z, =r)r*)reprrrrrrNr__name__)rparamsrBrds r__repr___EinmixMixin.__repr__sdll#C))*!,, ?? & DOO,A. .F --335LD 4&&* *F6..))*!F8155r)rrrYrrr#)r~ __module__ __qualname____firstlineno__rRrrrdictrrrOrSr__static_attributes__ __classcell__)rs@rrrs1 1 31 HSM1 jm1 1 fv v Cv XVY]v jnv pT%c]T&d^T'sm T 'tn TT66rrc P\rSrSrSrS\\S\\S\\S\\4SjrSr S r g ) _EinmixDebuggerzUsed only to test mixinr_r`rarpc4XlX lX0lX@lgr#)r_r`rarprus rrO(_EinmixDebugger._create_rearrange_layerss$7 #6 $8!$8!rcXlX0lgr#)saved_weight_shapesaved_bias_shaperxs rrS"_EinmixDebugger._create_parameterss". *r)rprar`r_rrN) r~rrr__doc__rrRrrOrSrrrrrsE! 9%c] 9&d^ 9'sm 9 'tn 9+rr)rWrPtypingrrrreinopsr einops.einopsreinops.parsingr r rHrRrrrrrrrsF ,,"67s7C7 @6@6F+l+r