+ PjC+^RIt^RIt^RIHtHtHtHt^RIHt^RI H t ^RI H t H t RRlt!RR 4t!R R ]4tR#) N)AnyDictListOptional) EinopsError)_product)ParsedExpression _ellipsisc0V^8dQhR\R\/#)axesreport_message)setstr)formats"n/Users/ahmed/devFolder/Ultron/claude-voice/gateway/.venv/lib/python3.14/site-packages/einops/layers/_einmix.py __annotate__r s77s7C7cZ\V4^8d\VPV44hR#)rN)lenrr)r rs&&r _report_axesr s( 4y1}.//566rcraa]tRt^toR V3RlV3RllltV3RlRltV3RlRltRtRtR t Vt V;t #) _EinmixMixincB<V^8dQhRS[RS[RS[S[,RS[/#r pattern weight_shape bias_shape axes_lengths)rrr)r __classdict__s"rr_EinmixMixin.__annotate__s.1 1 1 31 HSM1 jm1 rc z<\SV`4WnW nW0nW@nVP WW4R7R#)a EinMix - Einstein summation with automated tensor management and axis packing/unpacking. EinMix is a combination of einops and MLP, see tutorial: https://github.com/arogozhnikov/einops/blob/main/docs/3-einmix-layer.ipynb Imagine taking einsum with two arguments, one of each input, and one - tensor with weights >>> einsum('time batch channel_in, channel_in channel_out -> time batch channel_out', input, weight) This layer manages weights for you, syntax highlights a special role of weight matrix >>> EinMix('time batch channel_in -> time batch channel_out', weight_shape='channel_in channel_out') But otherwise it is the same einsum under the hood. Plus einops-rearrange. Simple linear layer with a bias term (you have one like that in your framework) >>> EinMix('t b cin -> t b cout', weight_shape='cin cout', bias_shape='cout', cin=10, cout=20) There is no restriction to mix the last axis. Let's mix along height >>> EinMix('h w c-> hout w c', weight_shape='h hout', bias_shape='hout', h=32, hout=32) Example of channel-wise multiplication (like one used in normalizations) >>> EinMix('t b c -> t b c', weight_shape='c', c=128) Multi-head linear layer (each head is own linear layer): >>> EinMix('t b (head cin) -> t b (head cout)', weight_shape='head cin cout', ...) ... and yes, you need to specify all dimensions of weight shape/bias shape in parameters. Use cases: - when channel dimension is not last, use EinMix, not transposition - patch/segment embeddings - when need only within-group connections to reduce number of weights and computations - next-gen MLPs (follow tutorial link above to learn more!) - in general, any time you want to combine linear layer and einops.rearrange Uniform He initialization is applied to weight tensor. This accounts for the number of elements mixed and produced. Parameters :param pattern: transformation pattern, left side - dimensions of input, right side - dimensions of output :param weight_shape: axes of weight. A tensor of this shape is created, stored, and optimized in a layer If bias_shape is not specified, bias is not created. :param bias_shape: axes of bias added to output. Weights of this shape are created and stored. If `None` (the default), no bias is added. :param axes_lengths: dimensions of weight tensor )rrrrN)super__init__rrrrinitialize_einmix)selfrrrr __class__s&&&&,rr$_EinmixMixin.__init__s?T  ($( :  rcB<V^8dQhRS[RS[RS[S[,RS[/#r)rrdict)rr s"rrr!Cs5v v v Cv XVY]v jnv rc aVPR4wrV\V4p\V4p\V4p \\P VP 0VP mV P m4R4V P 'd \R4hVP 'gVP 'dSVP 'dVP 'g\RV 24hVP'd\RV 24h\;QJdRWxV 34F 'gK RM RM!RWxV 344'd \R4hR V9gR V9d\R V 24hR p R p R p \;QJd&R VP4F 'gK RM RM!R VP44'd.p VPF pW, p K V Uu.uFq\8wdTMRNK p pRPV 4pV RV 2p VP4UUu/uFwppW9gKVVbK p pp\;QJd&RVP4F 'gK RM RM!RVP44'gVP'dS.p VPF pW, p K V Uu.uFq\8wdTMRNK p pRPV 4pV RV 2p VPWV /4V P FpVV9gK \RV R24h \\P \V40VP mV P m4R4\\P V P 0VP mVP m4R4\V P 4^8Xd\ P"!R^R7V PUu.uFwpVV,NK pp\%V PUu.uF!wpVVP 9gKVV,NK# up4pVEe3\'V\(4'g \R4h\V4p\\P VP VP 4R4\\P VP \V44R4.pRpVPFpV\8XdV'd \R4hK#VFapV\8XdV'd \R4hK#VVP 9dVP+VV,4RpKPVP+^4Kc K MR p^V, R,p^V, R,pVP-VVVV40VP mVP mV P mp\V9dVP/\4\1V4p\3\4P6V4UUu/uF wppVVbK uppoRS\&RV3RllpV!V4 R V!V 4 RV!V4 2VnR #uupiuuppiuupiuupiuupiuuppi)!z->z7Unrecognized identifiers on the right side of EinMix {}zKEllipsis is not supported in weight, as its shape should be fully specifiedz,Ellipsis in EinMix should be on both sides, z3Ellipsis on left side can't be in parenthesis, got c38"TFqPxK R#5iN)has_non_unitary_anonymous_axes).0xs& r 1_EinmixMixin.initialize_einmix..SsO9NA//9NsTFz2Anonymous axes (numbers) are not allowed in EinMix()z,Parenthesis is not allowed in weight shape: Nc3>"TFp\V4^8gxK R#5iNrr/groups& rr1r2[s=,<5s5zQ,<... z-> c3>"TFp\V4^8gxK R#5ir6r8r9s& rr1r2ds>,=5s5zQ,=r;z ->z Dimension z of weight should be specifiedzAxes {} are not used in patternzWeight axes {} are redundantzCEinMix: weight has no dimensions (means multiplication by a number)) stacklevelzAbias shape should be string specifying which axes bias depends onz"Bias axes {} not present in outputz#Sizes not provided for bias axes {}z:all bias dimensions should go after ellipsis in the outputg?c$V^8dQhR\/#)r r )r )rs"rr4_EinmixMixin.initialize_einmix..__annotate__s # #&6 #rc<.pVPFap\V\4'd+TPVUu.uF pSV,NK up4KCV\8XgQhVP R4Kc RP V4#uupi)r<) composition isinstancelistextendr appendjoin)r result composed_axisaxismapping2letterss& rwrite_flat_remapped;_EinmixMixin.initialize_einmix..write_flat_remappedsxF!%!1!1 mT22MM]"S]T?4#8#8]"ST(I555MM%( "2 776? " #TsB ,)splitr rr difference identifiers has_ellipsisrhas_ellipsis_parenthesizedanyrDr rIitems_create_rearrange_layersrwarningswarnrrErrH_create_parametersremovesortedzipstringascii_lowercaseeinsum_pattern) r&rrrr left_pattern right_patternleftrightweightpre_reshape_patternpre_reshape_lengthspost_reshape_patternnamesr:namerDlengthrL _weight_shape_fan_inbias _bias_shapeused_non_trivial_sizer weight_bound bias_boundmapped_identifiersletterkrNrMs &&&&& @rr%_EinmixMixin.initialize_einmixCs&-mmD&9#  - /!,/ NN5,,.V0@0@.V6CUCU.V W E     kl l     2 2 2%%%%*<*<*<!$PQXPY"Z[[...!$WX_W`"abb 3O$v9NO333O$v9NO O ORS S , #"5 L\N[\ \""# 3=D,<,<=333=D,<,<= = =!E))*FKLedY.TE9eEL((5/K%1N#k]"C DPDVDVDX"jDXLD&\`\i<4E,=,=>333>E,=,=> > >%BbBbBbE**+FKLedY.TE9eEL((5/K&1]#m_#E  %%&9Pdfhi&&D<'!Jtf4R"STT'  NN3|,.V0@0@.V6CUCU.V W -   NN6--/V1A1A/VEDUDU/V WYw  v!! "a ' MM_lm n;A;M;MN;Md++;M Nf>P>Pr>P7DTX`e`q`qTq.L..>Prs  !j#..!"eff#J/D t//1B1BC4  t//\1BC5  K$) !))9$,)*fgg-!%9,4&12n&o o 5!T%5%55'..|D/AB481'..q1!% * KG + 'kc)   |[*UZt//Y%2C2CYfFXFXY * *  % %i 0#$6769&:P:PRd6ef6e1f96ef%* " # ##4()+>v+F*GrJ]^cJdIe f qM#k M&OrZgs0%Y.) Y39Y3Y94Y>Z7 Z&Zc r<V^8dQhRS[S[,RS[S[,RS[S[,RS[S[,/#r rgrhripost_reshape_lengthsrrr)rr s"rrr!sITT%c]T&d^T'sm T 'tn Trc\R4h).Should be defined in framework implementationsNotImplementedErrorr&rgrhrirzs&&&&&rrX%_EinmixMixin._create_rearrange_layerss""RSSrc \R4h)zShape and implementationsr}r~r&rrrrrss&&&&&rr[_EinmixMixin._create_parameterss!"RSSrc@\VP4pVRVP R2, pVPeVRVP R2, pVPP 4Fwr#VRV RV 2, pK VP P RV R2#)z, ''z, =r3r4)reprrrrrrWr'__name__)r&paramsrLrls& r__repr___EinmixMixin.__repr__sdll#C))*!,, ?? & DOO,A. .F --335LD 4&&* *F6..))*!F8155r)rrrarrr-) r __module__ __qualname____firstlineno__r$r%rXr[r__static_attributes____classdictcell__ __classcell__)r'r s@@rrrs91 1 fv v pTTT66rrc<a]tRt^toRtV3RlRltRtRtVtR#)_EinmixDebuggerzUsed only to test mixinc r<V^8dQhRS[S[,RS[S[,RS[S[,RS[S[,/#ryr{)rr s"rr_EinmixDebugger.__annotate__sC 9 9%c] 9&d^ 9'sm 9 'tn 9rc6WnW nW0nW@nR#r-)rgrhrirzrs&&&&&rrX(_EinmixDebugger._create_rearrange_layerss$7 #6 $8!$8!rcWnW0nR#r-)saved_weight_shapesaved_bias_shapers&&&&&rr["_EinmixDebugger._create_parameterss". *r)rzrirhrgrrN) rrrr__doc__rXr[rr)r s@rrrs! 9 9++rr)r_rYtypingrrrreinopsr einops.einopsreinops.parsingr r rrrrrrs8 ,,"67 @6@6F+l+r