+ PjLRt^RIt^RIt^RIHt^RIHt^RIHtRt RRlt !RR]P4t !R R ]P4t !R R ]P4t!R R]P4t!RR]P4t!RR]P4tR#)zk Taken from https://github.com/LTH14/mar/blob/fe470ac24afbee924668d8c5c83e9fec60af3a73/models/diffloss.py N)Self) FlowLMConfigc0V^V,,V,#)xshiftscales&&&o/Users/ahmed/devFolder/Ultron/claude-voice/gateway/.venv/lib/python3.14/site-packages/pocket_tts/modules/mlp.pymodulater s E ?U ""cdV^8dQhR\PR\PR\/#)ralphaeps)torchTensorfloat)formats"r __annotate__rs)   ell  r cVP4VP48gQhVPpW PRRR7,pWPV4\P !V4,,PV4pV#)rTdimkeepdim)rdtypevartorrsqrt)rrrx_dtyperys&&& r _rms_normr"sh 557eiik !! !ggG "d+ +C hhsmekk#.. /33G) __name__ __module__ __qualname____firstlineno__r,r8__static_attributes____classdictcell__ __classcell__r2r's@@r r$r$s TT 222r r$c@aa]tRt^'toRtRV3RlltRtRtVtV;t #) LayerNormzJReimplementation of LayerNorm because the default one doesn't support jvp.c<\SV`4W nV'da\P!\ P !V44Vn\P!\ P!V44Vn R#R#r7) r+r,rr-r.ronesweightzerosbias)r0channelsrelementwise_affiner2s&&&&r r,LayerNorm.__init__*sM  ,,uzz(';T r:r;r<r=__doc__r,r8r>r?r@rAs@@r rCrC'sT<r rCcLaa]tRt^:toRtRV3RlV3RllltRtRtVtV;t #)TimestepEmbedderz4Embeds scalar timesteps into vector representations.c,<V^8dQhRS[RS[RS[/#)r hidden_sizefrequency_embedding_size max_period)r&)rr's"r rTimestepEmbedder.__annotate__=s%   := QT r c <\SV`4\P!W!RR7\P!4\P!WRR7.pVP \ V44\P!V!VnW n V^,pVPR\P!\P!V4)\P!^VR7,V, 44R#)TrHfreqs)startendN)r+r,r-LinearSiLUappendr$ SequentialmlprYregister_bufferrexpmathlogarange)r0rXrYrZblockshalfr2s&&&& r r,TimestepEmbedder.__init__=s  II.$ G GGI IIkT :   gk*+==&)(@%'1,  UYY 44u||!QU7VVY]]^ r c8WPPVP4,p\P!\P !V4\P !V4.RR7pVP^,'dQhVPV4pV#)rrr) r^rrrcatcossinrYre)r0targs embeddingt_embs&& r r8TimestepEmbedder.forwardNsj::==))IIuyy $@bI 11A5566# r )rYre)i'rSrAs@@r rVrV:s>  "r rVc<aa]tRt^VtoRtV3RltRtRtVtV;t #)ResBlockzt A residual block that can optionally change the number of channels. :param channels: the number of input channels. c <\SV`4Wn\VRR7Vn\ P !\ P!WRR7\ P!4\ P!WRR74Vn \ P !\ P!4\ P!V^V,RR74Vn R#)rR)rTr]N) r+r,rIrCin_lnr-rdrarbreadaLN_modulation)r0rIr2s&&r r,ResBlock.__init__\s  xT2 == IIht 4 GGI IIht 4  !# GGIryy1x)>q)A)G)Gr)G)R& h TZZ]I 9 HHQKa<r )r}rIr|rerSrAs@@r rzrzVs    r rzc<aa]tRt^rtoRtV3RltRtRtVtV;t #) FinalLayerz# The final layer adopted from DiT. c <\SV`4\VRRR7Vn\P !WRR7Vn\P!\P!4\P !V^V,RR74Vn R#)FrR)rJrTr]N) r+r,rC norm_finalr-ralinearrdrbr})r0model_channels out_channelsr2s&&&r r,FinalLayer.__init__ws_ #NuRVWii4H " GGIryy^1C$O! r cVPV4P^RR7wr4\VPV4W44pVP V4pV#)rror)r}rr rr)r0rcr r s&&& r r8FinalLayer.forwardsK,,Q/55aR5@  T__Q' 6 KKNr )r}rrrSrAs@@r rrrs r rchaa]tRt^toRtRV3Rllt]V3RlRl4tV3RlRltRt Vt V;t #) SimpleMLPAdaLNa[Taken from https://arxiv.org/abs/2406.11838. The MLP for Diffusion Loss. :param in_channels: channels in the input Tensor. :param model_channels: base channel count for the model. :param out_channels: channels in the output Tensor. :param cond_channels: channels in the condition. :param num_res_blocks: number of residual blocks per downsample. c<\S V`4WnW nW0nWPnW`nV^8wgQh\P!\V4Uu.uFp\V4NK up4Vn \P!WB4Vn \P!W4Vn.p\V4Fp VP\!V44K \P!V4Vn\%W#4VnR#uupi)rN)r+r, in_channelsrrnum_res_blocksnum_time_condsr- ModuleListrangerV time_embedra cond_embed input_projrcrz res_blocksr final_layer) r0rrr cond_channelsrr_rir2s &&&&&&& r r,SimpleMLPAdaLN.__init__s &,(,,"""--7<^7L M7L! n -7L M ))MB))K@ ~&A   h~6 7'-- 3%nC NsDc2<V^8dQhRS[RS[RS[RS[/#)rcfg latent_dimcond_dimreturn)rr&r)rr's"r rSimpleMLPAdaLN.__annotate__s+  |  PS X\ r c jVPpVPpVPp^p\W%W#WgR7#)r)r)flowrdepthr)clsrrrconfigflow_dim flow_depthrs&&&& r from_pydantic_config#SimpleMLPAdaLN.from_pydantic_configs6::\\  *   r c <V^8dQhRS[PRS[PRS[PRS[PRS[P/#)rrsrsrrr5)rr's"r rrsI&&&"',,&38<<&DILL& &r c aaW#.oSPV4p\S4SP8Xg!QRSP R\S4 24hSP^8wgQh\VV3Rl\ SP444SP, pSP V4pWQ,pSP F pV!WF4pK SPWF4#)z Apply the model to an input batch. :param c: conditioning from AR transformer. :param s: start time tensor. :param t: target time tensor. :param x: an [N x C] Tensor of inputs. :return: an [N x C] Tensor of outputs. z Expected z time conditions, got c3d<"TF%pSPV,!SV,4xK' R#5ir7)r).0rr0tss& r )SimpleMLPAdaLN.forward..s(N3Ma"2a5))3Ms-0)rlenrsumrrrr) r0rrrsr t_combinedr!blockrs f&&&& @r r8SimpleMLPAdaLN.forwardsV OOA 2w$--- ++,,B3r7) L -""a''' N59L9L3MN NQUQdQd d  OOA  N__Ea A%%%r ) rrrrrrrrrrr) r:r;r<r=rTr, classmethodrr8r>r?r@rAs@@r rrs4D@  &&&r r)rTrhrtorch.nnr-typing_extensionsrpocket_tts.utils.configrr r"Moduler$rCrVrzrrrr r rs  "0# 2bii2 &ryy8 ryy 8(Q&RYYQ&r