GJjddlZddlmZmZddlmZddlmZddl m Z m Z GddeZ GddeZ Gd d eZy) N)AnyOptional)Module)QQLinearQuantizedLinearcheZdZdZdededdffd ZdejdejfdZxZ S) IdentityzA placeholder identity operator that is argument-insensitive. Args: args: any argument (unused) kwargs: any keyword argument (unused) argskwargsreturnNc"t|yN)super__init__)selfr r __class__s ^/Users/ahmed/devFolder/claude-voice/.venv/lib/python3.12/site-packages/mlx/nn/layers/linear.pyrzIdentity.__init__s xc|Srrrs r__call__zIdentity.__call__sr) __name__ __module__ __qualname____doc__rrmxarrayr __classcell__rs@rr r s=cST"((rxxrr c eZdZdZddedededdffd ZdefdZd e jde jfd Z dd e ed e ed edefdZ xZS)LinearaApplies an affine transformation to the input. Concretely: .. math:: y = x W^\top + b where: where :math:`W` has shape ``[output_dims, input_dims]`` and :math:`b` has shape ``[output_dims]``. The values are initialized from the uniform distribution :math:`\mathcal{U}(-{k}, {k})`, where :math:`k = \frac{1}{\sqrt{D_i}}` and :math:`D_i` is equal to ``input_dims``. Args: input_dims (int): The dimensionality of the input features output_dims (int): The dimensionality of the output features bias (bool, optional): If set to ``False`` then the layer will not use a bias. Default is ``True``. input_dims output_dimsbiasr Nct|tjd|z }tj j | |||f|_|r*tj j | ||f|_yyNg?)lowhighshape rrmathsqrtrrandomuniformweightr&)rr$r%r&scalers rrzLinear.__init__0s~  # *+ii'' +(   ))F"n*DI rcxd|jjdd|jjddd|vS)Nz input_dims=, output_dims=r, bias=r&r1r+)rs r _extra_reprzLinear._extra_repr?sET[[..q12.ARARSTAU@VV]^dhl^l]mnnrrcd|vr)tj|d||dj}|S||djz}|S)Nr&r1)raddmmTrs rrzLinear.__call__BsK T>fq$x.*:*:;ADN$$$Ar group_sizebitsmodequantize_inputc|r+|dvrtd|dtj||||Stj||||S)aReturn a quantized approximation of this layer. If ``quantize_input`` is ``False``, returns a :obj:`QuantizedLinear` (weights are quantized). If ``quantize_input`` is ``True``, returns a :obj:`QQLinear` (weights and activations are quantized). Args: group_size (Optional[int]): The quantization group size (see :func:`mlx.core.quantize`). Default: ``None``. bits (Optional[int]): The number of bits per parameter (see :func:`mlx.core.quantize`). Default: ``None``. mode (str): The quantization method to use (see :func:`mlx.core.quantize`). Default: ``"affine"``. quantize_input (bool): Whether to quantize input. Default: ``False``. Returns: QuantizedLinear or QQLinear: A quantized version of this layer. Notes: Quantized input is only supported for ``"nvfp4"`` and ``"mxfp8"`` modes. )nvfp4mxfp8zLQuantized activations are only supported for 'nvfp4' and 'mxfp8' modes, got .) ValueErrorr from_linearr)rr<r=r>r?s r to_quantizedzLinear.to_quantizedIs\: -- bcgbhhij''j$E E**4T4HHrT)NNaffineF)rrrrintboolrstrr8rrrrrFr r!s@rr#r#s* 3 S  PT oSo"((rxx%)"$ #ISM#Ism#I #I  #Irr#c eZdZdZ d dededededdf fd Zdefd Zd e jd e jde jfd Z xZ S)Bilineara!Applies a bilinear transformation to the inputs. Concretely: .. math:: y_i = x_1^\top W_i x_2 + b_i where: :math:`W` has shape ``[output_dims, input1_dims, input2_dims]``, :math:`b` has shape ``[output_dims ]``, and :math:`i` indexes the output dimension. The values are initialized from the uniform distribution :math:`\mathcal{U}(-{k}, {k})`, where :math:`k = \frac{1}{\sqrt{D_1}}` and :math:`D_1` is ``input1_dims``. Args: input1_dims (int): The dimensionality of the input1 features input2_dims (int): The dimensionality of the input2 features output_dims (int): The dimensionality of the output features bias (bool, optional): If set to ``False`` then the layer will not use a bias. Default is ``True``. input1_dims input2_dimsr%r&r Nct|tjd|z }tj j | ||||f|_|r*tj j | ||f|_yyr(r,)rrNrOr%r&r2rs rrzBilinear.__init__s  # +,ii'' [9(   ))F"n*DI rc V|jj\}}}d|d|d|dd|vS)Nz input1_dims=z, input2_dims=r5r6r&r7)routin2in1s rr8zBilinear._extra_reprsD )) S#3%~cU.FdN# % rx1x2c|jj\}}}|jdd}|jd|}|jdd|}|jj||z|}||jz}|jd||j dd}||z}|j d}|jg||}d|vr||j z}|S)Nr4r&)r1r+reshaper;swapaxessqueezer&) rrUrVrRrSrTxshapewys rrzBilinear.__call__s )) S##2 ZZC  ZZAs # KK  c 3 / H IIb#s # , ,R 4 F IIaL AII #v #s # T>DII ArrG) rrrrrIrJrrKr8rrrr r!s@rrMrMosn0RV-0?BJN " S 288bhhrrM)r-typingrrmlx.corecorermlx.nn.layers.basermlx.nn.layers.quantizedrrr r#rMrrrresA %= v RIVRIjEvEr