Class QwenRMSNormGated

java.lang.Object

smile.llm.model.qwen.QwenRMSNormGated

All Implemented Interfaces:

Function<Tensor,Tensor>, Layer

public class QwenRMSNormGated
extends Object
implements Layer

Gated RMSNorm used inside Gated DeltaNet: weight * rms_norm(x) * silu(gate) with weight initialized to ones.

Constructor Summary

Constructors

Constructor	Description
QwenRMSNormGated(int dim, double eps)	Constructor.

Method Summary

Modifier and Type	Method	Description
Tensor	forward(Tensor input)	Forward propagation (or forward pass) through the layer.
Tensor	forward(Tensor input, Tensor gate)	Forward with gating.
MemorySegment	module()	Returns the native ST_Module handle for this layer.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface Function

andThen, compose

Methods inherited from interface Layer

apply, name, to, to

Constructor Details

QwenRMSNormGated

public QwenRMSNormGated(int dim,
 double eps)

Constructor.

Parameters:

dim - feature dimension.

eps - numerical stability epsilon.

Method Details

forward

public Tensor forward(Tensor input,
 Tensor gate)

Forward with gating.

Parameters:

input - hidden states.

gate - gate tensor (same shape as input).

Returns:

gated normalized tensor.

forward

public Tensor forward(Tensor input)

Description copied from interface: Layer

Forward propagation (or forward pass) through the layer.

Specified by:

forward in interface Layer

Parameters:

input - the input tensor.

Returns:

the output tensor.

module

public MemorySegment module()

Description copied from interface: Layer

Returns the native ST_Module handle for this layer.

Specified by:

module in interface Layer

Returns:

the native ST_Module handle.