Class GatedDeltaNet

java.lang.Object

smile.llm.model.qwen.GatedDeltaNet

public class GatedDeltaNet
extends Object

Qwen3.5 Gated DeltaNet linear-attention mixer (reference / torch path).

Constructor Summary

Constructors

Constructor	Description
GatedDeltaNet(QwenModelArgs args, int linearLayerId, DeltaNetStatePool statePool)	Constructor.
GatedDeltaNet(QwenModelArgs args, int linearLayerId, DeltaNetStatePool statePool, TensorShardSpec shard, TensorParallelGroup tpGroup)	Tensor-parallel constructor using local head counts from shard.

Method Summary

Modifier and Type	Method	Description
Tensor	forward(Tensor x)	Forward pass.
MemorySegment	module()	Returns the native module handle for weight registration.
void	replaceGemmProjections(LinearOp qkv, LinearOp z, LinearOp out)	Replaces GEMM projections typically present as FP8 in Qwen checkpoints (in_proj_qkv, in_proj_z, out_proj), leaving dense in_proj_a/in_proj_b for the residual load path.
void	replaceProjections(LinearOp qkv, LinearOp z, LinearOp b, LinearOp a, LinearOp out)	Replaces the five DeltaNet projections with quantized ops (already sharded).

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

GatedDeltaNet

public GatedDeltaNet(QwenModelArgs args,
 int linearLayerId,
 DeltaNetStatePool statePool)

Constructor.

Parameters:

args - model hyperparameters.

linearLayerId - ordinal among linear-attention layers.

statePool - shared DeltaNet state pool.

GatedDeltaNet

public GatedDeltaNet(QwenModelArgs args,
 int linearLayerId,
 DeltaNetStatePool statePool,
 TensorShardSpec shard,
 TensorParallelGroup tpGroup)

Tensor-parallel constructor using local head counts from shard.

Parameters:

args - model hyperparameters.

linearLayerId - ordinal among linear-attention layers.

statePool - shared DeltaNet state pool.

shard - local head / rank shard description, or null for full width.

tpGroup - tensor-parallel group, or null for single-device.

Method Details

replaceProjections

public void replaceProjections(LinearOp qkv,
 LinearOp z,
 LinearOp b,
 LinearOp a,
 LinearOp out)

Replaces the five DeltaNet projections with quantized ops (already sharded).

Parameters:

qkv - packed QKV projection

z - z projection

b - beta projection

a - a projection

out - output projection

replaceGemmProjections

public void replaceGemmProjections(LinearOp qkv,
 LinearOp z,
 LinearOp out)

Replaces GEMM projections typically present as FP8 in Qwen checkpoints (in_proj_qkv, in_proj_z, out_proj), leaving dense in_proj_a/in_proj_b for the residual load path.

Parameters:

qkv - combined QKV projection op.

z - z-gate projection op.

out - output projection op.

module

public MemorySegment module()

Returns the native module handle for weight registration.

Returns:

module handle.

forward

public Tensor forward(Tensor x)

Forward pass.

Parameters:

x - hidden states [B, S, D].

Returns:

mixer output [B, S, D].