Class LlamaBlock

java.lang.Object

smile.llm.model.llama.LlamaBlock

public class LlamaBlock
extends Object

A Llama dense-decoder block: grouped-query attention followed by a SwiGLU feed-forward network, each with a residual connection and RMSNorm.

Constructor Summary

Constructors

Constructor	Description
LlamaBlock(int layerId, int dim, int numHeads, int numKvHeads, Integer intermediateSize, int multipleOf, Double ffnDimMultiplier, double normEps)	Constructor.
LlamaBlock(int layerId, int dim, int numHeads, int numKvHeads, Integer intermediateSize, int multipleOf, Double ffnDimMultiplier, double normEps, KvCacheLayout layout)	Convenience constructor that allocates a private test-sized KV pool.

Method Summary

Modifier and Type	Method	Description
Attention	attention()	Returns the attention submodule.
FeedForward	feedForward()	Returns the feed-forward submodule.
Tensor	forward(Tensor x, int[] positions, Tensor cis, Tensor mask)	Decode / prefill forward with per-row cache write positions.
Tensor	forward(Tensor x, int startPos, Tensor cis, Tensor mask)	Forward pass through the block.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

LlamaBlock

public LlamaBlock(int layerId,
 int dim,
 int numHeads,
 int numKvHeads,
 Integer intermediateSize,
 int multipleOf,
 Double ffnDimMultiplier,
 double normEps)

Constructor. The attention module receives its KvCachePool later via LlamaModel.setKvCachePool(KvCachePool).

Parameters:

layerId - the identifier of the block.

dim - token embedding dimension.

numHeads - number of query heads.

numKvHeads - number of key/value heads.

intermediateSize - explicit FFN hidden size, or null to derive it.

multipleOf - FFN rounding multiple when intermediateSize is null.

ffnDimMultiplier - optional FFN dim multiplier when deriving size.

normEps - RMSNorm epsilon.

LlamaBlock

public LlamaBlock(int layerId,
 int dim,
 int numHeads,
 int numKvHeads,
 Integer intermediateSize,
 int multipleOf,
 Double ffnDimMultiplier,
 double normEps,
 KvCacheLayout layout)

Convenience constructor that allocates a private test-sized KV pool.

Parameters:

layerId - the identifier of the block.

dim - token embedding dimension.

numHeads - number of query heads.

numKvHeads - number of key/value heads.

intermediateSize - explicit FFN hidden size, or null.

multipleOf - FFN rounding multiple when deriving size.

ffnDimMultiplier - optional FFN dim multiplier.

normEps - RMSNorm epsilon.

layout - cache layout for the private test pool.

Method Details

attention

public Attention attention()

Returns the attention submodule.

Returns:

attention submodule.

feedForward

public FeedForward feedForward()

Returns the feed-forward submodule.

Returns:

feed-forward submodule.

forward

public Tensor forward(Tensor x,
 int startPos,
 Tensor cis,
 Tensor mask)

Forward pass through the block.

Parameters:

x - the input tensor.

startPos - the starting position for attention caching.

cis - the precomputed frequency tensor.

mask - the attention mask tensor.

Returns:

the output tensor.

forward

public Tensor forward(Tensor x,
 int[] positions,
 Tensor cis,
 Tensor mask)

Decode / prefill forward with per-row cache write positions.

Parameters:

x - input hidden states.

positions - absolute write position per batch row.

cis - RoPE frequency tensor (uniform [S,…] or gathered [B,…]).

mask - attention mask, or null.

Returns:

block output.