Class MiniMaxBlock

java.lang.Object
smile.llm.model.minimax.MiniMaxBlock
public class MiniMaxBlock extends Object
One MiniMax-M3 decoder block: pre-norm, GQA/MSA attention, residual, post-norm, dense FFN or MoE, residual. Norms use the Gemma 1 + weight convention.

  • Constructor Details

    • MiniMaxBlock

      public MiniMaxBlock(int layerId, MiniMaxModelArgs args)
      Constructor.
      Parameters:
      layerId - decoder layer index.
      args - model hyperparameters.

    • MiniMaxBlock

      public MiniMaxBlock(int layerId, MiniMaxModelArgs args, MiniMaxShard shard, TensorParallelGroup group)
      Expert-parallel constructor.
      Parameters:
      layerId - decoder layer index.
      args - model hyperparameters.
      shard - this rank's slice of the routed experts.
      group - all-reduce group (needed when shard.size() > 1).

  • Method Details

    • layerId

      public int layerId()
      Returns the decoder layer index.
      Returns:
      layer index.

    • selfAttn

      public MiniMaxAttention selfAttn()
      Returns the attention mixer.
      Returns:
      attention.

    • mlp

      public MiniMaxMlp mlp()
      Returns the dense FFN.
      Returns:
      the FFN, or null for MoE layers.

    • moe

      public MiniMaxMoE moe()
      Returns the mixture of experts.
      Returns:
      the MoE, or null for dense layers.

    • forward

      public Tensor forward(Tensor x, int[] positions, Tensor cos, Tensor sin, Tensor mask)
      Forward pass.
      Parameters:
      x - hidden states [B, S, D].
      positions - KV write position per row.
      cos - RoPE cosines.
      sin - RoPE sines.
      mask - causal mask for multi-token windows, or null.
      Returns:
      block output.