Class InterleavedMRope

java.lang.Object

smile.llm.model.qwen.InterleavedMRope

public final class InterleavedMRope
extends Object

Interleaved multimodal RoPE for Qwen3.8 / Qwen3.5 VL text towers.

When T/H/W position planes are equal, the result matches classic 1D partial RoPE (PartialRotaryEncoding.computeCosSin(int, int, double)).

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final record	InterleavedMRope.MropePositions	mRoPE position planes for one sequence.

Method Summary

Modifier and Type	Method	Description
static PartialRotaryEncoding.CosSin	computeCosSin(int rotaryDim, double theta, int[] mropeSection, int[] posT, int[] posH, int[] posW)	Builds cos/sin for a sequence from three position planes.
static InterleavedMRope.MropePositions	getRopeIndex(int[] mmTokenTypeIds, int[][] imageGridThw, int[][] videoGridThw, int spatialMerge)	Builds per-token mRoPE position planes for a multimodal prompt.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

computeCosSin

public static PartialRotaryEncoding.CosSin computeCosSin(int rotaryDim,
 double theta,
 int[] mropeSection,
 int[] posT,
 int[] posH,
 int[] posW)

Builds cos/sin for a sequence from three position planes.

Parameters:

rotaryDim - even rotary feature count (e.g. 64).

theta - RoPE base.

mropeSection - half-freq section widths summing to rotaryDim/2.

posT - temporal positions length S.

posH - height positions length S.

posW - width positions length S.

Returns:

cos/sin of shape [S, rotaryDim] float32 (caller owns).

getRopeIndex

public static InterleavedMRope.MropePositions getRopeIndex(int[] mmTokenTypeIds,
 int[][] imageGridThw,
 int[][] videoGridThw,
 int spatialMerge)

Builds per-token mRoPE position planes for a multimodal prompt.

Parameters:

mmTokenTypeIds - 0=text, 1=image, 2=video; length S.

imageGridThw - [nImg][3] patch grids (T,H,W), or empty.

videoGridThw - [nVid][3] patch grids, or empty.

spatialMerge - spatial merge size.

Returns:

planes and rope_delta = max(pos)+1 - S.