Record Class MiniMaxMultimodal

java.lang.Object
java.lang.Record
smile.llm.model.minimax.MiniMaxMultimodal
Record Components:
inputIds - expanded prompt token ids.
imagePixels - patches of all images back to back, [N, patchDim] float32 on the CPU, or null.
imageGrids - one (t, h, w) patch grid per image, in prompt order.
videoPixels - patches of all videos back to back, or null.
videoGrids - one (t, h, w) patch grid per video, in prompt order.
All Implemented Interfaces:
MultimodalPrompt
public record MiniMaxMultimodal(int[] inputIds, Tensor imagePixels, int[][] imageGrids, Tensor videoPixels, int[][] videoGrids) extends Record implements MultimodalPrompt
A MiniMax-M3 prompt with its images and videos preprocessed: the token ids (every media placeholder expanded to one id per vision token) plus the patch matrices and grids the vision tower encodes during prefill.

  • Constructor Details

    • MiniMaxMultimodal

      public MiniMaxMultimodal(int[] inputIds, Tensor imagePixels, int[][] imageGrids, Tensor videoPixels, int[][] videoGrids)
      Creates an instance of a MiniMaxMultimodal record class.
      Parameters:
      inputIds - the value for the inputIds record component
      imagePixels - the value for the imagePixels record component
      imageGrids - the value for the imageGrids record component
      videoPixels - the value for the videoPixels record component
      videoGrids - the value for the videoGrids record component

  • Method Details

    • hasVision

      public boolean hasVision()
      Description copied from interface: MultimodalPrompt
      Returns whether the prompt carries vision tensors.
      Specified by:
      hasVision in interface MultimodalPrompt
      Returns:
      true when media must be encoded during prefill.

    • close

      public void close()
      Releases the CPU patch matrices.

    • toString

      public final String toString()
      Returns a string representation of this record class. The representation contains the name of the class, followed by the name and value of each of the record components.
      Specified by:
      toString in class Record
      Returns:
      a string representation of this object

    • hashCode

      public final int hashCode()
      Returns a hash code value for this object. The value is derived from the hash code of each of the record components.
      Specified by:
      hashCode in class Record
      Returns:
      a hash code value for this object

    • equals

      public final boolean equals(Object o)
      Indicates whether some other object is "equal to" this one. The objects are equal if the other object is of the same class and if all the record components are equal. All components in this record class are compared with Objects::equals(Object,Object).
      Specified by:
      equals in class Record
      Parameters:
      o - the object with which to compare
      Returns:
      true if this object is the same as the o argument; false otherwise.

    • inputIds

      public int[] inputIds()
      Returns the value of the inputIds record component.
      Specified by:
      inputIds in interface MultimodalPrompt
      Returns:
      the value of the inputIds record component

    • imagePixels

      public Tensor imagePixels()
      Returns the value of the imagePixels record component.
      Returns:
      the value of the imagePixels record component

    • imageGrids

      public int[][] imageGrids()
      Returns the value of the imageGrids record component.
      Returns:
      the value of the imageGrids record component

    • videoPixels

      public Tensor videoPixels()
      Returns the value of the videoPixels record component.
      Returns:
      the value of the videoPixels record component

    • videoGrids

      public int[][] videoGrids()
      Returns the value of the videoGrids record component.
      Returns:
      the value of the videoGrids record component