Class MiniMaxVisionProcessor

java.lang.Object

smile.llm.model.minimax.MiniMaxVisionProcessor

public final class MiniMaxVisionProcessor
extends Object

Turns decoded images and video frames into the patch matrices of the MiniMax-M3 vision tower, following the reference MiniMaxM3VLImageProcessor / MiniMaxM3VLVideoProcessor: smart_resize to a multiple of patch * merge pixels under a pixel budget, bicubic antialiased resize, rescale and normalize, pad the frame count to a multiple of the temporal patch size by repeating the last frame, then cut into patches ordered by (time group, merge block row, merge block column, row in block, column in block), each patch flattened as (channel, time, row, column).

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final record	MiniMaxVisionProcessor.Patches	Patches of one image or video and its (t, h, w) grid.

Field Summary

Fields

Modifier and Type	Field	Description
static final int	IMAGE_MAX_PIXELS	Pixel budget of one image.
static final long	MAX_SOURCE_PIXELS	Largest accepted source image, to bound memory (width * height).
static final int	MIN_PIXELS	Lower pixel bound of images and video frames.
static final int	VIDEO_MAX_PIXELS	Pixel budget of one video frame.
static final long	VIDEO_TOTAL_PIXELS	Pixel budget of all frames of one video (about 64k vision tokens).

Constructor Summary

Constructors

Constructor	Description
MiniMaxVisionProcessor(MiniMaxVisionArgs args)	Constructor.

Method Summary

Modifier and Type	Method	Description
MiniMaxVisionProcessor.Patches	image(BufferedImage image)	Preprocesses one image.
static int[]	smartResize(int height, int width, int factor, int minPixels, int maxPixels)	The reference smart_resize: both sides become multiples of factor, the area stays within [minPixels, maxPixels] and the aspect ratio is kept approximately.
MiniMaxVisionProcessor.Patches	video(List<BufferedImage> frames)	Preprocesses one video given its sampled frames.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

IMAGE_MAX_PIXELS

public static final int IMAGE_MAX_PIXELS

Pixel budget of one image.

See Also:

• Constant Field Values

VIDEO_MAX_PIXELS

public static final int VIDEO_MAX_PIXELS

Pixel budget of one video frame.

See Also:

• Constant Field Values

MIN_PIXELS

public static final int MIN_PIXELS

Lower pixel bound of images and video frames.

See Also:

• Constant Field Values

VIDEO_TOTAL_PIXELS

public static final long VIDEO_TOTAL_PIXELS

Pixel budget of all frames of one video (about 64k vision tokens).

See Also:

• Constant Field Values

MAX_SOURCE_PIXELS

public static final long MAX_SOURCE_PIXELS

Largest accepted source image, to bound memory (width * height).

See Also:

• Constant Field Values

Constructor Details

MiniMaxVisionProcessor

public MiniMaxVisionProcessor(MiniMaxVisionArgs args)

Constructor.

Parameters:

args - the vision hyperparameters.

Method Details

smartResize

public static int[] smartResize(int height,
 int width,
 int factor,
 int minPixels,
 int maxPixels)

The reference smart_resize: both sides become multiples of factor, the area stays within [minPixels, maxPixels] and the aspect ratio is kept approximately.

Parameters:

height - source image height in pixels.

width - source image width in pixels.

factor - divisibility factor for output dimensions.

minPixels - minimum total pixel count.

maxPixels - maximum total pixel count.

Returns:

{height, width}.

Throws:

IllegalArgumentException - if the aspect ratio exceeds 200.

image

public MiniMaxVisionProcessor.Patches image(BufferedImage image)

Preprocesses one image.

Parameters:

image - the decoded image (any type; alpha is dropped).

Returns:

patches [gh * gw, patchDim] (float32, CPU) and the grid (1, gh, gw).

video

public MiniMaxVisionProcessor.Patches video(List<BufferedImage> frames)

Preprocesses one video given its sampled frames.

Parameters:

frames - the frames in time order (all the same size).

Returns:

patches and the grid (ceil(frames / temporalPatchSize), gh, gw).