Class MarlinLinear

java.lang.Object

smile.llm.quant.MarlinLinear

All Implemented Interfaces:

AutoCloseable, LinearOp

public final class MarlinLinear
extends Object
implements LinearOp, AutoCloseable

Marlin FP16×INT4 linear layer (Ampere/Ada failover for GPTQ/AWQ).

Constructor Summary

Constructors

Constructor	Description
MarlinLinear(Tensor qweight, Tensor scales, Tensor bias, int inFeatures, int outFeatures, int groupSize)	Creates a Marlin FP16×INT4 linear layer.

Method Summary

Modifier and Type	Method	Description
void	close()
Tensor	forward(Tensor input)	Applies the linear transformation.
int	groupSize()	Returns the quantization group size.
int	inFeatures()	Returns the input feature count.
int	outFeatures()	Returns the output feature count.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

MarlinLinear

public MarlinLinear(Tensor qweight,
 Tensor scales,
 Tensor bias,
 int inFeatures,
 int outFeatures,
 int groupSize)

Creates a Marlin FP16×INT4 linear layer.

Parameters:

qweight - Marlin-packed weights.

scales - group scales.

bias - optional bias, or null.

inFeatures - input features (K).

outFeatures - output features (N).

groupSize - quantization group size (typically 128).

Method Details

inFeatures

public int inFeatures()

Returns the input feature count.

Returns:

input features (K).

outFeatures

public int outFeatures()

Returns the output feature count.

Returns:

output features (N).

groupSize

public int groupSize()

Returns the quantization group size.

Returns:

group size.

forward

public Tensor forward(Tensor input)

Description copied from interface: LinearOp

Applies the linear transformation.

Specified by:

forward in interface LinearOp

Parameters:

input - activations (caller retains ownership).

Returns:

owned output tensor (caller must close).

close

public void close()

Specified by:

close in interface AutoCloseable