Class Fp8Linear

java.lang.Object

smile.llm.quant.Fp8Linear

All Implemented Interfaces:

AutoCloseable, LinearOp

public final class Fp8Linear
extends Object
implements LinearOp, AutoCloseable

Weight-only FP8 linear layer using cuBLASLt via LibTorch _scaled_mm.

Holds Float8_e4m3fn weights in [N,K] layout expected by at::_scaled_mm, plus a float weight scale. Activations are cast to FP8 with a dynamic (or fixed) scale; output is BF16 or FP16.

Constructor Summary

Constructors

Constructor	Description
Fp8Linear(Tensor weight, Tensor weightScale, Tensor bias, ScalarType outDtype)	Creates a weight-only FP8 linear layer.

Method Summary

Modifier and Type	Method	Description
void	close()
Tensor	forward(Tensor input)	Applies the linear transformation.
int	inFeatures()	Returns the input feature count.
int	outFeatures()	Returns the output feature count.
Tensor	weight()	Returns the FP8 weight tensor.
Tensor	weightScale()	Returns the float weight scale tensor.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

Fp8Linear

public Fp8Linear(Tensor weight,
 Tensor weightScale,
 Tensor bias,
 ScalarType outDtype)

Creates a weight-only FP8 linear layer.

Parameters:

weight - FP8 weight [outFeatures, inFeatures].

weightScale - float scale for weight.

bias - optional bias (same compute dtype), or null.

outDtype - output dtype (ScalarType.BFloat16 or ScalarType.Half).

Method Details

inFeatures

public int inFeatures()

Returns the input feature count.

Returns:

input feature count.

outFeatures

public int outFeatures()

Returns the output feature count.

Returns:

output feature count.

weight

public Tensor weight()

Returns the FP8 weight tensor.

Returns:

FP8 weight [outFeatures, inFeatures].

weightScale

public Tensor weightScale()

Returns the float weight scale tensor.

Returns:

float weight scale.

forward

public Tensor forward(Tensor input)

Description copied from interface: LinearOp

Applies the linear transformation.

Specified by:

forward in interface LinearOp

Parameters:

input - activations (caller retains ownership).

Returns:

owned output tensor (caller must close).

close

public void close()

Specified by:

close in interface AutoCloseable