Class Fp8BlockLinear

java.lang.Object

smile.llm.quant.Fp8BlockLinear

All Implemented Interfaces:

AutoCloseable, LinearOp

public final class Fp8BlockLinear
extends Object
implements LinearOp, AutoCloseable

Block-scaled FP8 linear for DeepSeek / Qwen fine-grained FP8 checkpoints (weight_block_size=[128,128] + weight_scale_inv).

Uses LibTorch at::_scaled_mm_v2 with activation BlockWise1x128 and weight BlockWise128x128 scales.

Constructor Summary

Constructors

Constructor	Description
Fp8BlockLinear(Tensor weight, Tensor weightScaleInv, Tensor bias, ScalarType outDtype)	Creates a block-scaled FP8 linear layer.

Method Summary

Modifier and Type	Method	Description
void	close()
Tensor	forward(Tensor input)	Applies the linear transformation.
int	inFeatures()	Returns the input feature count.
int	outFeatures()	Returns the output feature count.
Tensor	weight()	Returns the FP8 weight tensor.
Tensor	weightScaleInv()	Returns the block inverse scale tensor.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

Fp8BlockLinear

public Fp8BlockLinear(Tensor weight,
 Tensor weightScaleInv,
 Tensor bias,
 ScalarType outDtype)

Creates a block-scaled FP8 linear layer.

Parameters:

weight - FP8 weight [outFeatures, inFeatures].

weightScaleInv - float32 block inverse scales [ceil(out/128), ceil(in/128)].

bias - optional bias, or null.

outDtype - output dtype (BF16 or FP16).

Method Details

inFeatures

public int inFeatures()

Returns the input feature count.

Returns:

input feature count.

outFeatures

public int outFeatures()

Returns the output feature count.

Returns:

output feature count.

weight

public Tensor weight()

Returns the FP8 weight tensor.

Returns:

FP8 weight [outFeatures, inFeatures].

weightScaleInv

public Tensor weightScaleInv()

Returns the block inverse scale tensor.

Returns:

block inverse scales.

forward

public Tensor forward(Tensor input)

Description copied from interface: LinearOp

Applies the linear transformation.

Specified by:

forward in interface LinearOp

Parameters:

input - activations (caller retains ownership).

Returns:

owned output tensor (caller must close).

close

public void close()

Specified by:

close in interface AutoCloseable