Class Fp8BlockGemm

java.lang.Object

smile.llm.quant.Fp8BlockGemm

public final class Fp8BlockGemm
extends Object

DeepSeek-style W8A8 block FP8 matrix multiply: activations are quantized on the fly to E4M3 with one float32 scale per 1x128 group, weights are E4M3 with one float32 scale per 128x128 tile.

The native path calls the Hopper block-scaled GEMM exactly as Fp8BlockLinear does. The emulated path performs the same arithmetic (same activation quantizer, exact dequantization of both operands, float32 matmul) on any CUDA GPU, so everything except the tensor-core call itself can be validated without Hopper hardware.

Method Summary

Modifier and Type	Method	Description
static Tensor	mm(Tensor a, Tensor weight, Tensor scaleInv, ScalarType outDtype, boolean emulate)	Computes A @ W^T.
static String	selfCheck(Device device)	Runs the native block-scaled GEMM on a small random problem and compares it with the emulated arithmetic.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

mm

public static Tensor mm(Tensor a,
 Tensor weight,
 Tensor scaleInv,
 ScalarType outDtype,
 boolean emulate)

Computes A @ W^T.

Parameters:

a - activations [M, K], bfloat16 or float16, K a multiple of 128.

weight - FP8 weight [N, K].

scaleInv - float32 tile scales [N/128, K/128] (W = fp8 * scaleInv).

outDtype - result dtype.

emulate - true to emulate the tensor-core GEMM with dequantization.

Returns:

the product [M, N]; the caller owns it.

selfCheck

public static String selfCheck(Device device)

Runs the native block-scaled GEMM on a small random problem and compares it with the emulated arithmetic. Use it before trusting the native path on hardware this code has not run on.

Parameters:

device - a CUDA device.

Returns:

null when the native result matches, otherwise a description of the failure.