Interface MxFp8Layer

All Superinterfaces:
AutoCloseable, LinearOp
All Known Implementing Classes:
MxFp8BlockLinear, MxFp8MarlinLinear
public interface MxFp8Layer extends LinearOp, AutoCloseable
A linear layer that keeps its weight in MXFP8 form (in a backend-specific layout) and receives it from the checkpoint after construction.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    close()
     
    boolean
    isLoaded()
    Returns whether the weight has been installed.
    void
    setWeight(Tensor weight, Tensor scale)
    Installs the weight from checkpoint tensors.

    Methods inherited from interface LinearOp

    forward

  • Method Details

    • setWeight

      void setWeight(Tensor weight, Tensor scale)
      Installs the weight from checkpoint tensors.
      Parameters:
      weight - FP8 E4M3 [N, K] on the target CUDA device.
      scale - E8M0 scales as uint8 [N, K / 32] on the same device.

    • isLoaded

      boolean isLoaded()
      Returns whether the weight has been installed.
      Returns:
      true once setWeight(Tensor, Tensor) has run.

    • close

      void close()
      Specified by:
      close in interface AutoCloseable