Class QuantLinearFactory

java.lang.Object
smile.llm.quant.QuantLinearFactory
public final class QuantLinearFactory extends Object
Factory for LinearOp backends from checkpoint tensors + selected backend.

  • Method Details

    • dense

      public static LinearOp dense(int inFeatures, int outFeatures, boolean bias)
      Builds a dense LinearLayer wrapped as LinearOp.
      Parameters:
      inFeatures - input features.
      outFeatures - output features.
      bias - whether to include bias.
      Returns:
      dense linear op.

    • fp8

      public static LinearOp fp8(Tensor weightFp8, Tensor weightScale, Tensor bias, ScalarType outDtype)
      Builds an FP8 linear from native FP8 weight + tensor scale.
      Parameters:
      weightFp8 - FP8 weight.
      weightScale - float scale.
      bias - optional bias, or null.
      outDtype - output dtype.
      Returns:
      FP8 linear op.

    • fp8Block

      public static LinearOp fp8Block(Tensor weightFp8, Tensor weightScaleInv, Tensor bias, ScalarType outDtype)
      Builds a block-scaled FP8 linear (DeepSeek / Qwen block-128 layout).
      Parameters:
      weightFp8 - FP8 weight.
      weightScaleInv - block inverse scales.
      bias - optional bias, or null.
      outDtype - output dtype.
      Returns:
      block FP8 linear op.

    • nvfp4

      public static LinearOp nvfp4(Tensor weight, Tensor weightScale, Tensor bias)
      Builds an NVFP4 linear (may throw at forward until LibTorch NVFP4 is wired).
      Parameters:
      weight - NVFP4 weight.
      weightScale - weight scale.
      bias - optional bias, or null.
      Returns:
      NVFP4 linear op.

    • marlinFromGptq

      public static LinearOp marlinFromGptq(Tensor qweight, Tensor scales, Tensor qzeros, Tensor gIdx, int groupSize, Device device)
      Packs GPTQ tensors and builds MarlinLinear (Ampere/Ada only). Uses direct GPTQ→Marlin pack (act-order checkpoints fail fast).
      Parameters:
      qweight - GPTQ packed weights.
      scales - group scales.
      qzeros - packed zeros, or null.
      gIdx - GPTQ g_idx, or null.
      groupSize - quantization group size.
      device - destination device.
      Returns:
      Marlin linear op.

    • marlinFromAwq

      public static LinearOp marlinFromAwq(Tensor qweight, Tensor scales, Tensor qzeros, int groupSize, Device device)
      Packs AWQ tensors and builds MarlinLinear via direct AWQ→Marlin pack.
      Parameters:
      qweight - AWQ packed weights.
      scales - group scales.
      qzeros - packed zeros, or null.
      groupSize - quantization group size.
      device - destination device.
      Returns:
      Marlin linear op.