Class FlashInferArtifacts

java.lang.Object

smile.llm.attention.FlashInferArtifacts

public final class FlashInferArtifacts
extends Object

Resolves FlashInfer AOT / jit-cache directory for hybrid packaging: explicit dir → env / bundled default → cache-dir → optional download.

Download-only (no nvcc). Artifacts are flashinfer-jit-cache wheels, not flashinfer-cubin.

Field Summary

Fields

Modifier and Type	Field	Description
static final String	BUNDLED_AOT_DIR	Default install path inside the GPU Docker image.
static final String	DEFAULT_JIT_CACHE_VERSION	Pinned jit-cache release used when smile.chat.flashinfer-download=true.

Method Summary

Modifier and Type	Method	Description
static Optional<Path>	resolveAndInstall(String aotDir, String cacheDir, boolean allowDownload, String cudaTag)	Resolves and installs the AOT directory into the native library.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

BUNDLED_AOT_DIR

public static final String BUNDLED_AOT_DIR

Default install path inside the GPU Docker image.

See Also:

• Constant Field Values

DEFAULT_JIT_CACHE_VERSION

public static final String DEFAULT_JIT_CACHE_VERSION

Pinned jit-cache release used when smile.chat.flashinfer-download=true.

See Also:

• Constant Field Values

Method Details

resolveAndInstall

public static Optional<Path> resolveAndInstall(String aotDir,
 String cacheDir,
 boolean allowDownload,
 String cudaTag)

Resolves and installs the AOT directory into the native library.

Parameters:

aotDir - explicit config path (may be blank).

cacheDir - download / extract cache (may be blank).

allowDownload - when true, fetch jit-cache into cacheDir if missing.

cudaTag - wheel CUDA tag such as cu130 (FlashInfer jit-cache max for 0.6.6; LibTorch may still be cu132).

Returns:

resolved directory, or empty when none available.