Class GenAI

java.lang.Object
smile.onnx.genai.GenAI
public final class GenAI extends Object
Process-wide ONNX Runtime GenAI controls: library availability, telemetry, logging helpers, and shutdown.

Both onnxruntime and onnxruntime-genai shared libraries must be on the OS library search path. The JVM also needs --enable-native-access=ALL-UNNAMED.

See Also:

  • Field Details

    • ORT_NATIVE_PATH_PROPERTY

      public static final String ORT_NATIVE_PATH_PROPERTY
      Directory containing onnxruntime (system property onnxruntime.native.path).
      See Also:

    • GENAI_NATIVE_PATH_PROPERTY

      public static final String GENAI_NATIVE_PATH_PROPERTY
      Directory containing onnxruntime-genai (system property onnxruntime-genai.native.path).
      See Also:

    • PROVIDER_PROPERTY

      public static final String PROVIDER_PROPERTY
      Execution-provider preference: auto (default), cuda, npu, ryzenai/hybrid, openvino, qnn, dml/directml, or cpu. Environment variable SMILE_ONNX_GENAI_PROVIDER overrides the system property smile.onnx.genai.provider.

      Classical Vitis AI for general ONNX is configured via SessionOptions.appendVitisAiExecutionProvider(), not this GenAI preference.

      See Also:

    • PROVIDER_ENV

      public static final String PROVIDER_ENV
      Environment variable for PROVIDER_PROPERTY.
      See Also:

  • Method Details

    • providerPreference

      public static String providerPreference()
      Returns the preferred GenAI execution-provider mode.
      Returns:
      normalized preference: auto, cuda, npu, ryzenai, openvino, qnn, dml, or cpu.

    • cudaProviderUsable

      public static Boolean cudaProviderUsable()
      Returns whether a previous Model.open(Path) successfully used the CUDA EP, or null if CUDA has not been attempted yet in this process.
      Returns:
      true/false after a CUDA attempt, else null.

    • cudaNativesPresent

      public static boolean cudaNativesPresent()
      Returns whether CUDA-capable GenAI/ORT shared libraries were found on the configured native library path (e.g. pip onnxruntime-genai-cuda).
      Returns:
      true when CUDA companion natives are present.

    • supportsFp8

      public static boolean supportsFp8(String candidateId)
      Returns whether candidate id is known to support FP8 Olive builds.

      Olive optimize currently does not accept fp8 in its CLI precision enum; keep this false until that changes.

      Parameters:
      candidateId - GenAI cascade id (e.g. cuda).
      Returns:
      true when auto precision should try fp8 first.

    • resolveOliveTarget

      public static GenAIOliveTarget resolveOliveTarget()
      Resolves Olive optimize --device / --provider / default precision from providerPreference() and EP natives (same order as Model.open(Path)).
      Returns:
      Olive target; cpu/int4 when no accelerator is present.

    • available

      public static boolean available()
      Returns whether the onnxruntime-genai native library can be loaded.
      Returns:
      true when the FFM bindings initialize successfully.

    • init

      public static void init()
      Forces native library load. Prefer calling this early so missing natives fail fast with a clear exception.

      On Windows, if an older onnxruntime.dll lives in System32, set "onnxruntime.native.path" and "onnxruntime-genai.native.path" to the pip package directories so the correct DLLs are System.loadRESTRICTEDed first.

      Throws:
      GenAIException - if the native library cannot be loaded.

    • setTelemetry

      public static void setTelemetry(boolean enabled)
      Enables or disables non-essential GenAI telemetry events.
      Parameters:
      enabled - true to enable telemetry.

    • setLogBool

      public static void setLogBool(String name, boolean value)
      Sets a boolean logging option (e.g. "enabled", "append").
      Parameters:
      name - option name.
      value - option value.

    • setLogString

      public static void setLogString(String name, String value)
      Sets a string logging option (e.g. "filename", "level").
      Parameters:
      name - option name.
      value - option value.

    • isInitialized

      public static boolean isInitialized()
      Returns whether the GenAI library has been initialized in this process.
      Returns:
      true if initialized, false otherwise.

    • shutdown

      public static void shutdown()
      Tears down GenAI globals. Destroy every GenAI object before calling this. Safe to call multiple times or when GenAI was never initialized.

      After shutdown, the next GenAI API call re-initializes the library.