Class Model

java.lang.Object

smile.onnx.genai.Model

All Implemented Interfaces:

AutoCloseable

public final class Model
extends Object
implements AutoCloseable

An ONNX Runtime GenAI model. Outlives tokenizers and generators created from it.

Method Summary

Modifier and Type	Method	Description
void	close()
GeneratorParams	createGeneratorParams()	Creates generator parameters for this model.
Tokenizer	createTokenizer()	Creates a tokenizer bound to this model.
String	name()	Returns the model directory leaf name (or a placeholder when loaded from Config).
static Model	of(String modelDir)	Loads a model from a GenAI model directory using providers from genai_config.json.
static Model	of(Path modelDir)	Loads a model from a GenAI model directory using providers from genai_config.json.
static Model	of(Config config)	Loads a model from an existing Config.
static Model	open(String modelDir)	Opens a model with an accelerator cascade, then CPU.
static Model	open(Path modelDir)	Opens a model with an accelerator cascade, then CPU.
String	provider()	Returns which execution-provider path was selected for this instance.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

of

public static Model of(String modelDir)

Loads a model from a GenAI model directory using providers from genai_config.json.

Parameters:

modelDir - path containing genai_config.json and weights.

Returns:

a new model owned by the caller.

of

public static Model of(Path modelDir)

Loads a model from a GenAI model directory using providers from genai_config.json.

Parameters:

modelDir - path containing genai_config.json and weights.

Returns:

a new model owned by the caller.

open

public static Model open(Path modelDir)

Opens a model with an accelerator cascade, then CPU.

Default SMILE_ONNX_GENAI_PROVIDER=auto order: CUDA → RyzenAI → OpenVINO NPU → QNN → DirectML (Windows) → CPU (of(String)). Each accelerator is attempted only when matching EP natives are present; Java failures fall through to the next candidate.

Classical Vitis AI for general ONNX is SessionOptions.appendVitisAiExecutionProvider(), not part of this GenAI cascade.

Parameters:

modelDir - GenAI model directory.

Returns:

a new model owned by the caller.

open

public static Model open(String modelDir)

Opens a model with an accelerator cascade, then CPU.

Parameters:

modelDir - GenAI model directory.

Returns:

a new model owned by the caller.

See Also:

• open(Path)

of

public static Model of(Config config)

Loads a model from an existing Config.

Parameters:

config - config; may be closed after this returns (settings are copied).

Returns:

a new model owned by the caller.

createTokenizer

public Tokenizer createTokenizer()

Creates a tokenizer bound to this model.

Returns:

a new tokenizer owned by the caller.

createGeneratorParams

public GeneratorParams createGeneratorParams()

Creates generator parameters for this model.

Returns:

a new params object owned by the caller.

name

public String name()

Returns the model directory leaf name (or a placeholder when loaded from Config).

Returns:

display name.

provider

public String provider()

Returns which execution-provider path was selected for this instance.

Returns:

cuda, ryzenai, openvino, qnn, dml, default, or config.

close

public void close()

Specified by:

close in interface AutoCloseable