Class SimpleGenAI

java.lang.Object

smile.onnx.genai.SimpleGenAI

All Implemented Interfaces:

AutoCloseable

public final class SimpleGenAI
extends Object
implements AutoCloseable

Convenience façade for single-prompt text generation with an optional streaming callback.

try (var genai = SimpleGenAI.of("models/phi-3")) {
    try (var params = genai.createGeneratorParams()) {
        params.setSearchOption("max_length", 256);
        String text = genai.generate(params, "Hello!", System.out::print);
    }
}

Method Summary

Modifier and Type	Method	Description
void	close()
GeneratorParams	createGeneratorParams()	Creates generator parameters for this model.
String	generate(GeneratorParams params, String prompt)	Generates text without a streaming callback.
String	generate(GeneratorParams params, String prompt, Consumer<String> onChunk)	Generates text for a single prompt.
Model	model()	Returns the underlying model.
static SimpleGenAI	of(String modelDir)	Loads a GenAI model and its tokenizer from a model directory.
static SimpleGenAI	of(Config config)	Loads a GenAI model from config and creates its tokenizer.
static SimpleGenAI	open(String modelDir)	Opens a GenAI model preferring CUDA when available (see Model.open(String)).
static SimpleGenAI	open(Path modelDir)	Opens a GenAI model preferring CUDA when available.
Tokenizer	tokenizer()	Returns the underlying tokenizer.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

of

public static SimpleGenAI of(String modelDir)

Loads a GenAI model and its tokenizer from a model directory.

Parameters:

modelDir - GenAI model directory.

Returns:

façade owned by the caller.

open

public static SimpleGenAI open(String modelDir)

Opens a GenAI model preferring CUDA when available (see Model.open(String)).

Parameters:

modelDir - GenAI model directory.

Returns:

façade owned by the caller.

open

public static SimpleGenAI open(Path modelDir)

Opens a GenAI model preferring CUDA when available.

Parameters:

modelDir - GenAI model directory.

Returns:

façade owned by the caller.

of

public static SimpleGenAI of(Config config)

Loads a GenAI model from config and creates its tokenizer.

Parameters:

config - GenAI config.

Returns:

façade owned by the caller.

model

public Model model()

Returns the underlying model.

Returns:

model.

tokenizer

public Tokenizer tokenizer()

Returns the underlying tokenizer.

Returns:

tokenizer.

createGeneratorParams

public GeneratorParams createGeneratorParams()

Creates generator parameters for this model.

Returns:

params owned by the caller.

generate

public String generate(GeneratorParams params,
 String prompt,
 Consumer<String> onChunk)

Generates text for a single prompt.

Parameters:

params - search options (must outlive the call).

prompt - prompt text.

onChunk - optional per-token text callback; may be null.

Returns:

full decoded sequence (prompt + completion).

generate

public String generate(GeneratorParams params,
 String prompt)

Generates text without a streaming callback.

Parameters:

params - search options.

prompt - prompt text.

Returns:

full decoded sequence.

close

public void close()

Specified by:

close in interface AutoCloseable