Class GeneratorParams

java.lang.Object

smile.onnx.genai.GeneratorParams

All Implemented Interfaces:

AutoCloseable

public final class GeneratorParams
extends Object
implements AutoCloseable

Search and sampling options for a Generator.

Common search option names include max_length, temperature, top_p, top_k, do_sample, and repetition_penalty.

Method Summary

Modifier and Type	Method	Description
void	close()
static GeneratorParams	of(Model model)	Creates generator parameters for a model.
GeneratorParams	setGuidance(String type, String data, boolean enableFfTokens)	Sets guidance (e.g.
GeneratorParams	setSearchOption(String name, boolean value)	Sets a boolean search option.
GeneratorParams	setSearchOption(String name, double value)	Sets a numeric search option.
GeneratorParams	setSpeculativeOption(String name, boolean value)	Sets a boolean speculative-decoding option.
GeneratorParams	setSpeculativeOption(String name, double value)	Sets a numeric speculative-decoding option.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

of

public static GeneratorParams of(Model model)

Creates generator parameters for a model.

Parameters:

model - the model.

Returns:

a new params object owned by the caller.

setSearchOption

public GeneratorParams setSearchOption(String name,
 double value)

Sets a numeric search option.

Parameters:

name - option name.

value - option value.

Returns:

this for chaining.

setSearchOption

public GeneratorParams setSearchOption(String name,
 boolean value)

Sets a boolean search option.

Parameters:

name - option name.

value - option value.

Returns:

this for chaining.

setGuidance

public GeneratorParams setGuidance(String type,
 String data,
 boolean enableFfTokens)

Sets guidance (e.g. JSON schema / grammar) for constrained decoding.

Parameters:

type - guidance type.

data - guidance payload.

enableFfTokens - whether to enable feed-forward tokens.

Returns:

this for chaining.

setSpeculativeOption

public GeneratorParams setSpeculativeOption(String name,
 double value)

Sets a numeric speculative-decoding option.

Parameters:

name - option name.

value - option value.

Returns:

this for chaining.

setSpeculativeOption

public GeneratorParams setSpeculativeOption(String name,
 boolean value)

Sets a boolean speculative-decoding option.

Parameters:

name - option name.

value - option value.

Returns:

this for chaining.

close

public void close()

Specified by:

close in interface AutoCloseable