Class TextStreamer

java.lang.Object

smile.llm.engine.TextStreamer

public final class TextStreamer
extends Object

Coalesces generated token ids into streamed UTF-8 text chunks.

Field Summary

Fields

Modifier and Type	Field	Description
static final int	DEFAULT_CHUNK_TOKENS	Default coalesce length before attempting a decode flush.

Constructor Summary

Constructors

Constructor	Description
TextStreamer(int maxTokens, int chunkTokens, BiFunction<int[],Boolean,String> tryDecode)	Creates a text streamer with an explicit coalesce length.
TextStreamer(int maxTokens, BiFunction<int[],Boolean,String> tryDecode)	Creates a text streamer with DEFAULT_CHUNK_TOKENS.

Method Summary

Modifier and Type	Method	Description
void	accept(int tokenId)	Appends one generated token id.
void	maybeEmit(GenerationListener listener, boolean force)	Flushes a text chunk when enough tokens accumulated or force.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

DEFAULT_CHUNK_TOKENS

public static final int DEFAULT_CHUNK_TOKENS

Default coalesce length before attempting a decode flush.

See Also:

• Constant Field Values

Constructor Details

TextStreamer

public TextStreamer(int maxTokens,
 BiFunction<int[],Boolean,String> tryDecode)

Creates a text streamer with DEFAULT_CHUNK_TOKENS.

Parameters:

maxTokens - capacity of the completion token buffer.

tryDecode - (tokens, skipSpecial) -> text; may throw CharacterCodingException as unchecked via wrapper.

TextStreamer

public TextStreamer(int maxTokens,
 int chunkTokens,
 BiFunction<int[],Boolean,String> tryDecode)

Creates a text streamer with an explicit coalesce length.

Parameters:

maxTokens - capacity of the completion token buffer.

chunkTokens - coalesce length before attempting a decode flush.

tryDecode - (tokens, skipSpecial) -> text.

Method Details

accept

public void accept(int tokenId)

Appends one generated token id.

Parameters:

tokenId - generated token id.

maybeEmit

public void maybeEmit(GenerationListener listener,
 boolean force)

Flushes a text chunk when enough tokens accumulated or force.

Parameters:

listener - destination; may be null.

force - flush even if below chunkTokens.