Record Class GenerationRequest

java.lang.Object
java.lang.Record
smile.llm.engine.GenerationRequest
Record Components:
promptTokens - token ids (required unless dialog is set).
dialog - optional chat turns; encoded by the engine when promptTokens is null.
multimodal - optional Qwen VL preprocess result (images/video); when set, promptTokens should match multimodal.inputIds().
maxGenLen - max new tokens.
temperature - sampling temperature.
topp - nucleus top-p.
logprobs - whether to return log-probabilities.
seed - RNG seed; 0 = non-deterministic.
listener - optional per-request listener.
chatOptions - optional tool-calling options.
speculative - when true, use native MTP draft/verify if available.
numSpeculativeTokens - draft depth; <= 0 uses the model default.
public record GenerationRequest(int[] promptTokens, Message[] dialog, QwenVlProcessor.ProcessedMultimodal multimodal, int maxGenLen, double temperature, double topp, boolean logprobs, long seed, GenerationListener listener, ChatOptions chatOptions, boolean speculative, int numSpeculativeTokens) extends Record
One generation job submitted to InferenceEngine.

  • Constructor Details

    • GenerationRequest

      public GenerationRequest(int[] promptTokens, Message[] dialog, QwenVlProcessor.ProcessedMultimodal multimodal, int maxGenLen, double temperature, double topp, boolean logprobs, long seed, GenerationListener listener, ChatOptions chatOptions, boolean speculative, int numSpeculativeTokens)
      Creates an instance of a GenerationRequest record class.
      Parameters:
      promptTokens - the value for the promptTokens record component
      dialog - the value for the dialog record component
      multimodal - the value for the multimodal record component
      maxGenLen - the value for the maxGenLen record component
      temperature - the value for the temperature record component
      topp - the value for the topp record component
      logprobs - the value for the logprobs record component
      seed - the value for the seed record component
      listener - the value for the listener record component
      chatOptions - the value for the chatOptions record component
      speculative - the value for the speculative record component
      numSpeculativeTokens - the value for the numSpeculativeTokens record component

  • Method Details

    • ofTokens

      public static GenerationRequest ofTokens(int[] promptTokens, int maxGenLen, double temperature, double topp, boolean logprobs, long seed, GenerationListener listener)
      Builds a request from already-tokenized prompt ids.
      Parameters:
      promptTokens - prompt token ids.
      maxGenLen - maximum generated tokens.
      temperature - sampling temperature.
      topp - nucleus sampling threshold.
      logprobs - whether to return log probabilities.
      seed - RNG seed.
      listener - optional generation listener, or null.
      Returns:
      new generation request.

    • ofTokens

      public static GenerationRequest ofTokens(int[] promptTokens, int maxGenLen, double temperature, double topp, boolean logprobs, long seed, GenerationListener listener, ChatOptions chatOptions)
      Builds a request from already-tokenized prompt ids with chat options.
      Parameters:
      promptTokens - prompt token ids.
      maxGenLen - maximum generated tokens.
      temperature - sampling temperature.
      topp - nucleus sampling threshold.
      logprobs - whether to return log probabilities.
      seed - RNG seed.
      listener - optional generation listener, or null.
      chatOptions - optional chat options, or null.
      Returns:
      new generation request.

    • ofTokens

      public static GenerationRequest ofTokens(int[] promptTokens, int maxGenLen, double temperature, double topp, boolean logprobs, long seed, GenerationListener listener, ChatOptions chatOptions, boolean speculative, int numSpeculativeTokens)
      Builds a request from already-tokenized prompt ids with speculation flags.
      Parameters:
      promptTokens - prompt token ids.
      maxGenLen - maximum generated tokens.
      temperature - sampling temperature.
      topp - nucleus sampling threshold.
      logprobs - whether to return log probabilities.
      seed - RNG seed.
      listener - optional generation listener, or null.
      chatOptions - optional chat options, or null.
      speculative - whether speculative decoding is requested.
      numSpeculativeTokens - draft tokens when speculative.
      Returns:
      new generation request.

    • ofMultimodal

      public static GenerationRequest ofMultimodal(QwenVlProcessor.ProcessedMultimodal multimodal, int maxGenLen, double temperature, double topp, boolean logprobs, long seed, GenerationListener listener)
      Builds a multimodal request (vision embeds spliced during prefill).
      Parameters:
      multimodal - processed multimodal inputs.
      maxGenLen - maximum generated tokens.
      temperature - sampling temperature.
      topp - nucleus sampling threshold.
      logprobs - whether to return log probabilities.
      seed - RNG seed.
      listener - optional generation listener, or null.
      Returns:
      new generation request.

    • ofMultimodal

      public static GenerationRequest ofMultimodal(QwenVlProcessor.ProcessedMultimodal multimodal, int maxGenLen, double temperature, double topp, boolean logprobs, long seed, GenerationListener listener, ChatOptions chatOptions)
      Builds a multimodal request with chat options.
      Parameters:
      multimodal - processed multimodal inputs.
      maxGenLen - maximum generated tokens.
      temperature - sampling temperature.
      topp - nucleus sampling threshold.
      logprobs - whether to return log probabilities.
      seed - RNG seed.
      listener - optional generation listener, or null.
      chatOptions - optional chat options, or null.
      Returns:
      new generation request.

    • ofMultimodal

      public static GenerationRequest ofMultimodal(QwenVlProcessor.ProcessedMultimodal multimodal, int maxGenLen, double temperature, double topp, boolean logprobs, long seed, GenerationListener listener, ChatOptions chatOptions, boolean speculative, int numSpeculativeTokens)
      Builds a multimodal request with speculation flags.
      Parameters:
      multimodal - processed multimodal inputs.
      maxGenLen - maximum generated tokens.
      temperature - sampling temperature.
      topp - nucleus sampling threshold.
      logprobs - whether to return log probabilities.
      seed - RNG seed.
      listener - optional generation listener, or null.
      chatOptions - optional chat options, or null.
      speculative - whether speculative decoding is requested.
      numSpeculativeTokens - draft tokens when speculative.
      Returns:
      new generation request.

    • ofDialog

      public static GenerationRequest ofDialog(Message[] dialog, int maxGenLen, double temperature, double topp, boolean logprobs, long seed, GenerationListener listener)
      Builds a request from a chat dialog (encoded at execution time).
      Parameters:
      dialog - chat messages.
      maxGenLen - maximum generated tokens.
      temperature - sampling temperature.
      topp - nucleus sampling threshold.
      logprobs - whether to return log probabilities.
      seed - RNG seed.
      listener - optional generation listener, or null.
      Returns:
      new generation request.

    • ofDialog

      public static GenerationRequest ofDialog(Message[] dialog, ChatOptions chatOptions, int maxGenLen, double temperature, double topp, boolean logprobs, long seed, GenerationListener listener)
      Builds a request from a chat dialog with chat options.
      Parameters:
      dialog - chat messages.
      chatOptions - chat options, or null.
      maxGenLen - maximum generated tokens.
      temperature - sampling temperature.
      topp - nucleus sampling threshold.
      logprobs - whether to return log probabilities.
      seed - RNG seed.
      listener - optional generation listener, or null.
      Returns:
      new generation request.

    • ofDialog

      public static GenerationRequest ofDialog(Message[] dialog, ChatOptions chatOptions, int maxGenLen, double temperature, double topp, boolean logprobs, long seed, GenerationListener listener, boolean speculative, int numSpeculativeTokens)
      Builds a request from a chat dialog with speculation flags.
      Parameters:
      dialog - chat messages.
      chatOptions - chat options, or null.
      maxGenLen - maximum generated tokens.
      temperature - sampling temperature.
      topp - nucleus sampling threshold.
      logprobs - whether to return log probabilities.
      seed - RNG seed.
      listener - optional generation listener, or null.
      speculative - whether speculative decoding is requested.
      numSpeculativeTokens - draft tokens when speculative.
      Returns:
      new generation request.

    • hasMultimodal

      public boolean hasMultimodal()
      Returns whether vision prefill is required.
      Returns:
      true when vision prefill is required.

    • toString

      public final String toString()
      Returns a string representation of this record class. The representation contains the name of the class, followed by the name and value of each of the record components.
      Specified by:
      toString in class Record
      Returns:
      a string representation of this object

    • hashCode

      public final int hashCode()
      Returns a hash code value for this object. The value is derived from the hash code of each of the record components.
      Specified by:
      hashCode in class Record
      Returns:
      a hash code value for this object

    • equals

      public final boolean equals(Object o)
      Indicates whether some other object is "equal to" this one. The objects are equal if the other object is of the same class and if all the record components are equal. Reference components are compared with Objects::equals(Object,Object); primitive components are compared with the compare method from their corresponding wrapper classes.
      Specified by:
      equals in class Record
      Parameters:
      o - the object with which to compare
      Returns:
      true if this object is the same as the o argument; false otherwise.

    • promptTokens

      public int[] promptTokens()
      Returns the value of the promptTokens record component.
      Returns:
      the value of the promptTokens record component

    • dialog

      public Message[] dialog()
      Returns the value of the dialog record component.
      Returns:
      the value of the dialog record component

    • multimodal

      public QwenVlProcessor.ProcessedMultimodal multimodal()
      Returns the value of the multimodal record component.
      Returns:
      the value of the multimodal record component

    • maxGenLen

      public int maxGenLen()
      Returns the value of the maxGenLen record component.
      Returns:
      the value of the maxGenLen record component

    • temperature

      public double temperature()
      Returns the value of the temperature record component.
      Returns:
      the value of the temperature record component

    • topp

      public double topp()
      Returns the value of the topp record component.
      Returns:
      the value of the topp record component

    • logprobs

      public boolean logprobs()
      Returns the value of the logprobs record component.
      Returns:
      the value of the logprobs record component

    • seed

      public long seed()
      Returns the value of the seed record component.
      Returns:
      the value of the seed record component

    • listener

      public GenerationListener listener()
      Returns the value of the listener record component.
      Returns:
      the value of the listener record component

    • chatOptions

      public ChatOptions chatOptions()
      Returns the value of the chatOptions record component.
      Returns:
      the value of the chatOptions record component

    • speculative

      public boolean speculative()
      Returns the value of the speculative record component.
      Returns:
      the value of the speculative record component

    • numSpeculativeTokens

      public int numSpeculativeTokens()
      Returns the value of the numSpeculativeTokens record component.
      Returns:
      the value of the numSpeculativeTokens record component