Record Class MatchResult

java.lang.Object
java.lang.Record
smile.llm.cache.MatchResult
Record Components:
indices - a freshly-allocated 1-D int64 tensor containing the concatenated KV cache slot indices for the longest cached prefix found. Its length is always a multiple of the cache's pageSize. An empty tensor (length 0) means nothing was found in the cache.
lastNode - the deepest RadixTreeNode reached during matching. Callers should pass this node to RadixCache.incLockRef(RadixTreeNode) to protect the matched prefix from eviction while the request is in flight, and to RadixCache.decLockRef(RadixTreeNode) when done.
All Implemented Interfaces:
AutoCloseable
public record MatchResult(Tensor indices, RadixTreeNode lastNode) extends Record implements AutoCloseable
Result returned by RadixCache.matchPrefix(int[]).

This record implements AutoCloseable so callers can use try-with-resources to release the indices() tensor automatically: 

try (var result = cache.matchPrefix(tokenIds)) {
    process(result.indices());
}

The caller owns the indices() tensor. If try-with-resources is not used, the caller must call close() (or indices().close()) explicitly.

  • Constructor Details

    • MatchResult

      public MatchResult(Tensor indices, RadixTreeNode lastNode)
      Creates an instance of a MatchResult record class.
      Parameters:
      indices - the value for the indices record component
      lastNode - the value for the lastNode record component

  • Method Details

    • length

      public int length()
      Returns the number of matched token slots (equal to indices.length()).
      Returns:
      the number of matched tokens.

    • close

      public void close()
      Releases the indices() tensor. Called automatically when this result is used in a try-with-resources statement.
      Specified by:
      close in interface AutoCloseable

    • toString

      public final String toString()
      Returns a string representation of this record class. The representation contains the name of the class, followed by the name and value of each of the record components.
      Specified by:
      toString in class Record
      Returns:
      a string representation of this object

    • hashCode

      public final int hashCode()
      Returns a hash code value for this object. The value is derived from the hash code of each of the record components.
      Specified by:
      hashCode in class Record
      Returns:
      a hash code value for this object

    • equals

      public final boolean equals(Object o)
      Indicates whether some other object is "equal to" this one. The objects are equal if the other object is of the same class and if all the record components are equal. All components in this record class are compared with Objects::equals(Object,Object).
      Specified by:
      equals in class Record
      Parameters:
      o - the object with which to compare
      Returns:
      true if this object is the same as the o argument; false otherwise.

    • indices

      public Tensor indices()
      Returns the value of the indices record component.
      Returns:
      the value of the indices record component

    • lastNode

      public RadixTreeNode lastNode()
      Returns the value of the lastNode record component.
      Returns:
      the value of the lastNode record component