Class RadixTreeNode

java.lang.Object
smile.llm.cache.RadixTreeNode
public class RadixTreeNode extends Object
A node in the radix tree used for KV cache management.

Each node represents an edge labeled with a span of token IDs (key) and the corresponding KV cache indices (value). The tree is a compressed prefix tree: common token prefixes are shared, so a single node can store many tokens at once.

Nodes are reference-counted via lockRef: a node with lockRef > 0 is protected and cannot be evicted. When all references are released the node becomes evictable and may be reclaimed by the LRU eviction policy.

  • Field Details

    • id

      public final long id
      Unique, monotonically increasing node identifier.

  • Method Details

    • isEvicted

      public boolean isEvicted()
      Returns true if this node's KV cache entries have been freed. A node is evicted when value is set to null.
      Returns:
      true if evicted.

    • toString

      public String toString()
      Overrides:
      toString in class Object