Record Class KvCachePool.StaticKvBudget

java.lang.Object
java.lang.Record
smile.llm.cache.KvCachePool.StaticKvBudget
Record Components:
total - device total memory bytes.
used - bytes already held (typically weights + DeltaNet).
staticBudget - total × memFraction.
kvBudget - bytes available for KV (staticBudget − used).
dynamicReserve - total − staticBudget.
numSlots - page-aligned slot count after context cap.
maxUsefulSlots - maxBatchSize × maxSeqLen page-aligned.
Enclosing class:
KvCachePool
public static record KvCachePool.StaticKvBudget(long total, long used, long staticBudget, long kvBudget, long dynamicReserve, int numSlots, int maxUsefulSlots) extends Record
Result of SGLang-style static KV budget arithmetic (testable without CUDA).

  • Constructor Summary

    Constructors
    Constructor
    Description
    StaticKvBudget(long total, long used, long staticBudget, long kvBudget, long dynamicReserve, int numSlots, int maxUsefulSlots)
    Creates an instance of a StaticKvBudget record class.

  • Method Summary

    Modifier and Type
    Method
    Description
    long
    dynamicReserve()
    Returns the value of the dynamicReserve record component.
    final boolean
    equals(Object o)
    Indicates whether some other object is "equal to" this one.
    final int
    hashCode()
    Returns a hash code value for this object.
    long
    kvBudget()
    Returns the value of the kvBudget record component.
    int
    maxUsefulSlots()
    Returns the value of the maxUsefulSlots record component.
    int
    numSlots()
    Returns the value of the numSlots record component.
    long
    staticBudget()
    Returns the value of the staticBudget record component.
    final String
    toString()
    Returns a string representation of this record class.
    long
    total()
    Returns the value of the total record component.
    long
    used()
    Returns the value of the used record component.

    Methods inherited from class Object

    clone, finalize, getClass, notify, notifyAll, wait, wait, wait

  • Constructor Details

    • StaticKvBudget

      public StaticKvBudget(long total, long used, long staticBudget, long kvBudget, long dynamicReserve, int numSlots, int maxUsefulSlots)
      Creates an instance of a StaticKvBudget record class.
      Parameters:
      total - the value for the total record component
      used - the value for the used record component
      staticBudget - the value for the staticBudget record component
      kvBudget - the value for the kvBudget record component
      dynamicReserve - the value for the dynamicReserve record component
      numSlots - the value for the numSlots record component
      maxUsefulSlots - the value for the maxUsefulSlots record component

  • Method Details

    • toString

      public final String toString()
      Returns a string representation of this record class. The representation contains the name of the class, followed by the name and value of each of the record components.
      Specified by:
      toString in class Record
      Returns:
      a string representation of this object

    • hashCode

      public final int hashCode()
      Returns a hash code value for this object. The value is derived from the hash code of each of the record components.
      Specified by:
      hashCode in class Record
      Returns:
      a hash code value for this object

    • equals

      public final boolean equals(Object o)
      Indicates whether some other object is "equal to" this one. The objects are equal if the other object is of the same class and if all the record components are equal. All components in this record class are compared with the compare method from their corresponding wrapper classes.
      Specified by:
      equals in class Record
      Parameters:
      o - the object with which to compare
      Returns:
      true if this object is the same as the o argument; false otherwise.

    • total

      public long total()
      Returns the value of the total record component.
      Returns:
      the value of the total record component

    • used

      public long used()
      Returns the value of the used record component.
      Returns:
      the value of the used record component

    • staticBudget

      public long staticBudget()
      Returns the value of the staticBudget record component.
      Returns:
      the value of the staticBudget record component

    • kvBudget

      public long kvBudget()
      Returns the value of the kvBudget record component.
      Returns:
      the value of the kvBudget record component

    • dynamicReserve

      public long dynamicReserve()
      Returns the value of the dynamicReserve record component.
      Returns:
      the value of the dynamicReserve record component

    • numSlots

      public int numSlots()
      Returns the value of the numSlots record component.
      Returns:
      the value of the numSlots record component

    • maxUsefulSlots

      public int maxUsefulSlots()
      Returns the value of the maxUsefulSlots record component.
      Returns:
      the value of the maxUsefulSlots record component