Class DecodeStepTiming

java.lang.Object
smile.llm.engine.DecodeStepTiming
public final class DecodeStepTiming extends Object
Per-thread decode step timing filled by Qwen and read by InferenceEngine.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    long
    forwardNs
    Wall time for TP forward (max(rank) when tensor-parallel).
    long
    logitsNs
    Logits row extraction / copy after forward.
    long
    prepNs
    KV activate + host tensor prep before TP forward.
    DecodeForwardProfile.Snapshot
    profile
    Max-across-ranks phase breakdown when SMILE_DECODE_PROFILE=1; otherwise null.
    long
    sampleNs
    Greedy / sampled token selection after forward.
    long
    slowestRankNs
    Slowest TP rank forward (GPU work on critical path).
    long
    tpBarrierNs
    Thread-pool sync overhead (forwardNs - slowestRankNs).

  • Constructor Summary

    Constructors
    Constructor
    Description
    DecodeStepTiming()
    Creates an empty timing/snapshot holder.

  • Method Summary

    Modifier and Type
    Method
    Description
    static void
    clear()
    Clears the thread-local holder (call after logging).
    static DecodeStepTiming
    current()
    Returns the timing holder for the current decode worker thread.

    Methods inherited from class Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • prepNs

      public long prepNs
      KV activate + host tensor prep before TP forward.

    • forwardNs

      public long forwardNs
      Wall time for TP forward (max(rank) when tensor-parallel).

    • slowestRankNs

      public long slowestRankNs
      Slowest TP rank forward (GPU work on critical path).

    • tpBarrierNs

      public long tpBarrierNs
      Thread-pool sync overhead (forwardNs - slowestRankNs).

    • logitsNs

      public long logitsNs
      Logits row extraction / copy after forward.

    • sampleNs

      public long sampleNs
      Greedy / sampled token selection after forward.

    • profile

      public DecodeForwardProfile.Snapshot profile
      Max-across-ranks phase breakdown when SMILE_DECODE_PROFILE=1; otherwise null.

  • Constructor Details

    • DecodeStepTiming

      public DecodeStepTiming()
      Creates an empty timing/snapshot holder.

  • Method Details

    • current

      public static DecodeStepTiming current()
      Returns the timing holder for the current decode worker thread.
      Returns:
      timing holder for the current decode worker thread.

    • clear

      public static void clear()
      Clears the thread-local holder (call after logging).