Class DecodeForwardProfile

java.lang.Object
smile.llm.engine.DecodeForwardProfile
public final class DecodeForwardProfile extends Object
Optional decode-forward phase timers (attn / DeltaNet / MLP / NCCL).

Enable with SMILE_DECODE_PROFILE=1. Each TP rank accumulates on its worker thread; Qwen merges the max across ranks into DecodeStepTiming (NCCL barriers make wall times a reasonable proxy for GPU work between all-reduces).

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static final class 
    DecodeForwardProfile.Snapshot
    Per-thread / per-rank nanosecond accumulators.

  • Method Summary

    Modifier and Type
    Method
    Description
    static void
    addDeltaConv(long ns)
    DeltaNet: causal conv1d update / prefill.
    static void
    addDeltaGate(long ns)
    DeltaNet: beta/sigmoid + softplus gate g.
    static void
    addDeltaOut(long ns)
    DeltaNet: RMSNorm-gated + out projection.
    static void
    addDeltaProj(long ns)
    DeltaNet: input projections (qkv/z/b/a).
    static void
    addDeltaRecurrent(long ns)
    DeltaNet: recurrent gated-delta rule (native or Java).
    static void
    addEmbed(long ns)
    Adds embedding time on the current TP worker thread.
    static void
    addFullAttn(long ns)
    Adds full-attention mixer time (excluding the trailing all-reduce).
    static void
    addLinearAttn(long ns)
    Adds DeltaNet / linear-attention mixer time (excluding all-reduce).
    static void
    addLmHead(long ns)
    Adds final norm + lm_head time.
    static void
    addMlp(long ns)
    Adds MLP time (excluding the trailing all-reduce).
    static void
    addNccl(long ns)
    Adds NCCL / TP all-reduce wall time.
    static boolean
    enabled()
    Returns whether detailed decode phase timing is enabled.
    static DecodeForwardProfile.Snapshot
    snapshotAndReset()
    Returns the current thread's accumulators and resets them for the next forward.

    Methods inherited from class Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • enabled

      public static boolean enabled()
      Returns whether detailed decode phase timing is enabled.
      Returns:
      true when detailed decode phase timing is enabled.

    • addEmbed

      public static void addEmbed(long ns)
      Adds embedding time on the current TP worker thread.
      Parameters:
      ns - elapsed nanoseconds to add.

    • addFullAttn

      public static void addFullAttn(long ns)
      Adds full-attention mixer time (excluding the trailing all-reduce).
      Parameters:
      ns - elapsed nanoseconds to add.

    • addLinearAttn

      public static void addLinearAttn(long ns)
      Adds DeltaNet / linear-attention mixer time (excluding all-reduce).
      Parameters:
      ns - elapsed nanoseconds to add.

    • addMlp

      public static void addMlp(long ns)
      Adds MLP time (excluding the trailing all-reduce).
      Parameters:
      ns - elapsed nanoseconds to add.

    • addNccl

      public static void addNccl(long ns)
      Adds NCCL / TP all-reduce wall time.
      Parameters:
      ns - elapsed nanoseconds to add.

    • addLmHead

      public static void addLmHead(long ns)
      Adds final norm + lm_head time.
      Parameters:
      ns - elapsed nanoseconds to add.

    • addDeltaProj

      public static void addDeltaProj(long ns)
      DeltaNet: input projections (qkv/z/b/a).
      Parameters:
      ns - elapsed nanoseconds to add.

    • addDeltaConv

      public static void addDeltaConv(long ns)
      DeltaNet: causal conv1d update / prefill.
      Parameters:
      ns - elapsed nanoseconds to add.

    • addDeltaGate

      public static void addDeltaGate(long ns)
      DeltaNet: beta/sigmoid + softplus gate g.
      Parameters:
      ns - elapsed nanoseconds to add.

    • addDeltaRecurrent

      public static void addDeltaRecurrent(long ns)
      DeltaNet: recurrent gated-delta rule (native or Java).
      Parameters:
      ns - elapsed nanoseconds to add.

    • addDeltaOut

      public static void addDeltaOut(long ns)
      DeltaNet: RMSNorm-gated + out projection.
      Parameters:
      ns - elapsed nanoseconds to add.

    • snapshotAndReset

      public static DecodeForwardProfile.Snapshot snapshotAndReset()
      Returns the current thread's accumulators and resets them for the next forward. Call from the TP worker after forward returns.
      Returns:
      snapshot of accumulators, or null when profiling is off.