Class DecodeCudaGraphLog

java.lang.Object
smile.llm.engine.DecodeCudaGraphLog
public final class DecodeCudaGraphLog extends Object
INFO logging for decode CUDA graph bucket lifecycle (once per bucket event).

Used to compare on-demand capture hitches vs pre-captured bucket promotion.

  • Method Summary

    Modifier and Type
    Method
    Description
    static void
    bucketCapture(int tpRank, int batch, int numPages, long captureMs, boolean prefetched)
    Logs successful graph capture for a bucket.
    static void
    bucketReplay(int tpRank, int batch, int numPages)
    Logs the first replay for a bucket in this process.
    static void
    bucketWarmup(int tpRank, int batch, int numPages, int step, int total)
    Logs the first eager warmup step for a bucket.
    static void
    prefetchHit(int tpRank, int batch, int numPages)
    Logs promotion of a prefetched graph at a page boundary (no capture hitch).
    static void
    prefetchReady(int tpRank, int batch, int numPages, long captureMs)
    Logs when a prefetched bucket becomes replay-ready.
    static void
    prefetchStart(int tpRank, int batch, int numPages)
    Logs when background prefetch starts for the next page bucket.
    static void
    prefetchWarmup(int tpRank, int batch, int numPages, int step, int total)
    Logs the first eager warmup step during next-bucket prefetch capture.

    Methods inherited from class Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • bucketWarmup

      public static void bucketWarmup(int tpRank, int batch, int numPages, int step, int total)
      Logs the first eager warmup step for a bucket.
      Parameters:
      tpRank - tensor-parallel rank for logging.
      batch - decode batch size.
      numPages - KV page count for this bucket.
      step - current warmup step (1-based).
      total - total warmup steps before capture.

    • bucketCapture

      public static void bucketCapture(int tpRank, int batch, int numPages, long captureMs, boolean prefetched)
      Logs successful graph capture for a bucket.
      Parameters:
      tpRank - tensor-parallel rank for logging.
      batch - decode batch size.
      numPages - KV page count for this bucket.
      captureMs - capture duration in milliseconds.
      prefetched - true when capture was a next-bucket prefetch.

    • bucketReplay

      public static void bucketReplay(int tpRank, int batch, int numPages)
      Logs the first replay for a bucket in this process.
      Parameters:
      tpRank - tensor-parallel rank for logging.
      batch - decode batch size.
      numPages - KV page count for this bucket.

    • prefetchWarmup

      public static void prefetchWarmup(int tpRank, int batch, int numPages, int step, int total)
      Logs the first eager warmup step during next-bucket prefetch capture.
      Parameters:
      tpRank - tensor-parallel rank for logging.
      batch - decode batch size.
      numPages - KV page count for this bucket.
      step - current warmup step (1-based).
      total - total warmup steps before capture.

    • prefetchStart

      public static void prefetchStart(int tpRank, int batch, int numPages)
      Logs when background prefetch starts for the next page bucket.
      Parameters:
      tpRank - tensor-parallel rank for logging.
      batch - decode batch size.
      numPages - KV page count for this bucket.

    • prefetchReady

      public static void prefetchReady(int tpRank, int batch, int numPages, long captureMs)
      Logs when a prefetched bucket becomes replay-ready.
      Parameters:
      tpRank - tensor-parallel rank for logging.
      batch - decode batch size.
      numPages - KV page count for this bucket.
      captureMs - capture duration in milliseconds.

    • prefetchHit

      public static void prefetchHit(int tpRank, int batch, int numPages)
      Logs promotion of a prefetched graph at a page boundary (no capture hitch).
      Parameters:
      tpRank - tensor-parallel rank for logging.
      batch - decode batch size.
      numPages - KV page count for this bucket.