Interface SafeTensorsLoaderThreads

public interface SafeTensorsLoaderThreads
Resolves the safetensors loader thread-pool size.

Peak host RAM during load scales roughly as threads × in-flight shard footprint (each worker holds at most one shard on CPU at a time).

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final int
    AUTO_CAP
    Default auto cap when configured == 0.

  • Method Summary

    Static Methods
    Modifier and Type
    Method
    Description
    static int
    resolve(int configured, int numShards)
    Resolves loader concurrency.

  • Field Details

    • AUTO_CAP

      static final int AUTO_CAP
      Default auto cap when configured == 0.
      See Also:

  • Method Details

    • resolve

      static int resolve(int configured, int numShards)
      Resolves loader concurrency. 
      auto     = min(AUTO_CAP, availableProcessors)
threads  = configured > 0 ? configured : auto
result   = min(threads, numShards)   // at least 1 when numShards >= 1
      Parameters:
      configured - 0 means auto; must not be negative.
      numShards - number of safetensors shard files (>= 0).
      Returns:
      pool size; 0 when there are no shards.