tvmjs
    Preparing search index...

    Class SampledTokenReadbackRing

    Handle to a sampled-token readback ring created on a WebGPU device.

    Implements

    Index
    • Submit sampled-token readback from a GPU tensor.

      Parameters

      • from: Tensor

        GPU tensor containing int32 token ids.

      • tokenCount: number

        Number of token ids to copy.

      • fromTokenOffset: number = 0

        Token offset (in token units) from tensor start.

      • Optionalinto: Int32Array

        Host array to copy the token ids into when the batch maps, instead of a new array; the batch's tokens then view its first tokenCount elements. Reusing one array for large batches avoids an allocation per batch. The caller must not reuse it before it has consumed the batch; a ring disposed before the batch maps leaves it untouched.

      Returns number

      Monotonic ring-local batch sequence id.

    • Read raw 32-bit words from a WebGPU allocation without dtype conversion. This also supports floating-point probabilities and checkpoint bytes.

      Offsets and counts are in 32-bit words. The allocation bounds are checked. Callers must include any tensor byte offset and stay within its view. The destination array follows the same ownership rules as submit().

      Parameters

      • fromPtr: number
      • tokenCount: number
      • fromTokenOffset: number = 0
      • Optionalinto: Int32Array

      Returns number

    • Submit one sampled token from each GPU tensor as one contiguous readback batch.

      This avoids a GPU packing kernel when speculative decode produces K separate one-token tensors.

      Parameters

      • from: Tensor[]

        GPU tensors containing one int32 token id each.

      Returns number

      Monotonic ring-local batch sequence id.