tvmjs
    Preparing search index...

    Interface SampledTokenReadbackRingOptions

    Options for creating a sampled-token readback ring.

    The ring is intended for batched GPU->CPU token-id readback where each token is represented as int32.

    interface SampledTokenReadbackRingOptions {
        maxTokensPerBatch: number;
        slotCount?: number;
    }
    Index
    maxTokensPerBatch: number

    Maximum number of token ids (int32) per submitted batch.

    slotCount?: number

    Number of staging slots in the ring. Defaults to 3 when unspecified.