Options
All
  • Public
  • Public/Protected
  • All
Menu

Class Instance

TVM runtime instance.

All objects(NDArray, Module, PackedFunc) returned by TVM runtim function call and PackedFunc instance are tracked through a scope mechanism that will get auto-released when we call EndScope.

This is necessarily to be able to release the underlying WASM and WebGPU memory that are not tracked through JS native garbage collection mechanism.

This does mean that we have to get familar with the following functions:

Hierarchy

  • Instance

Implements

Index

Constructors

Properties

Methods

Constructors

constructor

  • new Instance(wasmModule: Module, importObject?: Record<string, any>, wasmInstance?: WebAssembly.Instance, env?: Environment): Instance

  • Constructor

    importObject can also be a LibraryProvider object, a WASI object, or an object containing wasmLibraryProvider field.

    see

    Please use the async version instantiate when targeting browsers.

    Parameters

    • wasmModule: Module

      The input module or instance.

    • Default value importObject: Record<string, any> = {}

      The imports to initialize the wasmInstance if it is not provided.

    • Optional wasmInstance: WebAssembly.Instance

      Additional wasm instance argument for deferred construction.

    • Optional env: Environment

      Directly specified environment module.

    Returns Instance

Properties

exports

exports: Record<string, Function>

memory

memory: Memory

Methods

attachToCurrentScope

  • attachToCurrentScope<T>(obj: T): T

  • Attach a detached obj to the auto-release pool of the current scope.

    note

    Normally user do not need to call this function explicitly, as all library call return values are explicitly attached to the current scope. You only need to do so when you call detachFromCurrentScope to create a detached object.

    Type parameters

    Parameters

    • obj: T

      The input obj.

    Returns T

beginScope

  • beginScope(): void

  • Begin a new scope for tracking object disposal.

    Returns void

benchmark

  • benchmark(run: () => void, dev: DLDevice, number?: number, repeat?: number): Promise<number[]>

  • Benchmark stable execution of the run function.

    params

    run The run function

    params

    dev The device to sync during each run.

    number

    The number of times to compute the average.

    repeat

    The number of times to repeat the run.

    Parameters

    • run: () => void
        • (): void

        • Returns void

    • dev: DLDevice
    • Default value number: number = 10
    • Default value repeat: number = 4

    Returns Promise<number[]>

cpu

detachFromCurrentScope

  • detachFromCurrentScope<T>(obj: T): T

  • Detach the object from the current scope so it won't be released via auto-release during endscope.

    User needs to either explicitly call obj.dispose(), or attachToCurrentScope to re-attach to the current scope.

    This function can be used to return values to the parent scope.

    Type parameters

    Parameters

    • obj: T

      The object.

    Returns T

device

  • device(deviceType: number | string, deviceId?: number): DLDevice

  • Create a new DLDevice

    Parameters

    • deviceType: number | string

      The device type.

    • Default value deviceId: number = 0

      The device index.

    Returns DLDevice

    The created device.

dispose

  • dispose(): void

empty

  • Create an empty NDArray with given shape and dtype.

    Parameters

    • shape: Array<number> | number

      The shape of the array.

    • Default value dtype: string | DLDataType = "float32"

      The data type of the array.

    • Default value dev: DLDevice = this.device("cpu", 0)

      The device of the ndarray.

    Returns NDArray

    The created ndarray.

endScope

  • endScope(): void

  • End a scope and release all created TVM objects under the current scope.

    Exception: one can call retainToParentScope to move a value to parent scope.

    Returns void

getGlobalFunc

  • Get global PackedFunc from the runtime.

    Parameters

    • name: string

      The name of the function.

    Returns PackedFunc

    The result function.

initWebGPU

  • initWebGPU(device: GPUDevice): void

  • Initialize webgpu in the runtime.

    Parameters

    • device: GPUDevice

      The given GPU device.

    Returns void

isPackedFunc

  • isPackedFunc(func: unknown): boolean

  • Check if func is PackedFunc.

    Parameters

    • func: unknown

      The input.

    Returns boolean

    The check result.

listGlobalFuncNames

  • listGlobalFuncNames(): Array<string>

  • List all the global function names registered in the runtime.

    Returns Array<string>

    The name list.

makeTVMArray

  • Create an tuple TVMArray input array.

    The input array can be passed to tvm runtime function and needs to b explicitly disposed.

    Parameters

    Returns TVMArray

    The result array.

moveToParentScope

  • moveToParentScope<T>(obj: T): T

  • Move obj's attachment to the parent scope.

    This function is useful to make sure objects are still alive when exit the current scope.

    Type parameters

    Parameters

    • obj: T

      The object to be moved.

    Returns T

    The input obj.

registerAsyncServerFunc

  • registerAsyncServerFunc(name: string, func: Function, override?: boolean): void

  • Register an asyncfunction to be global function in the server.

    note

    The async function will only be used for serving remote calls in the rpc.

    Parameters

    • name: string

      The name of the function.

    • func: Function

      function to be registered.

    • Default value override: boolean = false

      Whether overwrite function in existing registry.

    Returns void

registerFunc

  • registerFunc(name: string, func: PackedFunc | Function, override?: boolean): void

  • Register function to be global function in tvm runtime.

    Parameters

    • name: string

      The name of the function.

    • func: PackedFunc | Function
    • Default value override: boolean = false

      Whether overwrite function in existing registry.

    Returns void

registerObjectConstructor

  • registerObjectConstructor(typeKey: string, func: FObjectConstructor, override?: boolean): void

  • Register an object constructor.

    Parameters

    • typeKey: string

      The name of the function.

    • func: FObjectConstructor

      function to be registered.

    • Default value override: boolean = false

      Whether overwrite function in existing registry.

    Returns void

scalar

  • scalar(value: number, dtype: string): Scalar

  • Create a new Scalar that can be passed to a PackedFunc.

    Parameters

    • value: number

      The number value.

    • dtype: string

      The dtype string.

    Returns Scalar

    The created scalar.

setPackedArguments

  • Set packed function arguments into the location indicated by argsValue and argsCode. Allocate new temporary space from the stack if necessary.

    parma

    stack The call stack

    Parameters

    Returns void

systemLib

  • Get system-wide library module in the wasm. System lib is a global module that contains self register functions in startup.

    Returns Module

    The system library module.

toDLDataType

toPackedFunc

  • Convert func to PackedFunc

    Parameters

    • func: Function

      Input function.

    Returns PackedFunc

    The converted function.

typeKey2Index

  • typeKey2Index(typeKey: string): number

  • Get type index from type key.

    Parameters

    • typeKey: string

      The type key.

    Returns number

    The corresponding type index.

webgpu

withNewScope

  • withNewScope<T>(action: () => T): T

  • Perform action under a new scope.

    note

    For action to return a valid value, we will need to call {@link retainToParentScope} for the objects that are created in the scope.

    Type parameters

    • T

    Parameters

    • action: () => T

      The action function.

        • (): T

        • Returns T

    Returns T

    The result value.

Generated using TypeDoc