tvm.runtime

TVM runtime namespace.

Classes:

DataType(type_str)

TVM datatype structure

DataTypeCode()

DataType code in DLTensor.

Module(handle)

Runtime Module.

Object

Base class for all tvm’s runtime objects.

ObjectGeneric()

Base class for all classes that can be converted to object.

PackedFunc

The PackedFunc object used in TVM.

String(content)

TVM runtime.String object, represented as a python str.

TVMContext(device_type, device_id)

TVM context strucure.

Functions:

cl([dev_id])

Construct a OpenCL device

const(value[, dtype])

construct a constant

context(dev_type[, dev_id])

Construct a TVM context with given device type and id.

convert(value)

Convert value to TVM object or function.

convert_to_object(value)

Convert a Python value to corresponding object type.

cpu([dev_id])

Construct a CPU device

enabled(target)

Whether module runtime is enabled for target

ext_dev([dev_id])

Construct a extension device

gpu([dev_id])

Construct a GPU device

load_module(path[, fmt])

Load module from file.

metal([dev_id])

Construct a metal device

micro_dev([dev_id])

Construct a micro device

mtl([dev_id])

Construct a metal device

opencl([dev_id])

Construct a OpenCL device

rocm([dev_id])

Construct a ROCM device

system_lib()

Get system-wide library module singleton.

vpi([dev_id])

Construct a VPI simulated device

vulkan([dev_id])

Construct a Vulkan device

class tvm.runtime.PackedFunc

The PackedFunc object used in TVM.

Function plays an key role to bridge front and backend in TVM. Function provide a type-erased interface, you can call function with positional arguments.

The compiled module returns Function. TVM backend also registers and exposes its API as Functions.

The following are list of common usage scenario of tvm.runtime.PackedFunc.

  • Automatic exposure of C++ API into python

  • To call PackedFunc from python side

  • To call python callbacks to inspect results in generated code

  • Bring python hook into C++ backend

See also

tvm.register_func

How to register global function.

tvm.get_global_func

How to get global function.

class tvm.runtime.Object

Base class for all tvm’s runtime objects.

class tvm.runtime.ObjectGeneric

Base class for all classes that can be converted to object.

Methods:

asobject()

Convert value to object

asobject()

Convert value to object

class tvm.runtime.DataType(type_str)

TVM datatype structure

class tvm.runtime.DataTypeCode

DataType code in DLTensor.

class tvm.runtime.TVMContext(device_type, device_id)

TVM context strucure.

Attributes:

compute_version

Get compute verison number in string.

device_name

Return the string name of device.

exist

Whether this device exist.

max_clock_rate

Return the max clock frequency of device.

max_shared_memory_per_block

Total amount of shared memory per block in bytes.

max_thread_dimensions

Return the maximum size of each thread axis

max_threads_per_block

Maximum number of threads on each block.

multi_processor_count

Return the number of compute units of device.

warp_size

Number of threads that executes in concurrent.

Methods:

sync()

Synchronize until jobs finished at the context.

property exist

Whether this device exist.

property max_threads_per_block

Maximum number of threads on each block.

property warp_size

Number of threads that executes in concurrent.

property max_shared_memory_per_block

Total amount of shared memory per block in bytes.

property compute_version

Get compute verison number in string.

Currently used to get compute capability of CUDA device.

Returns

version – The version string in major.minor format.

Return type

str

property device_name

Return the string name of device.

property max_clock_rate

Return the max clock frequency of device.

property multi_processor_count

Return the number of compute units of device.

property max_thread_dimensions

Return the maximum size of each thread axis

Returns

dims – The maximum length of threadIdx.x, threadIdx.y, threadIdx.z

Return type

List of int

sync()

Synchronize until jobs finished at the context.

class tvm.runtime.Module(handle)

Runtime Module.

Attributes:

entry_func

Get the entry function

imported_modules

Get imported modules

type_key

Get type key of the module.

Methods:

export_library(file_name[, fcompile, addons])

Export the module and its imported device code one library.

get_function(name[, query_imports])

Get function from the module.

get_source([fmt])

Get source code from module, if available.

import_module(module)

Add module to the import list of current one.

save(file_name[, fmt])

Save the module to file.

time_evaluator(func_name, ctx[, number, …])

Get an evaluator that measures time cost of running function.

property entry_func

Get the entry function

Returns

f – The entry function if exist

Return type

tvm.runtime.PackedFunc

get_function(name, query_imports=False)

Get function from the module.

Parameters
  • name (str) – The name of the function

  • query_imports (bool) – Whether also query modules imported by this module.

Returns

f – The result function.

Return type

tvm.runtime.PackedFunc

import_module(module)

Add module to the import list of current one.

Parameters

module (tvm.runtime.Module) – The other module.

property type_key

Get type key of the module.

get_source(fmt='')

Get source code from module, if available.

Parameters

fmt (str, optional) – The specified format.

Returns

source – The result source code.

Return type

str

property imported_modules

Get imported modules

Returns

modules – The module

Return type

list of Module

save(file_name, fmt='')

Save the module to file.

This do not save the dependent device modules. See also export_shared

Parameters
  • file_name (str) – The name of the file.

  • fmt (str) – The format of the file.

See also

runtime.Module.export_library()

export the module to shared library.

time_evaluator(func_name, ctx, number=10, repeat=1, min_repeat_ms=0, f_preproc='')

Get an evaluator that measures time cost of running function.

Parameters
  • func_name (str) – The name of the function in the module.

  • ctx (TVMContext) – The context we should run this function on.

  • number (int) – The number of times to run this function for taking average. We call these runs as one repeat of measurement.

  • repeat (int, optional) – The number of times to repeat the measurement. In total, the function will be invoked (1 + number x repeat) times, where the first one is warm up and will be discarded. The returned result contains repeat costs, each of which is an average of number costs.

  • min_repeat_ms (int, optional) – The minimum duration of one repeat in milliseconds. By default, one repeat contains number runs. If this parameter is set, the parameters number will be dynamically adjusted to meet the minimum duration requirement of one repeat. i.e., When the run time of one repeat falls below this time, the number parameter will be automatically increased.

  • f_preproc (str, optional) – The preprocess function name we want to execute before executing the time evaluator.

Note

The function will be invoked (1 + number x repeat) times, with the first call discarded in case there is lazy initialization.

Returns

ftimer – The function that takes same argument as func and returns a ProfileResult. The ProfileResult reports repeat time costs in seconds.

Return type

function

export_library(file_name, fcompile=None, addons=None, **kwargs)

Export the module and its imported device code one library.

This function only works on host llvm modules. It will pack all the imported modules

Parameters
  • file_name (str) – The name of the shared library.

  • fcompile (function(target, file_list, kwargs), optional) – Compilation function to use create dynamic library. If fcompile has attribute object_format, will compile host library to that format. Otherwise, will use default format “o”.

  • kwargs (dict, optional) – Additional arguments passed to fcompile

tvm.runtime.convert_to_object(value)

Convert a Python value to corresponding object type.

Parameters

value (str) – The value to be inspected.

Returns

obj – The corresponding object value.

Return type

Object

tvm.runtime.convert(value)

Convert value to TVM object or function.

Parameters

value (python value) –

Returns

tvm_val – Converted value in TVM

Return type

Object or Function

tvm.runtime.const(value, dtype=None)

construct a constant

Parameters
  • value (number) – The content of the constant number.

  • dtype (str or None, optional) – The data type.

Returns

const_val – The result expression.

Return type

tvm.Expr

tvm.runtime.context(dev_type, dev_id=0)

Construct a TVM context with given device type and id.

Parameters
  • dev_type (int or str) – The device type mask or name of the device.

  • dev_id (int, optional) – The integer device id

Returns

ctx – The corresponding context.

Return type

tvm.runtime.TVMContext

Examples

Context can be used to create reflection of context by string representation of the device type.

assert tvm.context("cpu", 1) == tvm.cpu(1)
assert tvm.context("gpu", 0) == tvm.gpu(0)
assert tvm.context("cuda", 0) == tvm.gpu(0)
tvm.runtime.cpu(dev_id=0)

Construct a CPU device

Parameters

dev_id (int, optional) – The integer device id

Returns

ctx – The created context

Return type

TVMContext

tvm.runtime.gpu(dev_id=0)

Construct a GPU device

Parameters

dev_id (int, optional) – The integer device id

Returns

ctx – The created context

Return type

TVMContext

tvm.runtime.opencl(dev_id=0)

Construct a OpenCL device

Parameters

dev_id (int, optional) – The integer device id

Returns

ctx – The created context

Return type

TVMContext

tvm.runtime.cl(dev_id=0)

Construct a OpenCL device

Parameters

dev_id (int, optional) – The integer device id

Returns

ctx – The created context

Return type

TVMContext

tvm.runtime.vulkan(dev_id=0)

Construct a Vulkan device

Parameters

dev_id (int, optional) – The integer device id

Returns

ctx – The created context

Return type

TVMContext

tvm.runtime.metal(dev_id=0)

Construct a metal device

Parameters

dev_id (int, optional) – The integer device id

Returns

ctx – The created context

Return type

TVMContext

tvm.runtime.mtl(dev_id=0)

Construct a metal device

Parameters

dev_id (int, optional) – The integer device id

Returns

ctx – The created context

Return type

TVMContext

tvm.runtime.vpi(dev_id=0)

Construct a VPI simulated device

Parameters

dev_id (int, optional) – The integer device id

Returns

ctx – The created context

Return type

TVMContext

tvm.runtime.rocm(dev_id=0)

Construct a ROCM device

Parameters

dev_id (int, optional) – The integer device id

Returns

ctx – The created context

Return type

TVMContext

tvm.runtime.ext_dev(dev_id=0)

Construct a extension device

Parameters

dev_id (int, optional) – The integer device id

Returns

ctx – The created context

Return type

TVMContext

Note

This API is reserved for quick testing of new device by plugin device API as ext_dev.

tvm.runtime.micro_dev(dev_id=0)

Construct a micro device

Parameters

dev_id (int, optional) – The integer device id

Returns

ctx – The created context

Return type

TVMContext

tvm.runtime.load_module(path, fmt='')

Load module from file.

Parameters
  • path (str) – The path to the module file.

  • fmt (str, optional) – The format of the file, if not specified it will be inferred from suffix of the file.

Returns

module – The loaded module

Return type

runtime.Module

Note

This function will automatically call cc.create_shared if the path is in format .o or .tar

tvm.runtime.enabled(target)

Whether module runtime is enabled for target

Parameters

target (str) – The target device type.

Returns

enabled – Whether runtime is enabled.

Return type

bool

Examples

The following code checks if gpu is enabled.

>>> tvm.runtime.enabled("gpu")
tvm.runtime.system_lib()

Get system-wide library module singleton.

System lib is a global module that contains self register functions in startup. Unlike normal dso modules which need to be loaded explicitly. It is useful in environments where dynamic loading api like dlopen is banned.

To build system lib function, simply specify target option `llvm --system-lib` The system lib will be available as long as the result code is linked by the program.

The system lib is intended to be linked and loaded during the entire life-cyle of the program. If you want dynamic loading features, use dso modules instead.

Returns

module – The system-wide library module.

Return type

runtime.Module

class tvm.runtime.String(content)

TVM runtime.String object, represented as a python str.

Parameters

content (str) – The content string used to construct the object.