tvm_tirx.runtime

tvm_tirx.runtime#

Runtime tensors, devices, executable modules, and thread information.

tvm_tirx.runtime.enabled(target)#

Whether module runtime is enabled for target

Parameters:

target (str or Dict[str, Any] or tvm_tirx.target.Target) – The target device type.

Returns:

enabled – Whether runtime is enabled.

Return type:

bool

Examples

The following code checks if gpu is enabled.

>>> tvm_tirx.runtime.enabled("gpu")
tvm_tirx.runtime.num_threads() → int#

Get the number of threads in use by the TVM runtime.

Returns:

Number of threads in use.

Return type:

int

tvm.runtime#

Benchmarking belongs to the parent runtime. Use tvm.runtime.time_evaluator(module, func_name, device, ...) with an ordinary tvm_ffi.Module and either a local device or an RPC remote device. The returned callable reports a BenchmarkResult containing the repeated measurements.

tvm.runtime.time_evaluator(module, func_name, device, number=10, repeat=1, min_repeat_ms=0, limit_zero_time_iterations=100, cooldown_interval_ms=0, repeats_to_cooldown=1, cache_flush_bytes=0, f_preproc='')#

Get an evaluator that measures time cost of running function.

Parameters:

  • module (tvm_ffi.Module) – The module containing the function to benchmark.

  • func_name (str) – The name of the function in the module.

  • device (Device or tvm.rpc.RemoteDevice) – The device we should run this function on.

  • number (int) – The number of times to run this function for taking average. We call these runs as one repeat of measurement.

  • repeat (int, optional) – The number of times to repeat the measurement. In total, the function will be invoked (1 + number x repeat) times, where the first one is warm up and will be discarded. The returned result contains repeat costs, each of which is an average of number costs.

  • min_repeat_ms (int, optional) – The minimum duration of one repeat in milliseconds. By default, one repeat contains number runs. If this parameter is set, the parameters number will be dynamically adjusted to meet the minimum duration requirement of one repeat. i.e., When the run time of one repeat falls below this time, the number parameter will be automatically increased.

  • limit_zero_time_iterations (int, optional) – The maximum number of repeats when measured time is equal to 0. It helps to avoid hanging during measurements.

  • cooldown_interval_ms (int, optional) – The cooldown interval in milliseconds between the number of repeats defined by repeats_to_cooldown.

  • repeats_to_cooldown (int, optional) – The number of repeats before the cooldown is activated.

  • cache_flush_bytes (int, optional) – The number of bytes to flush from the cache before each repeat.

  • f_preproc (str, optional) – The preprocess function name we want to execute before executing the time evaluator.

Note

The function will be invoked (1 + number x repeat) times, with the first call discarded in case there is lazy initialization.

Returns:

ftimer – The function that takes same argument as func and returns a BenchmarkResult. The ProfileResult reports repeat time costs in seconds.

Return type:

function

class tvm.runtime.BenchmarkResult(results: Sequence[float])#

Runtimes from benchmarking

Contents