tvm.contrib.graph_runtime

Minimum graph runtime that executes graph containing TVM PackedFunc.

tvm.contrib.graph_runtime.create(graph_json_str, libmod, ctx)

Create a runtime executor module given a graph and module.

Parameters
  • graph_json_str (str) – The graph to be deployed in json format output by json graph. The graph can contain operator(tvm_op) that points to the name of PackedFunc in the libmod.

  • libmod (tvm.runtime.Module) – The module of the corresponding function

  • ctx (TVMContext or list of TVMContext) – The context to deploy the module. It can be local or remote when there is only one TVMContext. Otherwise, the first context in the list will be used as this purpose. All context should be given for heterogeneous execution.

Returns

graph_module – Runtime graph module that can be used to execute the graph.

Return type

GraphModule

tvm.contrib.graph_runtime.get_device_ctx(libmod, ctx)

Parse and validate all the device context(s).

Parameters
Returns

  • ctx (list of TVMContext)

  • num_rpc_ctx (Number of rpc contexts)

  • device_type_id (List of device type and device id)

class tvm.contrib.graph_runtime.GraphModule(module)

Wrapper runtime module.

This is a thin wrapper of the underlying TVM module. you can also directly call set_input, run, and get_output of underlying module functions

Parameters

module (tvm.runtime.Module) – The internal tvm module that holds the actual graph functions.

module

The internal tvm module that holds the actual graph functions.

Type

tvm.runtime.Module

set_input(key=None, value=None, **params)

Set inputs to the module via kwargs

Parameters
  • key (int or str) – The input key

  • value (the input value.) – The input key

  • params (dict of str to NDArray) – Additional arguments

run(**input_dict)

Run forward execution of the graph

Parameters

input_dict (dict of str to NDArray) – List of input values to be feed to

get_num_outputs()

Get the number of outputs from the graph

Returns

count – The number of outputs.

Return type

int

get_num_inputs()

Get the number of inputs to the graph

Returns

count – The number of inputs.

Return type

int

get_input(index, out=None)

Get index-th input to out

Parameters
  • index (int) – The input index

  • out (NDArray) – The output array container

get_output(index, out=None)

Get index-th output to out

Parameters
  • index (int) – The output index

  • out (NDArray) – The output array container

debug_get_output(node, out)

Run graph up to node and get the output to out

Parameters
  • node (int / str) – The node index or name

  • out (NDArray) – The output array container

load_params(params_bytes)

Load parameters from serialized byte array of parameter dict.

Parameters

params_bytes (bytearray) – The serialized parameter dict.

share_params(other, params_bytes)

Share parameters from pre-existing GraphRuntime instance.

Parameters
  • other (GraphRuntime) – The parent GraphRuntime from which this instance should share it’s parameters.

  • params_bytes (bytearray) – The serialized parameter dict (used only for the parameter names).