tvm::CompilationConfigNode Class Reference

Gathers the Targets and distinguished VirtualDevices in canonical form needed to compile a Relay module for execution over possibly heterogeneous devices. Centralizes the validation and canonicalization logic needed to transition from targets supplied by the Python APIs to a single internal representation. Also holds a cache of canonical VirtualDevices so that structural equal virtual devices have pointer equal canonical virtual devices. More...

#include <compilation_config.h>

Inheritance diagram for tvm::CompilationConfigNode:

Collaboration diagram for tvm::CompilationConfigNode:

Public Member Functions
void	VisitAttrs (AttrVisitor *v)
Target	FindPrimitiveTargetForDeviceOrFail (DLDeviceType device_type) const
	Returns the unique Target to use for device_type. Fail if no such target exists. More...
Optional< Target >	FindPrimitiveTargetForKind (const std::string &kind_name) const
	Returns the unique Target to use for kind_name. Returns null if none such. More...
Target	CanonicalTarget (const Target &target) const
	Returns a Target structurally equal to target, however prefer a structually equal known host or primitive target if the configuration has one. More...
VirtualDevice	CanonicalVirtualDevice (const VirtualDevice &virtual_device) const
	Returns a VirtualDevice which is structurally equal to virtual_device on all its constrained fields, however: More...
Public Member Functions inherited from tvm::runtime::Object
uint32_t	type_index () const
std::string	GetTypeKey () const
size_t	GetTypeKeyHash () const
template<typename TargetType >
bool	IsInstance () const
bool	unique () const
	Object ()
	Object (const Object &other)
	Object (Object &&other)
Object &	operator= (const Object &other)
Object &	operator= (Object &&other)

Public Attributes
Target	host_target
	The host target. Used for 'scalar' data and code (such as shapes and shape functions) and residual Relay expressions and data (such as conditionals and ADTs). Each primitive_target below will have this exact target object as its 'host'. More...
Array< Target >	primitive_targets
	Vector of all available Targets for partitioning or compiling primitive tensor operators (kernels). May contain a Target for the same device type as for the host_target, however the host_target should be used for all host computations and data. Each Target will have host_target as its 'host'. More...
VirtualDevice	default_primitive_virtual_device = VirtualDevice::FullyUnconstrained()
	VirtualDevice for primitive operators which are not otherwise constrained to a particular device. Used by the PlanDevices pass to determine a virtual device for every sub-expression. More...
VirtualDevice	host_virtual_device = VirtualDevice::FullyUnconstrained()
	VirtualDevice for the host. More...
Target	optional_homogeneous_target
	If defined then compile and/or run in 'homogenous execution mode'. In this mode all primitives are compiled for this target only. More...

Static Public Attributes
static constexpr const char *	_type_key = "CompilationConfig"
Static Public Attributes inherited from tvm::runtime::Object
static constexpr const char *	_type_key = "runtime.Object"
static constexpr bool	_type_final = false
static constexpr uint32_t	_type_child_slots = 0
static constexpr bool	_type_child_slots_can_overflow = true
static constexpr bool	_type_has_method_visit_attrs = true
static constexpr bool	_type_has_method_sequal_reduce = false
static constexpr bool	_type_has_method_shash_reduce = false
static constexpr uint32_t	_type_index = TypeIndex::kDynamic

Friends
class	CompilationConfig

Additional Inherited Members
Public Types inherited from tvm::runtime::Object
typedef void(*	FDeleter) (Object *self)
	Object deleter. More...
using	RefCounterType = std::atomic< int32_t >
Static Public Member Functions inherited from tvm::runtime::Object
static std::string	TypeIndex2Key (uint32_t tindex)
	Get the type key of the corresponding index from runtime. More...
static size_t	TypeIndex2KeyHash (uint32_t tindex)
	Get the type key hash of the corresponding index from runtime. More...
static uint32_t	TypeKey2Index (const std::string &key)
	Get the type index of the corresponding key from runtime. More...
static uint32_t	_GetOrAllocRuntimeTypeIndex ()
static uint32_t	RuntimeTypeIndex ()
Protected Member Functions inherited from tvm::runtime::Object
void	IncRef ()
	developer function, increases reference counter. More...
void	DecRef ()
	developer function, decrease reference counter. More...
Static Protected Member Functions inherited from tvm::runtime::Object
static uint32_t	GetOrAllocRuntimeTypeIndex (const std::string &key, uint32_t static_tindex, uint32_t parent_tindex, uint32_t type_child_slots, bool type_child_slots_can_overflow)
	Get the type index using type key. More...
Protected Attributes inherited from tvm::runtime::Object
uint32_t	type_index_ {0}
	Type index(tag) that indicates the type of the object. More...
RefCounterType	ref_counter_ {0}
	The internal reference counter. More...
FDeleter	deleter_ = nullptr
	deleter of this object to enable customized allocation. If the deleter is nullptr, no deletion will be performed. The creator of the object must always set the deleter field properly. More...

Detailed Description

Gathers the Targets and distinguished VirtualDevices in canonical form needed to compile a Relay module for execution over possibly heterogeneous devices. Centralizes the validation and canonicalization logic needed to transition from targets supplied by the Python APIs to a single internal representation. Also holds a cache of canonical VirtualDevices so that structural equal virtual devices have pointer equal canonical virtual devices.

The construction of CompilationConfig is idempotent, in that given the same PassContext ctx and an arbitrary Array<Target> raw_targets:

CompilationConfig(ctxt, raw_targets)
   is structurally equal to
CompilationConfig(ctxt, CompilationConfig(ctxt, raw_targets)->primitive_targets)

TODO(mbs): This is subject to change as we rework compilation options in general. This class is probably better called a 'CompositeTarget', and may be better made a sub-class of Target or some other common-target-root class.

Member Function Documentation

CanonicalTarget()

Target tvm::CompilationConfigNode::CanonicalTarget

(

const Target &

target

)

const

Returns a Target structurally equal to target, however prefer a structually equal known host or primitive target if the configuration has one.

CanonicalVirtualDevice()

VirtualDevice tvm::CompilationConfigNode::CanonicalVirtualDevice

(

const VirtualDevice &

virtual_device

)

const

Returns a VirtualDevice which is structurally equal to virtual_device on all its constrained fields, however:

• If virtual_device has a device type but not a target, fill in a target using FindPrimitiveTargetOrFail. This is the one place we allow targets to be defaulted from device types alone.
• If virtual_device has a target, also canonicalize it using CanonicalTarget. The returned object will be unique for the adjusted virtual device w.r.t. all other VirtualDevices returned by this method.

We call the result the 'canonical' VirtualDevice. Two canonical VirtualDevices are structurally equal if and only if they are pointer equal. In this way we can build maps from virtual devices using just pointer equality.

FindPrimitiveTargetForDeviceOrFail()

Target tvm::CompilationConfigNode::FindPrimitiveTargetForDeviceOrFail

(

DLDeviceType

device_type

)

const

Returns the unique Target to use for device_type. Fail if no such target exists.

This will be the first primitive target with matching device type.

FindPrimitiveTargetForKind()

Optional<Target> tvm::CompilationConfigNode::FindPrimitiveTargetForKind

(

const std::string &

kind_name

)

const

Returns the unique Target to use for kind_name. Returns null if none such.

VisitAttrs()

void tvm::CompilationConfigNode::VisitAttrs

(

AttrVisitor *

v

)

Friends And Related Function Documentation

CompilationConfig

friend class CompilationConfig

friend

Member Data Documentation

_type_key

constexpr const char* tvm::CompilationConfigNode::_type_key = "CompilationConfig"

staticconstexpr

default_primitive_virtual_device

VirtualDevice tvm::CompilationConfigNode::default_primitive_virtual_device = VirtualDevice::FullyUnconstrained()

VirtualDevice for primitive operators which are not otherwise constrained to a particular device. Used by the PlanDevices pass to determine a virtual device for every sub-expression.

host_target

Target tvm::CompilationConfigNode::host_target

The host target. Used for 'scalar' data and code (such as shapes and shape functions) and residual Relay expressions and data (such as conditionals and ADTs). Each primitive_target below will have this exact target object as its 'host'.

Note that it is possible for a Target used for primitive operations to be structurally equal to the host Target (up to the host field.) However the Target objects will be distinct, and can be used as keys within a Map without collision.

host_virtual_device

VirtualDevice tvm::CompilationConfigNode::host_virtual_device = VirtualDevice::FullyUnconstrained()

VirtualDevice for the host.

optional_homogeneous_target

Target tvm::CompilationConfigNode::optional_homogeneous_target

If defined then compile and/or run in 'homogenous execution mode'. In this mode all primitives are compiled for this target only.

This is to support legacy passes which have not been adapted to heterogeneous execution and rely on an implicit global Target to be in scope.

TODO(mbs): Remove once all passes are 'heterogeneous aware'.

primitive_targets

Array<Target> tvm::CompilationConfigNode::primitive_targets

Vector of all available Targets for partitioning or compiling primitive tensor operators (kernels). May contain a Target for the same device type as for the host_target, however the host_target should be used for all host computations and data. Each Target will have host_target as its 'host'.

Primitive targets must be unique by their kind name. In this way the FindPrimitiveTargetForKind method will find the unique target for the given kind name. This method is used when transitioning from an external codegen "Compiler" attribute value to the external codegen target representing that compiler.

It is possible to have multiple primitive targets for the same device type. However given primitive targets left and right where:

• left appears before right in the array
• left->GetTargetDeviceType() == right->GetTargetDeviceType() then:
• right.IsExternalCodegenFor(left) must be true In this way the FindPrimitiveTargetForDeviceOrFail method will find the 'most general' target for the requested device type. This method is used when transitioning from a device constraint to the target needed to compile for that device.

In the homogeneous case primitive_targets will have just one entry, which will be pointer equal to optional_homogeneous_target.

In the homogenous case where the 'host' is the same device as used for compiling kernels it is not the case that optional_homogenous_target == host_target. This is because all primitive always have their host field set to the host_target. Ie, it is valid to have:

host_target=Target("llvm")
optional_homogenous_target=Target("llvm", host=host_target)

---

The documentation for this class was generated from the following file:

• include/tvm/target/compilation_config.h