Attributes for the "on_device" annotation (ie operator). More...

#include <on_device.h>

Inheritance diagram for tvm::relay::OnDeviceAttrs:

Collaboration diagram for tvm::relay::OnDeviceAttrs:

Public Member Functions
bool	is_fixed () const
	Returns true if both the body and result are constrained. More...

bool	is_normal () const
	Returns true only the body is constrained (the 'normal' case). More...

	TVM_DECLARE_ATTRS (OnDeviceAttrs, "relay.attrs.OnDeviceAttrs")

Public Member Functions inherited from tvm::AttrsNode< OnDeviceAttrs >
void	VisitAttrs (AttrVisitor *v)

void	VisitNonDefaultAttrs (AttrVisitor *v)
	Visit attributes that do not equal the default value. More...

void	InitByPackedArgs (const runtime::TVMArgs &args, bool allow_unknown) final
	Initialize the attributes by arguments. More...

bool	SEqualReduce (const OnDeviceAttrs *other, SEqualReducer equal) const

void	SHashReduce (SHashReducer hash_reducer) const

Array< AttrFieldInfo >	ListFieldInfo () const final
	Get the field information. More...

Public Member Functions inherited from tvm::BaseAttrsNode
virtual	~BaseAttrsNode ()
	virtual destructor More...

template<typename... Args>
void	InitBySeq (Args &&... args)
	Initialize the attributes by sequence of arguments. More...

void	PrintDocString (std::ostream &os) const
	Print readible docstring to ostream, add newline. More...

	TVM_DECLARE_BASE_OBJECT_INFO (BaseAttrsNode, Object)

Public Member Functions inherited from tvm::runtime::Object
uint32_t	type_index () const

std::string	GetTypeKey () const

size_t	GetTypeKeyHash () const

template<typename TargetType >
bool	IsInstance () const

bool	unique () const

	Object ()

	Object (const Object &other)

	Object (Object &&other)

Object &	operator= (const Object &other)

Object &	operator= (Object &&other)

Public Attributes
VirtualDevice	virtual_device = VirtualDevice::FullyUnconstrained()
	The `VirtualDevice` to constraint to apply to the body, result, or both body and result of the "on_device" call. More...

bool	constrain_result = false
	If false (the default), the result of the "on_device" call is not constrained to be `virtual_device`. More...

bool	constrain_body = true
	If true (the default), the body of the "on_device" call is constrained to be `virtual_device`. More...

Additional Inherited Members
Public Types inherited from tvm::BaseAttrsNode
using	TVMArgs = runtime::TVMArgs

using	TVMRetValue = runtime::TVMRetValue

Public Types inherited from tvm::runtime::Object
typedef void(*	FDeleter) (Object *self)
	Object deleter. More...

using	RefCounterType = std::atomic< int32_t >

Static Public Member Functions inherited from tvm::runtime::Object
static std::string	TypeIndex2Key (uint32_t tindex)
	Get the type key of the corresponding index from runtime. More...

static size_t	TypeIndex2KeyHash (uint32_t tindex)
	Get the type key hash of the corresponding index from runtime. More...

static uint32_t	TypeKey2Index (const std::string &key)
	Get the type index of the corresponding key from runtime. More...

static uint32_t	_GetOrAllocRuntimeTypeIndex ()

static uint32_t	RuntimeTypeIndex ()

Static Public Attributes inherited from tvm::BaseAttrsNode
static constexpr const bool	_type_has_method_sequal_reduce = true

static constexpr const bool	_type_has_method_shash_reduce = true

static constexpr const char *	_type_key = "Attrs"

Static Public Attributes inherited from tvm::runtime::Object
static constexpr const char *	_type_key = "runtime.Object"

static constexpr bool	_type_final = false

static constexpr uint32_t	_type_child_slots = 0

static constexpr bool	_type_child_slots_can_overflow = true

static constexpr bool	_type_has_method_visit_attrs = true

static constexpr bool	_type_has_method_sequal_reduce = false

static constexpr bool	_type_has_method_shash_reduce = false

static constexpr uint32_t	_type_index = TypeIndex::kDynamic

Protected Member Functions inherited from tvm::runtime::Object
void	IncRef ()
	developer function, increases reference counter. More...

void	DecRef ()
	developer function, decrease reference counter. More...

Static Protected Member Functions inherited from tvm::runtime::Object
static uint32_t	GetOrAllocRuntimeTypeIndex (const std::string &key, uint32_t static_tindex, uint32_t parent_tindex, uint32_t type_child_slots, bool type_child_slots_can_overflow)
	Get the type index using type key. More...

Protected Attributes inherited from tvm::runtime::Object
uint32_t	type_index_ {0}
	Type index(tag) that indicates the type of the object. More...

RefCounterType	ref_counter_ {0}
	The internal reference counter. More...

FDeleter	deleter_ = nullptr
	deleter of this object to enable customized allocation. If the deleter is nullptr, no deletion will be performed. The creator of the object must always set the deleter field properly. More...

Detailed Description

Attributes for the "on_device" annotation (ie operator).

The Relay call:

on_device(sub_expr, virtual_device=S)

tvm::relay::OnDeviceAttrs::virtual_device

VirtualDevice virtual_device

The VirtualDevice to constraint to apply to the body, result, or both body and result of the "on_devi...

Definition: on_device.h:66

constrains sub_expr to execute and store its result on the VirtualDevice S. However the annotation itself may appear in an expression to be executed and stored on a different VirtualDevice. If so the compiler will automatically insert a "device_copy" call to mediate the transition between VirtualDevices.

E.g.: Assuming x and y reside on the GPU and z on the CPU then:

multiply(on_device(add(%x, %y), virtual_device=GPU), %z)

tvm::topi::multiply

tvm::PrimExpr multiply(const tvm::PrimExpr &a, const tvm::PrimExpr &b)

Definition: broadcast.h:225

tvm::add

PrimExpr add(PrimExpr a, PrimExpr b, Span span=Span())

add operator

indicates the add should execute on the GPU but the multiply should execute on the CPU. The compiler will rewrite this to:

multiply(device_copy(add(%x, %y), src_virtual_device=GPU, dst_virtual_device=CPU), %z)

The constraint_body (default true) and constraint_result (default false) fields can be used by passes for finer-grained control over how the VirtualDevice constraint should be applied.

Member Function Documentation

◆ is_fixed()

bool tvm::relay::OnDeviceAttrs::is_fixed ( ) const

inline

Returns true if both the body and result are constrained.

◆ is_normal()

bool tvm::relay::OnDeviceAttrs::is_normal ( ) const

inline

Returns true only the body is constrained (the 'normal' case).

◆ TVM_DECLARE_ATTRS()

tvm::relay::OnDeviceAttrs::TVM_DECLARE_ATTRS	(	OnDeviceAttrs	,
		"relay.attrs.OnDeviceAttrs"
	)

inline

Member Data Documentation

◆ constrain_body

bool tvm::relay::OnDeviceAttrs::constrain_body = true

If true (the default), the body of the "on_device" call is constrained to be virtual_device.

◆ constrain_result

bool tvm::relay::OnDeviceAttrs::constrain_result = false

If false (the default), the result of the "on_device" call is not constrained to be virtual_device.

◆ virtual_device

VirtualDevice tvm::relay::OnDeviceAttrs::virtual_device = VirtualDevice::FullyUnconstrained()

The VirtualDevice to constraint to apply to the body, result, or both body and result of the "on_device" call.

The documentation for this struct was generated from the following file:

include/tvm/relay/attrs/on_device.h

Public Member Functions

Public Attributes

Additional Inherited Members

Detailed Description

Member Function Documentation

◆ is_fixed()

◆ is_normal()

◆ TVM_DECLARE_ATTRS()

Member Data Documentation

◆ constrain_body

◆ constrain_result

◆ virtual_device