tvm
Public Member Functions | Static Public Member Functions | Public Attributes | Static Public Attributes | Friends | List of all members
tvm::VirtualDeviceNode Class Reference

Describes at compile time the constraints on where data is to be stored at runtime down to the (virtual) device and memory scope level, and how to compile code to compute that data. Used by the PlanDevices pass to collect and solve (virtual) device constraints for the whole Relax program. More...

#include <virtual_device.h>

Inheritance diagram for tvm::VirtualDeviceNode:
Collaboration diagram for tvm::VirtualDeviceNode:

Public Member Functions

DLDeviceType device_type () const
 
bool IsFullyUnconstrained () const
 Returns true if virtual device is 'fully unconstrained', ie no target/device type, device id or memory scope is specified. More...
 
bool IsFullyConstrained () const
 Returns true if virtual device is 'fully constrained', ie target, device id and memory scope are all specified. More...
 
Device ToDevice () const
 Returns the (virtual) Device implied by this VirtualDevice. Both the device_type and virtual_device_must be constrained. The returned Device may not correspond to any physical device available at compile time or even runtime: see "Virtual vs Physical Devices" above. More...
 
 TVM_FFI_DECLARE_FINAL_OBJECT_INFO (VirtualDeviceNode, BaseAttrsNode)
 
- Public Member Functions inherited from tvm::AttrsNodeReflAdapter< VirtualDeviceNode >
void InitByPackedArgs (const ffi::PackedArgs &args, bool allow_unknown) final
 Initialize the attributes by arguments. More...
 
- Public Member Functions inherited from tvm::BaseAttrsNode
virtual ~BaseAttrsNode ()
 virtual destructor More...
 
template<typename... Args>
void InitBySeq (Args &&... args)
 Initialize the attributes by sequence of arguments. More...
 
 TVM_DECLARE_BASE_OBJECT_INFO (BaseAttrsNode, Object)
 

Static Public Member Functions

static void RegisterReflection ()
 

Public Attributes

int virtual_device_id
 The device identifier for the virtual device. This must be resolved to a physical device identifier either during compilation or at runtime. More...
 
Target target
 The Target describing how to compile for the virtual device. More...
 
MemoryScope memory_scope
 The scope of memory w.r.t. the virtual device which holds data. More...
 

Static Public Attributes

static constexpr const char * _type_key = "target.VirtualDevice"
 
- Static Public Attributes inherited from tvm::BaseAttrsNode
static constexpr TVMFFISEqHashKind _type_s_eq_hash_kind = kTVMFFISEqHashKindTreeNode
 
static constexpr const char * _type_key = "ir.Attrs"
 

Friends

class VirtualDevice
 

Detailed Description

Describes at compile time the constraints on where data is to be stored at runtime down to the (virtual) device and memory scope level, and how to compile code to compute that data. Used by the PlanDevices pass to collect and solve (virtual) device constraints for the whole Relax program.

Is a quadruple of:

Some or all of these fields may be unconstrained, signaling that device planning is free to choose a value consistent with the whole program. However if a target is given then the device_type must equal target->GetTargetDeviceType().

Note that currently we assume if a function returns its result on a particular (virtual) device then the function body is also executed on that device.

By 'execution' we include both (fused) primitive operators, and all the Relax expressions surrounding them which coordinates data and control flow. Again, typically non-primitive operators must be executed on a 'CPU'-like device with good support for control flow.

Since TVM targets such a wide range of systems it is not possible for VirtualDevice to impose much semantics on these fields, particularly for virtual_device_id and memory_scope. Instead we assume downstream passes and codegen will interpret an validate these fields appropriately.

Targets vs Devices

Generally Targets (a compile-time only datastructue) describe compiler options for a specific microarchitecture and toolchain, while Devices (a runtime datastructure also available at compile time) describe a physical device on the target system. Obviously the target must agree with the device's microarchitecture, but we otherwise don't impose any constraints between them:

Traditionally TVM assumes at most one Target per DLDeviceType. We are moving away from that assumption.

Virtual vs Physical Devices

The virtual_device_id may be used by downstream passes or the runtime to help decide which device_id to use for a particular physical runtime Device. For example:

The virtual_device_id may also be left unconstrained if not needed.

Memory Scopes and Devices

Multi-device systems can have complex memory hierarchies. For example

(kDLCPU, 0, "llvm", "global")

and

(kDLCPU, 1, "llvm", "global")

could denote:

Similarly:

(kDLCPU, 0, "llvm", "global")

and

(kDLCUDA, 0, "cuda", "host")

could denote the same memory area, but with very different access costs.

Furthermore, not all memory scopes are accessible to all devices, and it is possible for a memory scope to only be accessible to a device when code is compiled with particular Target options.

VirtualDevices themselves have no system-level understanding. Currently the PlanDevices pass will simply insert "device_copy" operators wherever VirtualDevices are not exactly pointwise equal. We may revisit this in the future as the work on memory pools matures.

Joining and Defaulting

It is possible to 'join' two VirtualDevices to yield the most constrained VirtualDevice which agrees with both join arguments. Eg:

Join((kDLCPU, -1, "llvm", ""), (kInvalidDeviceType, 3, null, "global))
=> (kDLCPU, 3, "llvm", "global")
Join((kDLCPU, -1, "llvm", ""), (kInvalidDeviceType, 3, null, "local))
=> null (no join possible)
constexpr int kInvalidDeviceType
Definition: virtual_device.h:55

Related to 'join' is 'default', which only takes constrained fields from the rhs when the lhs is unconstrained:

Default(kDLCPU, -1, "llvm", "local"), (kDLCPU, 3, null, "global"))
=> (kDLCPU, 3, "llvm", "local")

These operations are needed during device planning.

Member Function Documentation

◆ device_type()

DLDeviceType tvm::VirtualDeviceNode::device_type ( ) const
inline

◆ IsFullyConstrained()

bool tvm::VirtualDeviceNode::IsFullyConstrained ( ) const
inline

Returns true if virtual device is 'fully constrained', ie target, device id and memory scope are all specified.

◆ IsFullyUnconstrained()

bool tvm::VirtualDeviceNode::IsFullyUnconstrained ( ) const
inline

Returns true if virtual device is 'fully unconstrained', ie no target/device type, device id or memory scope is specified.

◆ RegisterReflection()

static void tvm::VirtualDeviceNode::RegisterReflection ( )
inlinestatic

◆ ToDevice()

Device tvm::VirtualDeviceNode::ToDevice ( ) const
inline

Returns the (virtual) Device implied by this VirtualDevice. Both the device_type and virtual_device_must be constrained. The returned Device may not correspond to any physical device available at compile time or even runtime: see "Virtual vs Physical Devices" above.

◆ TVM_FFI_DECLARE_FINAL_OBJECT_INFO()

tvm::VirtualDeviceNode::TVM_FFI_DECLARE_FINAL_OBJECT_INFO ( VirtualDeviceNode  ,
BaseAttrsNode   
)

Friends And Related Function Documentation

◆ VirtualDevice

friend class VirtualDevice
friend

Member Data Documentation

◆ _type_key

constexpr const char* tvm::VirtualDeviceNode::_type_key = "target.VirtualDevice"
staticconstexpr

◆ memory_scope

MemoryScope tvm::VirtualDeviceNode::memory_scope

The scope of memory w.r.t. the virtual device which holds data.

Empty denotes unconstrained.

◆ target

Target tvm::VirtualDeviceNode::target

The Target describing how to compile for the virtual device.

Null denotes unconstrained. Note that if a target later becomes known for this VirtualDevice then it must be consistent with the device_type if already known. This is enforced by the Join and Default methods.

◆ virtual_device_id

int tvm::VirtualDeviceNode::virtual_device_id

The device identifier for the virtual device. This must be resolved to a physical device identifier either during compilation or at runtime.

-1 denotes unconstrained.


The documentation for this class was generated from the following file: