tvm::runtime::cuda_ipc::CUDAIPCMemoryObj Class Reference

The CUDA IPC (interprocess communication) memory object, which internally contains data pointers to CUDA IPC memory. It is be useful for efficient all-reduce implementation. More...

#include <cuda_ipc_memory.h>

Inheritance diagram for tvm::runtime::cuda_ipc::CUDAIPCMemoryObj:

Collaboration diagram for tvm::runtime::cuda_ipc::CUDAIPCMemoryObj:

Public Member Functions
	TVM_DECLARE_BASE_OBJECT_INFO (CUDAIPCMemoryObj, Object)
Public Member Functions inherited from tvm::runtime::Object
uint32_t	type_index () const
std::string	GetTypeKey () const
size_t	GetTypeKeyHash () const
template<typename TargetType >
bool	IsInstance () const
bool	unique () const
	Object ()
	Object (const Object &other)
	Object (Object &&other)
Object &	operator= (const Object &other)
Object &	operator= (Object &&other)

Public Attributes
int	num_workers
	The number of GPU workers. More...
int	worker_id
	The worker id corresponding to this IPC memory object. More...
std::vector< void * >	remote_data
	The data pointers of all all-reduce inputs. It has "num_workers" pointers. The i-th pointer is the data pointer on worker i. If "i != worker_id", the pointer is an IPC data pointer. Otherwise, the pointer is a local CUDA data pointer. More...
std::vector< void * >	barrier_in
	The pointers to input barrier signals of all workers for all-reduce. It has "num_workers" pointers, and the pointer arrangement is the same as "remote_data". More...
std::vector< void * >	barrier_out
	The pointers to output barrier signals of all workers for all-reduce. It has "num_workers" pointers, and the pointer arrangement is the same as "remote_data". More...
int	barrier_flag
	The integer buffer flag for all-reduce. More...

Static Public Attributes
static constexpr const char *	_type_key = "tvm.runtime.disco.cuda_ipc_memory"
static constexpr const bool	_type_has_method_sequal_reduce = false
static constexpr const bool	_type_has_method_shash_reduce = false
Static Public Attributes inherited from tvm::runtime::Object
static constexpr const char *	_type_key = "runtime.Object"
static constexpr bool	_type_final = false
static constexpr uint32_t	_type_child_slots = 0
static constexpr bool	_type_child_slots_can_overflow = true
static constexpr bool	_type_has_method_visit_attrs = true
static constexpr bool	_type_has_method_sequal_reduce = false
static constexpr bool	_type_has_method_shash_reduce = false
static constexpr uint32_t	_type_index = TypeIndex::kDynamic

Additional Inherited Members
Public Types inherited from tvm::runtime::Object
typedef void(*	FDeleter) (Object *self)
	Object deleter. More...
using	RefCounterType = std::atomic< int32_t >
Static Public Member Functions inherited from tvm::runtime::Object
static std::string	TypeIndex2Key (uint32_t tindex)
	Get the type key of the corresponding index from runtime. More...
static size_t	TypeIndex2KeyHash (uint32_t tindex)
	Get the type key hash of the corresponding index from runtime. More...
static uint32_t	TypeKey2Index (const std::string &key)
	Get the type index of the corresponding key from runtime. More...
static uint32_t	_GetOrAllocRuntimeTypeIndex ()
static uint32_t	RuntimeTypeIndex ()
Protected Member Functions inherited from tvm::runtime::Object
void	IncRef ()
	developer function, increases reference counter. More...
void	DecRef ()
	developer function, decrease reference counter. More...
Static Protected Member Functions inherited from tvm::runtime::Object
static uint32_t	GetOrAllocRuntimeTypeIndex (const std::string &key, uint32_t static_tindex, uint32_t parent_tindex, uint32_t type_child_slots, bool type_child_slots_can_overflow)
	Get the type index using type key. More...
Protected Attributes inherited from tvm::runtime::Object
uint32_t	type_index_ {0}
	Type index(tag) that indicates the type of the object. More...
RefCounterType	ref_counter_ {0}
	The internal reference counter. More...
FDeleter	deleter_ = nullptr
	deleter of this object to enable customized allocation. If the deleter is nullptr, no deletion will be performed. The creator of the object must always set the deleter field properly. More...

Detailed Description

The CUDA IPC (interprocess communication) memory object, which internally contains data pointers to CUDA IPC memory. It is be useful for efficient all-reduce implementation.

Note

Right now the class members are closely tied with customized all-reduce kernel. They may also be extended for other uses in the future.

Member Function Documentation

TVM_DECLARE_BASE_OBJECT_INFO()

tvm::runtime::cuda_ipc::CUDAIPCMemoryObj::TVM_DECLARE_BASE_OBJECT_INFO	(	CUDAIPCMemoryObj	,
		Object
	)

Member Data Documentation

_type_has_method_sequal_reduce

constexpr const bool tvm::runtime::cuda_ipc::CUDAIPCMemoryObj::_type_has_method_sequal_reduce = false

staticconstexpr

_type_has_method_shash_reduce

constexpr const bool tvm::runtime::cuda_ipc::CUDAIPCMemoryObj::_type_has_method_shash_reduce = false

staticconstexpr

_type_key

constexpr const char* tvm::runtime::cuda_ipc::CUDAIPCMemoryObj::_type_key = "tvm.runtime.disco.cuda_ipc_memory"

staticconstexpr

barrier_flag

int tvm::runtime::cuda_ipc::CUDAIPCMemoryObj::barrier_flag

The integer buffer flag for all-reduce.

barrier_in

std::vector<void*> tvm::runtime::cuda_ipc::CUDAIPCMemoryObj::barrier_in

The pointers to input barrier signals of all workers for all-reduce. It has "num_workers" pointers, and the pointer arrangement is the same as "remote_data".

barrier_out

std::vector<void*> tvm::runtime::cuda_ipc::CUDAIPCMemoryObj::barrier_out

The pointers to output barrier signals of all workers for all-reduce. It has "num_workers" pointers, and the pointer arrangement is the same as "remote_data".

num_workers

int tvm::runtime::cuda_ipc::CUDAIPCMemoryObj::num_workers

The number of GPU workers.

remote_data

std::vector<void*> tvm::runtime::cuda_ipc::CUDAIPCMemoryObj::remote_data

The data pointers of all all-reduce inputs. It has "num_workers" pointers. The i-th pointer is the data pointer on worker i. If "i != worker_id", the pointer is an IPC data pointer. Otherwise, the pointer is a local CUDA data pointer.

worker_id

int tvm::runtime::cuda_ipc::CUDAIPCMemoryObj::worker_id

The worker id corresponding to this IPC memory object.

---

The documentation for this class was generated from the following file:

• include/tvm/runtime/disco/cuda_ipc_memory.h