Managed NDArray. The array is backed by reference counted blocks. More...

#include <ndarray.h>

Inheritance diagram for tvm::runtime::NDArray:

Collaboration diagram for tvm::runtime::NDArray:

Classes
class	Container
	Object container class that backs NDArray. More...

class	ContainerBase
	The container base structure contains all the fields except for the Object header. More...

Public Types
using	ContainerType = Container
	Container type for Object system. More...

Public Types inherited from tvm::runtime::ObjectRef
using	ContainerType = Object
	type indicate the container type. More...

Public Member Functions
	NDArray ()
	default constructor More...

	NDArray (ObjectPtr< Object > data)
	constructor. More...

void	reset ()
	reset the content of NDArray to be nullptr More...

int	use_count () const

const DLTensor *	operator-> () const

bool	IsContiguous () const

void	CopyFrom (const DLTensor *other)
	Copy data content from another array. More...

void	CopyFrom (const NDArray &other)

void	CopyFromBytes (const void *data, size_t nbytes)
	Copy data content from a byte buffer. More...

void	CopyTo (DLTensor *other) const
	Copy data content into another array. More...

void	CopyTo (const NDArray &other) const

void	CopyToBytes (void *data, size_t nbytes) const
	Copy data content into another array. More...

NDArray	CopyTo (const Device &dev, Optional< String > mem_scope=NullOpt) const
	Copy the data to another device. More...

bool	Load (dmlc::Stream *stream)
	Load NDArray from stream. More...

void	Save (dmlc::Stream *stream) const
	Save NDArray to stream. More...

NDArray	CreateView (ShapeTuple shape, DLDataType dtype, uint64_t relative_byte_offset=0)
	Create a NDArray that shares the data memory with the current one. More...

DLManagedTensor *	ToDLPack () const
	Create a reference view of NDArray that represents as DLManagedTensor. More...

ShapeTuple	Shape () const

runtime::DataType	DataType () const

Public Member Functions inherited from tvm::runtime::ObjectRef
	ObjectRef ()=default
	default constructor More...

	ObjectRef (ObjectPtr< Object > data)
	Constructor from existing object ptr. More...

bool	same_as (const ObjectRef &other) const
	Comparator. More...

bool	operator== (const ObjectRef &other) const
	Comparator. More...

bool	operator!= (const ObjectRef &other) const
	Comparator. More...

bool	operator< (const ObjectRef &other) const
	Comparator. More...

bool	defined () const

const Object *	get () const

const Object *	operator-> () const

bool	unique () const

int	use_count () const

template<typename ObjectType , typename = std::enable_if_t<std::is_base_of_v<Object, ObjectType>>>
const ObjectType *	as () const
	Try to downcast the internal Object to a raw pointer of a corresponding type. More...

template<typename ObjectRefType , typename = std::enable_if_t<std::is_base_of_v<ObjectRef, ObjectRefType>>>
Optional< ObjectRefType >	as () const
	Try to downcast the ObjectRef to a Optional<T> of the requested type. More...

Static Public Member Functions
static NDArray	Empty (ShapeTuple shape, DLDataType dtype, Device dev, Optional< String > mem_scope=NullOpt)
	Create an empty NDArray. More...

static NDArray	FromExternalDLTensor (const DLTensor &dl_tensor)
	Create a NDArray backed by an external DLTensor without memory copying. More...

static NDArray	NewFromDLTensor (DLTensor *dl_tensor, const Device &dev)
	Create new NDArray, data is copied from DLTensor. More...

static NDArray	FromDLPack (DLManagedTensor *tensor)
	Create a NDArray backed by a dlpack tensor. More...

static void	CopyFromTo (const DLTensor from, DLTensor to, TVMStreamHandle stream=nullptr)
	Function to copy data from one array to another. More...

static bool	AbilityOfZeroCopyForDLTensor (DLTensor *tensor, const Device &dev)
	Check conditions for construction NDArray over DLTensor without copying. There are three conditions to check: More...

Protected Member Functions
Container *	get_mutable () const
	Get mutable internal container pointer. More...

Protected Member Functions inherited from tvm::runtime::ObjectRef
Object *	get_mutable () const

Static Protected Member Functions
static ObjectPtr< Object >	FFIDataFromHandle (TVMArrayHandle handle)
	Construct NDArray's Data field from array handle in FFI. More...

static void	FFIDecRef (TVMArrayHandle handle)
	DecRef resource managed by an FFI array handle. More...

static TVMArrayHandle	FFIGetHandle (const ObjectRef &nd)
	Get FFI Array handle from ndarray. More...

Static Protected Member Functions inherited from tvm::runtime::ObjectRef
template<typename T >
static T	DowncastNoCheck (ObjectRef ref)
	Internal helper function downcast a ref without check. More...

static void	FFIClearAfterMove (ObjectRef *ref)
	Clear the object ref data field without DecRef after we successfully moved the field. More...

template<typename ObjectType >
static ObjectPtr< ObjectType >	GetDataPtr (const ObjectRef &ref)
	Internal helper function get data_ as ObjectPtr of ObjectType. More...

Friends
class	TVMPODValue_

class	TVMRetValue

class	TVMArgsSetter

Additional Inherited Members
Static Public Attributes inherited from tvm::runtime::ObjectRef
static constexpr bool	_type_is_nullable = true

Protected Attributes inherited from tvm::runtime::ObjectRef
ObjectPtr< Object >	data_
	Internal pointer that backs the reference. More...

Detailed Description

Managed NDArray. The array is backed by reference counted blocks.

Member Typedef Documentation

◆ ContainerType

using tvm::runtime::NDArray::ContainerType = Container

Container type for Object system.

Constructor & Destructor Documentation

◆ NDArray() [1/2]

tvm::runtime::NDArray::NDArray ( )

inline

default constructor

◆ NDArray() [2/2]

tvm::runtime::NDArray::NDArray ( ObjectPtr< Object > data )

inlineexplicit

constructor.

Parameters

data	ObjectPtr to the data container.

Member Function Documentation

◆ AbilityOfZeroCopyForDLTensor()

static bool tvm::runtime::NDArray::AbilityOfZeroCopyForDLTensor	(	DLTensor *	tensor,
		const Device &	dev
	)

static

Check conditions for construction NDArray over DLTensor without copying. There are three conditions to check:

Destination device is the same as DLTensor device
Destination device id is the same as DLTensor device id
Memory in DLTensor is aligned as expected for NDArray
Parameters

tensor the DLTensor.

dev destination device.

Returns
true if all conditions are satisfied.

◆ CopyFrom() [1/2]

void tvm::runtime::NDArray::CopyFrom ( const DLTensor * other )

inline

Copy data content from another array.

Parameters

other The source array to be copied from.

Note: The copy may happen asynchronously if it involves a GPU context. TVMSynchronize is necessary.

◆ CopyFrom() [2/2]

void tvm::runtime::NDArray::CopyFrom ( const NDArray & other )

inline

◆ CopyFromBytes()

void tvm::runtime::NDArray::CopyFromBytes	(	const void *	data,
		size_t	nbytes
	)

Copy data content from a byte buffer.

Parameters

data	The source bytes to be copied from.
nbytes	The size of the buffer in bytes Must be equal to the size of the NDArray.

Note: The copy always triggers a TVMSynchronize.

◆ CopyFromTo()

static void tvm::runtime::NDArray::CopyFromTo	(	const DLTensor *	from,
		DLTensor *	to,
		TVMStreamHandle	stream = `nullptr`
	)

static

Function to copy data from one array to another.

Parameters

from	The source array.
to	The target array.
stream	The stream used in copy.

◆ CopyTo() [1/3]

NDArray tvm::runtime::NDArray::CopyTo	(	const Device &	dev,
		Optional< String >	mem_scope = `NullOpt`
	)		const

Copy the data to another device.

Parameters

dev	The target device.
mem_scope	The memory scope of the target array.

Returns: The array under another device.

Note: The copy always triggers a TVMSynchronize.

◆ CopyTo() [2/3]

void tvm::runtime::NDArray::CopyTo ( const NDArray & other ) const

inline

◆ CopyTo() [3/3]

void tvm::runtime::NDArray::CopyTo ( DLTensor * other ) const

inline

Copy data content into another array.

Parameters

other The source array to be copied from.

Note: The copy may happen asynchronously if it involves a GPU context. TVMSynchronize is necessary.

◆ CopyToBytes()

void tvm::runtime::NDArray::CopyToBytes	(	void *	data,
		size_t	nbytes
	)		const

Copy data content into another array.

Parameters

data	The source bytes to be copied from.
nbytes	The size of the data buffer. Must be equal to the size of the NDArray.

Note: The copy always triggers a TVMSynchronize.

◆ CreateView()

NDArray tvm::runtime::NDArray::CreateView	(	ShapeTuple	shape,
		DLDataType	dtype,
		uint64_t	relative_byte_offset = `0`
	)

Create a NDArray that shares the data memory with the current one.

Parameters

shape	The shape of the new array.
dtype	The data type of the new array.
relative_byte_offset	The offset of the output NDArray, relative to the current byte offset.

By default, the offset of the view is the same as the offset of the current array.

Note: The new array must not allow access of addresses which would be out of bounds in the current array. If the new array is larger than the current array, or if the relative_byte_offset would place the end of the new array outside the bounds of the current array, this function will raise an exception.

◆ DataType()

runtime::DataType tvm::runtime::NDArray::DataType ( ) const

◆ Empty()

static NDArray tvm::runtime::NDArray::Empty	(	ShapeTuple	shape,
		DLDataType	dtype,
		Device	dev,
		Optional< String >	mem_scope = `NullOpt`
	)

static

Create an empty NDArray.

Parameters

shape	The shape of the new array.
dtype	The data type of the new array.
dev	The device of the array.
mem_scope	The memory scope of the array.

Returns: The created Array

◆ FFIDataFromHandle()

ObjectPtr< Object > tvm::runtime::NDArray::FFIDataFromHandle ( TVMArrayHandle handle )

inlinestaticprotected

Construct NDArray's Data field from array handle in FFI.

Parameters

handle The array handle.

Returns: The corresponding ObjectPtr to the constructed container object.

Note: We keep a special calling convention for NDArray by passing ContainerBase pointer in FFI. As a result, the argument is compatible to DLTensor*.

◆ FFIDecRef()

void tvm::runtime::NDArray::FFIDecRef ( TVMArrayHandle handle )

inlinestaticprotected

DecRef resource managed by an FFI array handle.

Parameters

handle The array handle.

◆ FFIGetHandle()

TVMArrayHandle tvm::runtime::NDArray::FFIGetHandle ( const ObjectRef & nd )

inlinestaticprotected

Get FFI Array handle from ndarray.

Parameters

nd	The object with ndarray type.

Returns: The result array handle.

◆ FromDLPack()

static NDArray tvm::runtime::NDArray::FromDLPack ( DLManagedTensor * tensor )

static

Create a NDArray backed by a dlpack tensor.

This allows us to create a NDArray using the memory allocated by an external deep learning framework that is DLPack compatible.

The memory is retained until the NDArray went out of scope.

Parameters

tensor The DLPack tensor to copy from.

Returns: The created NDArray view.

◆ FromExternalDLTensor()

static NDArray tvm::runtime::NDArray::FromExternalDLTensor ( const DLTensor & dl_tensor )

static

Create a NDArray backed by an external DLTensor without memory copying.

If DLTensor is not contiguous or has bad aligned data, It fails. This allows us to create a NDArray using the memory allocated by an external source. Responsibility for memory retaining lies with the external source.

Parameters

dl_tensor The DLTensor for NDArray base.

Returns: The created NDArray view.

◆ get_mutable()

NDArray::Container * tvm::runtime::NDArray::get_mutable ( ) const

inlineprotected

Get mutable internal container pointer.

Returns: a mutable container pointer.

◆ IsContiguous()

bool tvm::runtime::NDArray::IsContiguous ( ) const

inline

Returns: Whether the tensor is contiguous

◆ Load()

bool tvm::runtime::NDArray::Load ( dmlc::Stream * stream )

inline

Load NDArray from stream.

Parameters

stream The input data stream

Returns: Whether load is successful

◆ NewFromDLTensor()

static NDArray tvm::runtime::NDArray::NewFromDLTensor	(	DLTensor *	dl_tensor,
		const Device &	dev
	)

static

Create new NDArray, data is copied from DLTensor.

Parameters

dl_tensor	The DLTensor to copy from.
dev	device location of the created NDArray.

Returns: The created NDArray view.

◆ operator->()

const DLTensor * tvm::runtime::NDArray::operator-> ( ) const

inline

Returns: Pointer to content of DLTensor

◆ reset()

void tvm::runtime::NDArray::reset ( )

inline

reset the content of NDArray to be nullptr

◆ Save()

void tvm::runtime::NDArray::Save ( dmlc::Stream * stream ) const

inline

Save NDArray to stream.

Parameters

stream The output data stream

◆ Shape()

ShapeTuple tvm::runtime::NDArray::Shape ( ) const

◆ ToDLPack()

DLManagedTensor* tvm::runtime::NDArray::ToDLPack ( ) const

Create a reference view of NDArray that represents as DLManagedTensor.

Returns: A DLManagedTensor

◆ use_count()

int tvm::runtime::NDArray::use_count ( ) const

inline

Returns: the reference counter

Note: this number is approximate in multi-threaded setting.

Friends And Related Function Documentation

◆ TVMArgsSetter

friend class TVMArgsSetter

friend

◆ TVMPODValue_

friend class TVMPODValue_

friend

◆ TVMRetValue

friend class TVMRetValue

friend

The documentation for this class was generated from the following file:

include/tvm/runtime/ndarray.h

Classes

Public Types

Public Member Functions

Static Public Member Functions

Protected Member Functions

Static Protected Member Functions

Friends

Additional Inherited Members

Detailed Description

Member Typedef Documentation

◆ ContainerType

Constructor & Destructor Documentation

◆ NDArray() [1/2]

◆ NDArray() [2/2]

Member Function Documentation

◆ AbilityOfZeroCopyForDLTensor()

◆ CopyFrom() [1/2]

◆ CopyFrom() [2/2]

◆ CopyFromBytes()

◆ CopyFromTo()

◆ CopyTo() [1/3]

◆ CopyTo() [2/3]

◆ CopyTo() [3/3]

◆ CopyToBytes()

◆ CreateView()

◆ DataType()

◆ Empty()

◆ FFIDataFromHandle()

◆ FFIDecRef()

◆ FFIGetHandle()

◆ FromDLPack()

◆ FromExternalDLTensor()

◆ get_mutable()

◆ IsContiguous()

◆ Load()

◆ NewFromDLTensor()

◆ operator->()

◆ reset()

◆ Save()

◆ Shape()

◆ ToDLPack()

◆ use_count()

Friends And Related Function Documentation

◆ TVMArgsSetter

◆ TVMPODValue_

◆ TVMRetValue