Managed Tensor. The array is backed by reference counted blocks. More...

#include <tensor.h>

Inheritance diagram for tvm::runtime::Tensor:

Collaboration diagram for tvm::runtime::Tensor:

Public Member Functions
	Tensor ()=default

	Tensor (ffi::ObjectPtr< ffi::TensorObj > data)
	constructor.

	Tensor (ffi::UnsafeInit tag)

	Tensor (ffi::Tensor &&other)

	Tensor (const ffi::Tensor &other)

ffi::ShapeView	Shape () const

DLDataType	DataType () const

const DLTensor *	operator-> () const

void	CopyFrom (const DLTensor *other)
	Copy data content from another array.

void	CopyFrom (const Tensor &other)

TVM_RUNTIME_DLL void	CopyFromBytes (const void *data, size_t nbytes)
	Copy data content from a byte buffer.

void	CopyTo (DLTensor *other) const
	Copy data content into another array.

void	CopyTo (const Tensor &other) const

TVM_RUNTIME_DLL void	CopyToBytes (void *data, size_t nbytes) const
	Copy data content into another array.

TVM_RUNTIME_DLL Tensor	CopyTo (const Device &dev, ffi::Optional< ffi::String > mem_scope=std::nullopt) const
	Copy the data to another device.

bool	Load (support::Stream *stream)
	Load Tensor from stream.

void	Save (support::Stream *stream) const
	Save Tensor to stream.

TVM_RUNTIME_DLL Tensor	CreateView (ffi::Shape shape, DLDataType dtype, uint64_t relative_byte_offset=0) const
	Create a Tensor that shares the data memory with the current one.

Static Public Member Functions
static Tensor	FromDLPack (DLManagedTensor *tensor)

static Tensor	FromDLPackVersioned (DLManagedTensorVersioned *tensor)

static TVM_RUNTIME_DLL Tensor	Empty (ffi::Shape shape, DLDataType dtype, Device dev, ffi::Optional< ffi::String > mem_scope=std::nullopt)
	Create an empty Tensor.

static TVM_RUNTIME_DLL void	CopyFromTo (const DLTensor from, DLTensor to, TVMStreamHandle stream=nullptr)
	Function to copy data from one array to another.

static TVM_RUNTIME_DLL void	CopyToBytes (const DLTensor from, void to, size_t nbytes, TVMStreamHandle stream=nullptr)
	Function to copy data from one array to a byte buffer.

static TVM_RUNTIME_DLL void	CopyFromBytes (const DLTensor to, void from, size_t nbytes, TVMStreamHandle stream=nullptr)
	Function to copy data from one array to a byte buffer.

static TVM_RUNTIME_DLL bool	IsStorageShared (const DLTensor a, const DLTensor b)
	Check if two tensors share the same underlying storage.

static bool	IsStorageShared (const Tensor &a, const Tensor &b)
	Tensor overload of IsStorageShared.

Detailed Description

Managed Tensor. The array is backed by reference counted blocks.

Constructor & Destructor Documentation

◆ Tensor() [1/5]

tvm::runtime::Tensor::Tensor ( )

default

◆ Tensor() [2/5]

tvm::runtime::Tensor::Tensor ( ffi::ObjectPtr< ffi::TensorObj > data )

inlineexplicit

constructor.

Parameters

data	ffi::ObjectPtr to the data container.

◆ Tensor() [3/5]

tvm::runtime::Tensor::Tensor ( ffi::UnsafeInit tag )

inlineexplicit

◆ Tensor() [4/5]

tvm::runtime::Tensor::Tensor ( ffi::Tensor && other )

inline

◆ Tensor() [5/5]

tvm::runtime::Tensor::Tensor ( const ffi::Tensor & other )

inline

Member Function Documentation

◆ CopyFrom() [1/2]

void tvm::runtime::Tensor::CopyFrom ( const DLTensor * other )

inline

Copy data content from another array.

Parameters

other The source array to be copied from.

Note: The copy may happen asynchronously if it involves a GPU context. TVMSynchronize is necessary.

◆ CopyFrom() [2/2]

void tvm::runtime::Tensor::CopyFrom ( const Tensor & other )

inline

◆ CopyFromBytes() [1/2]

static TVM_RUNTIME_DLL void tvm::runtime::Tensor::CopyFromBytes	(	const DLTensor *	to,
		void *	from,
		size_t	nbytes,
		TVMStreamHandle	stream = `nullptr`
	)

static

Function to copy data from one array to a byte buffer.

Parameters

from	The source array.
to	The target byte buffer.
nbytes	The size of the data buffer.
stream	The stream used in copy.

◆ CopyFromBytes() [2/2]

TVM_RUNTIME_DLL void tvm::runtime::Tensor::CopyFromBytes	(	const void *	data,
		size_t	nbytes
	)

Copy data content from a byte buffer.

Parameters

data	The source bytes to be copied from.
nbytes	The size of the buffer in bytes Must be equal to the size of the Tensor.

Note: The copy always triggers a TVMSynchronize.

◆ CopyFromTo()

static TVM_RUNTIME_DLL void tvm::runtime::Tensor::CopyFromTo	(	const DLTensor *	from,
		DLTensor *	to,
		TVMStreamHandle	stream = `nullptr`
	)

static

Function to copy data from one array to another.

Parameters

from	The source array.
to	The target array.
stream	The stream used in copy.

◆ CopyTo() [1/3]

TVM_RUNTIME_DLL Tensor tvm::runtime::Tensor::CopyTo	(	const Device &	dev,
		ffi::Optional< ffi::String >	mem_scope = `std::nullopt`
	)		const

Copy the data to another device.

Parameters

dev	The target device.
mem_scope	The memory scope of the target array.

Returns: The array under another device.

Note: The copy always triggers a TVMSynchronize.

◆ CopyTo() [2/3]

void tvm::runtime::Tensor::CopyTo ( const Tensor & other ) const

inline

◆ CopyTo() [3/3]

void tvm::runtime::Tensor::CopyTo ( DLTensor * other ) const

inline

Copy data content into another array.

Parameters

other The source array to be copied from.

Note: The copy may happen asynchronously if it involves a GPU context. TVMSynchronize is necessary.

◆ CopyToBytes() [1/2]

static TVM_RUNTIME_DLL void tvm::runtime::Tensor::CopyToBytes	(	const DLTensor *	from,
		void *	to,
		size_t	nbytes,
		TVMStreamHandle	stream = `nullptr`
	)

static

Function to copy data from one array to a byte buffer.

Parameters

from	The source array.
to	The target byte buffer.
nbytes	The size of the data buffer.
stream	The stream used in copy.

◆ CopyToBytes() [2/2]

TVM_RUNTIME_DLL void tvm::runtime::Tensor::CopyToBytes	(	void *	data,
		size_t	nbytes
	)		const

Copy data content into another array.

Parameters

data	The source bytes to be copied from.
nbytes	The size of the data buffer. Must be equal to the size of the Tensor.

Note: The copy always triggers a TVMSynchronize.

◆ CreateView()

TVM_RUNTIME_DLL Tensor tvm::runtime::Tensor::CreateView	(	ffi::Shape	shape,
		DLDataType	dtype,
		uint64_t	relative_byte_offset = `0`
	)		const

Create a Tensor that shares the data memory with the current one.

Parameters

shape	The shape of the new array.
dtype	The data type of the new array.
relative_byte_offset	The offset of the output Tensor, relative to the current byte offset.

By default, the offset of the view is the same as the offset of the current array.

Note: The new array must not allow access of addresses which would be out of bounds in the current array. If the new array is larger than the current array, or if the relative_byte_offset would place the end of the new array outside the bounds of the current array, this function will raise an exception.

◆ DataType()

DLDataType tvm::runtime::Tensor::DataType ( ) const

inline

◆ Empty()

static TVM_RUNTIME_DLL Tensor tvm::runtime::Tensor::Empty	(	ffi::Shape	shape,
		DLDataType	dtype,
		Device	dev,
		ffi::Optional< ffi::String >	mem_scope = `std::nullopt`
	)

static

Create an empty Tensor.

Parameters

shape	The shape of the new array.
dtype	The data type of the new array.
dev	The device of the array.
mem_scope	The memory scope of the array.

Returns: The created Array

◆ FromDLPack()

static Tensor tvm::runtime::Tensor::FromDLPack ( DLManagedTensor * tensor )

inlinestatic

◆ FromDLPackVersioned()

static Tensor tvm::runtime::Tensor::FromDLPackVersioned ( DLManagedTensorVersioned * tensor )

inlinestatic

◆ IsStorageShared() [1/2]

static TVM_RUNTIME_DLL bool tvm::runtime::Tensor::IsStorageShared	(	const DLTensor *	a,
		const DLTensor *	b
	)

static

Check if two tensors share the same underlying storage.

This detects runtime storage aliasing (e.g. views from CreateView, etc.) but does not imply either tensor was created by CreateView.

Parameters

a	The first tensor.
b	The second tensor.

Returns: True if the tensors share the same storage.

◆ IsStorageShared() [2/2]

static bool tvm::runtime::Tensor::IsStorageShared	(	const Tensor &	a,
		const Tensor &	b
	)

static

Tensor overload of IsStorageShared.

Parameters

a	The first tensor.
b	The second tensor.

Returns: True if the tensors share the same storage.

◆ Load()

bool tvm::runtime::Tensor::Load ( support::Stream * stream )

inline

Load Tensor from stream.

Parameters

stream The input data stream

Returns: Whether load is successful

◆ operator->()

const DLTensor * tvm::runtime::Tensor::operator-> ( ) const

inline

◆ Save()

void tvm::runtime::Tensor::Save ( support::Stream * stream ) const

inline

Save Tensor to stream.

Parameters

stream The output data stream

◆ Shape()

ffi::ShapeView tvm::runtime::Tensor::Shape ( ) const

inline

The documentation for this class was generated from the following file:

include/tvm/runtime/tensor.h

Public Member Functions

Static Public Member Functions

Detailed Description

Constructor & Destructor Documentation

◆ Tensor() [1/5]

◆ Tensor() [2/5]

◆ Tensor() [3/5]

◆ Tensor() [4/5]

◆ Tensor() [5/5]

Member Function Documentation

◆ CopyFrom() [1/2]

◆ CopyFrom() [2/2]

◆ CopyFromBytes() [1/2]

◆ CopyFromBytes() [2/2]

◆ CopyFromTo()

◆ CopyTo() [1/3]

◆ CopyTo() [2/3]

◆ CopyTo() [3/3]

◆ CopyToBytes() [1/2]

◆ CopyToBytes() [2/2]

◆ CreateView()

◆ DataType()

◆ Empty()

◆ FromDLPack()

◆ FromDLPackVersioned()

◆ IsStorageShared() [1/2]

◆ IsStorageShared() [2/2]

◆ Load()

◆ operator->()

◆ Save()

◆ Shape()