tvm::runtime::TensorMemoryPool Class Reference

A cache of owning device tensors. More...

#include <tensor.h>

Inheritance diagram for tvm::runtime::TensorMemoryPool:

Collaboration diagram for tvm::runtime::TensorMemoryPool:

Public Member Functions
	TensorMemoryPool (size_t page_size=4096)
virtual	~TensorMemoryPool ()
virtual ffi::Tensor	Empty (ffi::ShapeView shape, DLDataType dtype, DLDevice device, ffi::Optional< ffi::String > mem_scope=std::nullopt)
	Return an owning compact tensor in ordinary global memory.
virtual ffi::Tensor	CreateView (ffi::Tensor backing, ffi::Shape shape, DLDataType dtype, uint64_t byte_offset=0, ffi::Optional< ffi::String > mem_scope=std::nullopt)
	Create a bounded owning view of a backing tensor.
void	Clear ()
	Release cached backing; live tensors remain valid and may return later.
size_t	UsedMemory () const
	Bytes held by this pool, including checked-out backing.
	TVM_FFI_DECLARE_OBJECT_INFO ("runtime.TensorMemoryPool", TensorMemoryPool, ffi::Object)

Static Public Attributes
static constexpr bool	_type_mutable = true

Protected Member Functions
virtual ffi::Tensor	AllocBacking (size_t nbytes, size_t alignment, DLDevice device)
	Allocate fresh owning uint8[nbytes] backing at the requested alignment.

Detailed Description

A cache of owning device tensors.

Construct pools with ffi::make_object. Live tensors retain the pool directly; only unpooled backing tensors enter its cache. A backend supplies independent backing owners, never a deleter that calls or retains this pool.

This class protects host ownership, not device execution. An adapter must keep separate pools for incompatible contexts/streams, or establish device ordering before reuse. Callers must finish using tensors before releasing their owners.

Constructor & Destructor Documentation

TensorMemoryPool()

tvm::runtime::TensorMemoryPool::TensorMemoryPool ( size_t  page_size = 4096 )

inlineexplicit

~TensorMemoryPool()

virtual tvm::runtime::TensorMemoryPool::~TensorMemoryPool ( )

inlinevirtual

Member Function Documentation

AllocBacking()

virtual ffi::Tensor tvm::runtime::TensorMemoryPool::AllocBacking ( size_t  nbytes, size_t  alignment, DLDevice  device  )

inlineprotectedvirtual

Allocate fresh owning uint8[nbytes] backing at the requested alignment.

Note

Opaque handles must satisfy backend alignment requirements. The backing owner releases its resources independently of the pool and must not allocate through a DeviceAPI::Empty implementation that routes back to this pool.

Clear()

void tvm::runtime::TensorMemoryPool::Clear ( )

inline

Release cached backing; live tensors remain valid and may return later.

CreateView()

virtual ffi::Tensor tvm::runtime::TensorMemoryPool::CreateView ( ffi::Tensor  backing, ffi::Shape  shape, DLDataType  dtype, uint64_t  byte_offset = 0, ffi::Optional< ffi::String >  mem_scope = std::nullopt  )

inlinevirtual

Create a bounded owning view of a backing tensor.

Empty()

virtual ffi::Tensor tvm::runtime::TensorMemoryPool::Empty ( ffi::ShapeView  shape, DLDataType  dtype, DLDevice  device, ffi::Optional< ffi::String >  mem_scope = std::nullopt  )

inlinevirtual

Return an owning compact tensor in ordinary global memory.

TVM_FFI_DECLARE_OBJECT_INFO()

tvm::runtime::TensorMemoryPool::TVM_FFI_DECLARE_OBJECT_INFO	(	"runtime.TensorMemoryPool"	,
		TensorMemoryPool	,
		ffi::Object
	)

UsedMemory()

size_t tvm::runtime::TensorMemoryPool::UsedMemory ( ) const

inline

Bytes held by this pool, including checked-out backing.

Member Data Documentation

_type_mutable

constexpr bool tvm::runtime::TensorMemoryPool::_type_mutable = true

staticconstexpr

---

The documentation for this class was generated from the following file:

• include/tvm/runtime/tensor.h