Class TensorMemoryPool

Class TensorMemoryPool#

Nested Relationships#

Nested Types#

Inheritance Relationships#

Base Type#

  • public ffi::Object

Class Documentation#

class TensorMemoryPool : public ffi::Object#

A cache of owning device tensors.

Construct pools with ffi::make_object. Live tensors retain the pool directly; only unpooled backing tensors enter its cache. A backend supplies independent backing owners, never a deleter that calls or retains this pool.

This class protects host ownership, not device execution. An adapter must keep separate pools for incompatible contexts/streams, or establish device ordering before reuse. Callers must finish using tensors before releasing their owners.

Public Functions

inline explicit TensorMemoryPool(size_t page_size = 4096)#
inline virtual ~TensorMemoryPool()#
inline virtual ffi::Tensor Empty(ffi::ShapeView shape, DLDataType dtype, DLDevice device, ffi::Optional<MemoryScope> mem_scope = std::nullopt)#

Return an owning compact tensor in ordinary global memory.

inline virtual ffi::Tensor CreateView(ffi::Tensor backing, ffi::Shape shape, DLDataType dtype, uint64_t byte_offset = 0, ffi::Optional<MemoryScope> mem_scope = std::nullopt)#

Create a bounded owning view of a backing tensor.

inline void Clear()#

Release cached backing; live tensors remain valid and may return later.

inline size_t UsedMemory() const#

Bytes held by this pool, including checked-out backing.

Public Static Functions

static inline int32_t _GetOrAllocRuntimeTypeIndex()#
static inline int32_t RuntimeTypeIndex() noexcept#

Protected Functions

inline virtual ffi::Tensor AllocBacking(size_t nbytes, size_t alignment, DLDevice device)#

Allocate fresh owning uint8[nbytes] backing at the requested alignment.

Note

Opaque handles must satisfy backend alignment requirements. The backing owner releases its resources independently of the pool and must not allocate through a DeviceAPI::Empty implementation that routes back to this pool.