Class CubinModule

Class CubinModule#

Class Documentation#

class CubinModule#

CUDA CUBIN module loader and manager.

This class provides a RAII wrapper around CUDA Runtime API’s library management. It loads a CUBIN module from memory and manages the library handle automatically. The library is unloaded when the CubinModule object is destroyed.

See also

TVM_FFI_EMBED_CUBIN for embedding CUBIN at compile time

See also

CubinKernel for kernel launching

Features

  • Load CUBIN from memory (embedded data or runtime-generated)

  • Automatic resource management (RAII pattern)

  • Multi-GPU execution using CUDA primary contexts

  • Retrieve multiple kernels from the same module

Example Usage
// Load CUBIN from memory
tvm::ffi::Bytes cubin_data = ...;
tvm::ffi::CubinModule module(cubin_data);

// Get kernels by name
tvm::ffi::CubinKernel kernel1 = module["add_one"];
tvm::ffi::CubinKernel kernel2 = module.GetKernel("mul_two");

// Launch kernels
void* args[] = {...};
tvm::ffi::dim3 grid(32), block(256);
cudaStream_t stream = ...;
kernel1.Launch(args, grid, block, stream);

Note

This class is movable but not copyable.

Public Functions

inline explicit CubinModule(const Bytes &bytes)#

Load CUBIN module from memory.

Parameters:

bytes – CUBIN binary data as a Bytes object.

inline explicit CubinModule(const char *code)#

Load CUBIN module from raw memory buffer.

Note

The code buffer points to an ELF image.

Parameters:

code – Pointer to CUBIN binary data.

inline ~CubinModule()#

Destructor unloads the library.

inline CubinKernel GetKernel(const char *name)#

Get a kernel function from the module by name.

Parameters:

name – Name of the kernel function.

Returns:

CubinKernel object representing the loaded kernel.

inline CubinKernel GetKernelWithMaxDynamicSharedMemory(const char *name, int64_t dynamic_smem_max)#

Get a kernel function from the module by name with maximum dynamic shared memory.

Parameters:

  • name – Name of the kernel function.

  • dynamic_smem_max – Maximum dynamic shared memory in bytes to set for this kernel. -1 (default) means maximum available dynamic shared memory (device max - static shared memory used by kernel).

Returns:

CubinKernel object representing the loaded kernel.

inline cudaLibrary_t GetHandle() const#

Get the underlying cudaLibrary_t handle.

CubinModule(const CubinModule&) = delete#
inline CubinModule(CubinModule &&other) noexcept#

Move constructor for CubinModule.

Transfers ownership of the CUDA library handle from another CubinModule instance.

Parameters:

other – The source CubinModule to move from (will be left in an empty state).

Contents