C++ Tooling

C++ Tooling#

This guide covers the TVM-FFI C++ toolchain: header layout, build integration, library distribution, and editor setup. For an end-to-end walkthrough that builds, loads, and calls a C++/CUDA kernel, see Quick Start.

Prerequisite

  • Compiler: C++17-capable toolchain (GCC/Clang/MSVC)

  • CMake: 3.18 or newer

  • TVM-FFI installed via

    pip install --reinstall --upgrade apache-tvm-ffi

C++ Headers#

Core APIs. A single umbrella header exposes most of the API surface, including functions, objects, Any/AnyView, tensors, and exception handling.

#include <tvm/ffi/tvm_ffi.h>

Extra features live in dedicated headers under tvm/ffi/extra/ and should be included only when needed:

  • Environment API – allocator, stream, and device access (see Tensor and DLPack for usage): #include <tvm/ffi/extra/c_env_api.h>

  • Dynamic module loading (see Modules): #include <tvm/ffi/extra/module.h>

  • CUBIN launcher (see CUBIN Launcher Guide): #include <tvm/ffi/extra/cuda/cubin_launcher.h>

Build with TVM-FFI#

CMake#

TVM-FFI ships CMake utilities and imported targets through its package configuration. The two primary functions are tvm_ffi_configure_target and tvm_ffi_install, both defined in cmake/Utils/Library.cmake.

Hint

See examples/python_packaging/CMakeLists.txt for a complete working example.

Configure Target#

tvm_ffi_configure_target wires a CMake target to TVM-FFI with sensible defaults: it links headers and the shared library, configures debug symbols, and optionally runs stub generation as a post-build step.

tvm_ffi_configure_target(
  <target>
  [LINK_SHARED  ON|OFF  ]
  [LINK_HEADER  ON|OFF  ]
  [DEBUG_SYMBOL ON|OFF  ]
  [MSVC_FLAGS   ON|OFF  ]
  [STUB_INIT    ON|OFF  ]
  [STUB_DIR     <dir>   ]
  [STUB_PKG     <pkg>   ]
  [STUB_PREFIX  <prefix>]
)
LINK_SHARED:

(default: ON) Link against the TVM-FFI shared library tvm_ffi::shared. Set OFF for header-only usage or deferred runtime loading.

LINK_HEADER:

(default: ON) Link against the TVM-FFI headers via tvm_ffi::header. Set OFF when you manage include paths and compile definitions manually.

DEBUG_SYMBOL:

(default: ON) Enable debug symbol post-processing hooks. On Apple platforms this runs dsymutil.

MSVC_FLAGS:

(default: ON) Apply MSVC compatibility flags.

STUB_DIR:

(default: “”) Output directory for generated Python stubs. When set, stub generation runs as a post-build step.

STUB_INIT:

(default: OFF) Allow the stub generator to initialize a new package layout. Requires STUB_DIR.

STUB_PKG:

(default: “”) Package name passed to the stub generator. Requires STUB_DIR and STUB_INIT=ON.

STUB_PREFIX:

(default: “”) Module prefix passed to the stub generator. Requires STUB_DIR and STUB_INIT=ON.

See CMake-based Generation for a detailed explanation of each STUB_* option and the generation modes they control.

Install Target#

tvm_ffi_install installs platform-specific artifacts for a target. On Apple platforms it installs the .dSYM bundle when present; on other platforms this is currently a no-op.

tvm_ffi_install(
  <target>
  [DESTINATION <dir>]
)
DESTINATION:

Install destination directory relative to CMAKE_INSTALL_PREFIX.

Set tvm_ffi_ROOT#

If find_package(tvm_ffi CONFIG REQUIRED) fails because CMake cannot locate the package, pass tvm_ffi_ROOT explicitly:

cmake -S . -B build \
  -Dtvm_ffi_ROOT="$(tvm-ffi-config --cmakedir)"

Note

When packaging Python wheels with scikit-build-core, tvm_ffi_ROOT is discovered automatically from the active Python environment.

GCC/NVCC#

For quick prototyping or CI scripts without CMake, invoke g++ or nvcc directly with flags from tvm-ffi-config. The examples below are from the Quick Start tutorial:

g++ -shared -O3 compile/add_one_cpu.cc  \
    -fPIC -fvisibility=hidden           \
    $(tvm-ffi-config --cxxflags)        \
    $(tvm-ffi-config --ldflags)         \
    $(tvm-ffi-config --libs)            \
    -o $BUILD_DIR/add_one_cpu.so

The three tvm-ffi-config flags provide:

--cxxflags:

Include paths and compile definitions (-I..., -D...)

--ldflags:

Library search paths (-L..., -Wl,-rpath,...)

--libs:

Libraries to link (-ltvm_ffi)

RPATH handling. The resulting shared library links against libtvm_ffi.so, so the dynamic linker must be able to find it at load time:

  • Python distribution. import tvm_ffi preloads libtvm_ffi.so into the process before any user library is loaded, so the RPATH requirement is already satisfied without additional linker flags.

  • Pure C++ distribution. You must ensure libtvm_ffi.so is on the library search path. Either set -Wl,-rpath,$(tvm-ffi-config --libdir) at link time, or place libtvm_ffi.so alongside your binary.

Library Distribution#

When distributing pre-built shared libraries on Linux, glibc symbol versioning can cause load-time failures on systems with a different glibc version. The standard solution is the manylinux approach: build on old glibc, run on new.

Build environment. Use a manylinux Docker image:

docker pull quay.io/pypa/manylinux2014_x86_64

Build host and device code inside the container. For CUDA:

nvcc -shared -Xcompiler -fPIC your_kernel.cu -o kernel.so \
    $(tvm-ffi-config --cxxflags) \
    $(tvm-ffi-config --ldflags) \
    $(tvm-ffi-config --libs)

Verify glibc requirements. Inspect the minimum glibc version your binary requires:

objdump -T your_kernel.so | grep GLIBC_

The apache-tvm-ffi wheel is already manylinux-compatible, so linking against it inside a manylinux build environment produces portable binaries.

Editor Setup#

The following configuration enables code completion and diagnostics in VSCode, Cursor, or any editor backed by clangd.

CMake Tools (VSCode/Cursor). Add these workspace settings so CMake Tools can locate TVM-FFI and generate compile_commands.json:

{
    "cmake.buildDirectory": "${workspaceFolder}/build-vscode",
    "cmake.configureArgs": [
        "-DCMAKE_EXPORT_COMPILE_COMMANDS=ON"
    ],
    "cmake.configureSettings": {
        "Python_EXECUTABLE": "${workspaceFolder}/.venv/bin/python3",
        "tvm_ffi_ROOT": "${workspaceFolder}/.venv/lib/pythonX.Y/site-packages/tvm_ffi/share/cmake/tvm_ffi"
    }
}

Important

Make sure Python_EXECUTABLE and tvm_ffi_ROOT match the virtual environment you intend to use.

clangd. Create a .clangd file at the project root pointing to the CMake compilation database. The snippet below also strips NVCC flags that clangd does not recognize:

CompileFlags:
  CompilationDatabase: build-vscode/
  Remove: # for NVCC compatibility
    - -forward-unknown-to-host-compiler
    - --generate-code*
    - -Xcompiler*

Further Reading#

Contents