CUDA Programming Guide

CUDA Programming Guide#

Note

Native-level kernel authoring for the CUDA backend (the "cuda" target): the thread hierarchy, memory scopes, the Tx.cuda.* / Tx.ptx.* intrinsics, and the compile / run / inspect loop. The chapters build from a complete scale example through shared-memory and warp-level kernels.

What “native level” means#

A native-level TIRx kernel reads like a structured device kernel: you place threads yourself, allocate shared/per-thread local buffers, write loops and barriers, and call device intrinsics directly. You explicitly choose the orchestration and layouts; standard lowering still dispatches primitives, applies layouts, and vectorizes or unrolls marked loops. This is the foundation the tile primitives (Tile Primitives) are built on; everything here is what those primitives ultimately lower to, so it is also where you go when a hardware feature does not have a primitive yet.

The authoring model#

  • @Tx.prim_func (or @Tx.jit for compile-time-specialized) kernels, written with from tvm.script import tirx as Tx;

  • Tx.device_entry() plus scope-id intrinsics for thread binding;

  • Tx.match_buffer parameters and Tx.alloc_* scratch buffers;

  • ordinary loops, branches, and scalar math;

  • tvm.compile(mod, target=..., tir_pipeline="tirx") to build, then call the result directly.

All native authoring uses these imports. The __future__ import lets @Tx.jit kernels reference compile-time parameters inside type annotations (see Defining a function); it is harmless for ordinary kernels:

from __future__ import annotations
import tvm
from tvm.script import tirx as Tx

Start here#

If this is your first TIRx kernel, complete First Kernel before using the chapters below as a language reference.

Language guide#

CUDA execution and compilation#

Advanced topics#

Contents