tvm
Loading...
Searching...
No Matches
Namespaces | Enumerations | Functions
builtin.h File Reference

TIR builtin intrinsics. More...

#include <tvm/ir/op.h>
#include <tvm/ir/prim/expr.h>
Include dependency graph for builtin.h:
This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Namespaces

namespace  tvm
 An object that builds and maintains block scope and StmtSref mapping for Dependence analysis.
 
namespace  tvm::tirx
 
namespace  tvm::tirx::builtin
 Collection of builtin intrinsics as ops.
 

Enumerations

enum  tvm::tirx::builtin::TVMStructFieldKind : int {
  tvm::tirx::builtin::kDLTensorAddr , tvm::tirx::builtin::kDLTensorData , tvm::tirx::builtin::kDLTensorShape , tvm::tirx::builtin::kDLTensorStrides ,
  tvm::tirx::builtin::kDLTensorNDim , tvm::tirx::builtin::kDLTensorTypeCode , tvm::tirx::builtin::kDLTensorTypeBits , tvm::tirx::builtin::kDLTensorTypeLanes ,
  tvm::tirx::builtin::kDLTensorByteOffset , tvm::tirx::builtin::kDLTensorDeviceId , tvm::tirx::builtin::kDLTensorDeviceType , tvm::tirx::builtin::kDLTensorKindBound_ ,
  tvm::tirx::builtin::kTVMValueContent , tvm::tirx::builtin::kTVMFFIAnyTypeIndex , tvm::tirx::builtin::kTVMFFIAnyZeroPadding , tvm::tirx::builtin::kTVMFFIAnyUnionValue ,
  tvm::tirx::builtin::kTVMValueKindBound_ , tvm::tirx::builtin::kInt64ArrayElem
}
 The kind of structure field info used in intrinsic. More...
 

Functions

const Op & tvm::tirx::builtin::thread_return ()
 Return from a GPU thread.
 
const Op & tvm::tirx::builtin::continue_loop ()
 Loop continue.
 
const Op & tvm::tirx::builtin::break_loop ()
 Loop break.
 
const Op & tvm::tirx::builtin::reinterpret ()
 Reinterpret the value using the target type.
 
const Op & tvm::tirx::builtin::filter ()
 Thread-set filter predicate. Used as the condition of an IfThenElse to narrow the active thread set A for the then-branch. Two forms: filter(var, lo, hi) – range form, true iff var in [lo, hi) filter(var, cond) – predicate form (e.g. var == k); true iff cond var must be a ScopeIdDef-declared Var at parse time (Verifier Rule 2).
 
const Op & tvm::tirx::builtin::selector ()
 Analysis-only active-thread selector.
 
const Op & tvm::tirx::builtin::large_uint_imm ()
 See pesudo code.
 
const Op & tvm::tirx::builtin::q_multiply_shift ()
 Execute a multiplication between two Q-numbers x and y followed by a right shift s The default rounding rule is to the nearest value, rounding half up (i.e., round(x.1) = x and round (x.5) = x+1)
 
const Op & tvm::tirx::builtin::q_multiply_shift_per_axis ()
 
const Op & tvm::tirx::builtin::address_of ()
 Returns the address of an element in the buffer (see pseudocode below).
 
const Op & tvm::tirx::builtin::isnullptr ()
 See pesudo code.
 
const Op & tvm::tirx::builtin::isnan ()
 Check if value is nan.
 
const Op & tvm::tirx::builtin::popcount ()
 Popcount.
 
const Op & tvm::tirx::builtin::fma ()
 Fused multiply add.
 
const Op & tvm::tirx::builtin::call_extern ()
 Call an extern C function with given name and signature from the types of args in the runtime environment.
 
const Op & tvm::tirx::builtin::call_pure_extern ()
 Call an pure extern C function with given name and signature from the types of args in the runtime environment.
 
const Op & tvm::tirx::builtin::call_llvm_intrin ()
 Call an LLVM intrinsic with a given intrinsic id and signature from the types of args in the runtime environment.
 
const Op & tvm::tirx::builtin::call_llvm_pure_intrin ()
 Call an LLVM pure intrinsic with a given intrinsic id and signature from the types of args in the runtime environment.
 
const Op & tvm::tirx::builtin::call_spirv_pure_glsl450 ()
 Call an SPIRV pure GLSL450 intrinsic.
 
const Op & tvm::tirx::builtin::prefetch ()
 same signature as llvm.prefetch
 
const Op & tvm::tirx::builtin::tvm_access_ptr ()
 Get head access address with memory access pattern info.
 
const Op & tvm::tirx::builtin::ptr_byte_offset ()
 Cast a handle to a typed pointer after adding a byte offset.
 
const Op & tvm::tirx::builtin::tvm_static_handle ()
 Create a function local static handle that iniitalizes to nullptr. can be used to cache function local static resources.
 
const Op & tvm::tirx::builtin::tvm_context_id ()
 Return a unique context id, used for hint of workspace separation. Different context id ganrantees not having overlapping workspace.
 
const Op & tvm::tirx::builtin::tvm_tuple ()
 tvm_tuple is not an actual function and cannot codegen. It is used to represent tuple structure in value field of AttrStmt, for the sake of giving hint to optimization.
 
const Op & tvm::tirx::builtin::handle_add_byte_offset ()
 See pesudo code.
 
const Op & tvm::tirx::builtin::tvm_struct_get ()
 See pesudo code.
 
const Op & tvm::tirx::builtin::tvm_struct_set ()
 See pesudo code.
 
const Op & tvm::tirx::builtin::lookup_param ()
 See pseudo code Type lookup_param(ffi::String param_name) { return __tvm_param__param_name; }.
 
const Op & tvm::tirx::builtin::tvm_throw_last_error ()
 See pesudo code.
 
const Op & tvm::tirx::builtin::tvm_stack_alloca ()
 See pesudo code.
 
const Op & tvm::tirx::builtin::tvm_stack_make_shape ()
 Allocate a shape tuple on stack, return the handle.
 
const Op & tvm::tirx::builtin::tvm_stack_make_array ()
 Allocate a Tensor(DLTensor) on stack, return the handle.
 
const Op & tvm::tirx::builtin::tvm_call_packed ()
 See pesudo code.
 
const Op & tvm::tirx::builtin::tvm_call_cpacked ()
 See pesudo code.
 
const Op & tvm::tirx::builtin::tvm_call_trace_packed ()
 See pesudo code.
 
const Op & tvm::tirx::builtin::tvm_thread_invariant ()
 Mark a condition to be thread invariant. This means the condition must be the same for all threads.
 
const Op & tvm::tirx::builtin::tvm_call_packed_lowered ()
 Lowered version of call packed, the space of value and type codes are explicitly allocated.
 
const Op & tvm::tirx::builtin::tvm_call_cpacked_lowered ()
 Lowered version of call c-packed, the space of value and type codes are explicitly allocated.
 
const Op & tvm::tirx::builtin::tvm_call_trace_packed_lowered ()
 Lowered version of trace intrinsic, the space of value and type codes are explicitly allocated. The return value is the (end - 1) value on the stack.
 
const Op & tvm::tirx::builtin::tvm_storage_sync ()
 See pseudo code.
 
const Op & tvm::tirx::builtin::tvm_kernel_replace_point ()
 Marker where a transform should replace generated kernel initialization.
 
const Op & tvm::tirx::builtin::tvm_warp_shuffle ()
 See pseudo code.
 
const Op & tvm::tirx::builtin::tvm_warp_shuffle_up ()
 
const Op & tvm::tirx::builtin::tvm_warp_shuffle_down ()
 
const Op & tvm::tirx::builtin::tvm_warp_shuffle_xor ()
 
const Op & tvm::tirx::builtin::tvm_warp_activemask ()
 
const Op & tvm::tirx::builtin::tvm_global_barrier_kinit ()
 Initialize the global barrier. Call this at beginning of kernel that need global barrier.
 
const Op & tvm::tirx::builtin::tvm_thread_allreduce ()
 See pesudo code.
 
const Op & tvm::tirx::builtin::cooperative_tensor_fill ()
 Fill a cooperative_tensor with a given value.
 
const Op & tvm::tirx::builtin::cooperative_tensor_load ()
 Load data from device or threadgroup memory into a cooperative_tensor.
 
const Op & tvm::tirx::builtin::cooperative_tensor_store ()
 Store data from a cooperative_tensor to device or threadgroup memory.
 
const Op & tvm::tirx::builtin::cooperative_tensor_multiply_accumulate ()
 Multiply and accumulate two matrices using cooperative_tensor (MetalPerformancePrimitives matmul2d).
 
const Op & tvm::tirx::builtin::vectorhigh ()
 Get the high level half of the vector.
 
const Op & tvm::tirx::builtin::vectorlow ()
 Get the low-level half of the vector.
 
const Op & tvm::tirx::builtin::vectorcombine ()
 Concat two vectors.
 
const Op & tvm::tirx::builtin::dp4a ()
 Dot product of two int8x4 vectors and add an optional accumulator.
 
const Op & tvm::tirx::builtin::atomic_add ()
 atomic add instruction, corresponding e.g. to atomicAdd in CUDA
 
const Op & tvm::tirx::builtin::nd_mem_alloc_with_scope ()
 Create an Nd memory allocation with storage scope.
 
const Op & tvm::tirx::builtin::texture2d_store ()
 Store to texture 2d memory.
 
const Op & tvm::tirx::builtin::texture2d_load ()
 Load from texture 2d memory.
 
const Op & tvm::tirx::builtin::dma_copy ()
 Initiate a non-blocking DMA copy from source to destination.
 
const Op & tvm::tirx::builtin::dma_wait ()
 Wait until the number of DMA groups in flight is less than or equal to some maximum.
 
const Op & tvm::tirx::builtin::dma_start_group ()
 Start a group of DMA copies.
 
const Op & tvm::tirx::builtin::dma_end_group ()
 End a group of DMA copies.
 
const Op & tvm::tirx::builtin::assume ()
 Provide a true statement that can be used for simplifications.
 
const Op & tvm::tirx::builtin::undef ()
 Returns an initialized but arbitrary value.
 
const Op & tvm::tirx::builtin::start_profile_intrinsic ()
 Profiling intrinsic.
 
const Op & tvm::tirx::builtin::end_profile_intrinsic ()
 Profiling intrinsic.
 
const Op & tvm::tirx::builtin::anylist_getitem ()
 Get a item from any list and return it.
 
const Op & tvm::tirx::builtin::anylist_resetitem ()
 Reset and clear a item in any list.
 
const Op & tvm::tirx::builtin::anylist_setitem_call_packed ()
 Set an item into any list by running packed function call.
 
const Op & tvm::tirx::builtin::anylist_setitem_call_cpacked ()
 Same as anylist_setitem_call_packed but use C calling convention.
 
const Op & tvm::tirx::builtin::get_active_lane_mask ()
 Calculate a predicate mask given an upper bound (limit) and a current value (base).
 
const Op & tvm::tirx::builtin::masked_load ()
 Masked buffer load.
 
const Op & tvm::tirx::builtin::masked_store ()
 Masked buffer store.
 
const Op & tvm::tirx::builtin::ignore_loop_partition ()
 Annotate a predicate not be considered as target condition of loop partition.
 
const Op & tvm::tirx::builtin::buffer_offset ()
 Get the element offset of a buffer given logical indices.
 
const Op & tvm::tirx::builtin::buffer_data ()
 Project the physical pointer associated with a BufferVar definition.
 
const Op & tvm::tirx::builtin::print_buffer ()
 Print the content of a buffer during runtime.
 

Detailed Description

TIR builtin intrinsics.

TIR builtin intrinsics are stored as tvm:Op. They are processed in the same way as we process Ops.

It is not necessary to create a function for every Op, as we can obtain them through Op::Get.

This file contains the most commonly used intrinsics or those that have special semantics and need compiler support.

Generated by doxygen 1.9.8