tvm
Public Member Functions | Static Public Member Functions | Public Attributes | List of all members
tvm::runtime::vm::Executable Class Reference

The executable emitted by the VM compiler. More...

#include <executable.h>

Inheritance diagram for tvm::runtime::vm::Executable:
Collaboration diagram for tvm::runtime::vm::Executable:

Public Member Functions

 TVM_MODULE_VTABLE_BEGIN ("VMExecutable")
 
 TVM_MODULE_VTABLE_ENTRY ("get_lib", &Executable::GetLib)
 
 TVM_MODULE_VTABLE_ENTRY ("get_bytecode", &Executable::GetBytecode)
 
 TVM_MODULE_VTABLE_ENTRY ("get_constants", &Executable::GetConstants)
 
 TVM_MODULE_VTABLE_ENTRY ("get_virtual_devices", &Executable::GetVirtualDevices)
 
 TVM_MODULE_VTABLE_ENTRY ("get_primitives", &Executable::GetPrimitives)
 
 TVM_MODULE_VTABLE_ENTRY ("get_stats", &Executable::Stats)
 
 TVM_MODULE_VTABLE_ENTRY ("save", &Executable::Save)
 
 TVM_MODULE_VTABLE_ENTRY ("get_function_arity", &Executable::GetFunctionArity)
 
 TVM_MODULE_VTABLE_ENTRY ("get_function_param_name", &Executable::GetFunctionParameterName)
 
 TVM_MODULE_VTABLE_ENTRY ("vm_load_executable", &Executable::VMLoadExecutable)
 
 TVM_MODULE_VTABLE_ENTRY ("move_late_bound_consts", &Executable::MoveLateBoundConstantsToFile)
 
 TVM_MODULE_VTABLE_ENTRY ("get_late_bound_consts", &Executable::GetLateBoundConstants)
 
 TVM_MODULE_VTABLE_ENTRY ("load_late_bound_consts", &Executable::LoadLateBoundConstantsFromFile)
 
 TVM_MODULE_VTABLE_ENTRY ("load_late_bound_consts_from_map", &Executable::LoadLateBoundConstantsFromMap)
 
 TVM_MODULE_VTABLE_END ()
 
int GetPropertyMask () const final
 Get the property of the runtime module . More...
 
Module VMLoadExecutable ()
 Creates a VM that loads this as the executable. More...
 
void SaveToBinary (dmlc::Stream *stream) final
 Write the Executable to the binary stream in serialized form. More...
 
void SaveToFile (const String &path, const String &format) final
 Write the Executable to the provided path as a file containing its serialized content. More...
 
TVMByteArray Save ()
 Serialize the executable into global section, constant section, and code section. This object must outlive the returned byte array. More...
 
void MoveLateBoundConstantsToStream (dmlc::Stream *stream, int64_t byte_limit)
 Returns the late-bound constants for the executable (if any) as a byte-stream. Leaves the executable's late-bound constants map empty. Only constants who's byte tensor size is greater than or equal to byte_limit are marked as late-bound. byte_limit may be zero. More...
 
void MoveLateBoundConstantsToFile (const std::string &path, int64_t byte_limit)
 As for MoveLateBoundConstantsToStream, but save to file at path. More...
 
Map< String, NDArrayGetLateBoundConstants (int64_t byte_limit)
 Get a map of all constants with larger that byte_limit in size. More...
 
void LoadLateBoundConstantsFromStream (dmlc::Stream *stream)
 Restores the late-bound constants for the executable (if any) from given byte-stream. More...
 
void LoadLateBoundConstantsFromMap (Map< String, NDArray > map)
 Restores the late-bound constants for the executable (if any) from given map. More...
 
void LoadLateBoundConstantsFromFile (const std::string &path)
 As for LoadLateBoundConstantsFromStream, but load from file at path. More...
 
std::string GetBytecode () const
 Get the serialized form of the functions. This is essentially bytecode serialization. More...
 
std::string GetConstants () const
 Returns a description of all the constants in the executable in human-readable format. Intended for debugging and diff-testing. More...
 
std::string GetVirtualDevices () const
 Returns a description of all the (virtual) devices in the executable in human-readable format. Intended for debugging and diff-testing. More...
 
std::string GetPrimitives () const
 Returns a description of all the 'primitive' (ie PackedFuncs) in the executable in human-readable format. These correspond either to PrimFuncs we've compiled locally, or functions compiled by a BYOC external codegen. Intended for debugging and diff-testing. More...
 
std::string Stats () const
 Print the detailed statistics of the given code, i.e. number of globls and constants, etc. More...
 
runtime::Module GetLib () const
 Get the lib module in an executable. Users have the flexibility to call export_library from the frontend to save the library to disk. More...
 
void SetLib (const runtime::Module &lib)
 Set the lib module in an executable. More...
 
const VMFunctionGetVMFunctionWithName (const std::string &func_name) const
 Get VMFunction. More...
 
int GetFunctionArity (std::string func) const
 Get the arity of the VMFunction. More...
 
std::string GetFunctionParameterName (std::string func, int index) const
 Get the parameter name given the function name and parameter index. More...
 
virtual ~Executable ()
 
- Public Member Functions inherited from tvm::runtime::ModuleNode
virtual ~ModuleNode ()=default
 virtual destructor More...
 
virtual const char * type_key () const =0
 
virtual PackedFunc GetFunction (const String &name, const ObjectPtr< Object > &sptr_to_self)=0
 Get a PackedFunc from module. More...
 
virtual String GetSource (const String &format="")
 Get the source code of module, when available. More...
 
virtual String GetFormat ()
 Get the format of the module, when available. More...
 
PackedFunc GetFunction (const String &name, bool query_imports=false)
 Get packed function from current module by name. More...
 
void Import (Module other)
 Import another module into this module. More...
 
const PackedFuncGetFuncFromEnv (const String &name)
 Get a function from current environment The environment includes all the imports as well as Global functions. More...
 
void ClearImports ()
 Clear all imports of the module. More...
 
const std::vector< Module > & imports () const
 
bool IsDSOExportable () const
 Returns true if this module is 'DSO exportable'. More...
 
bool IsBinarySerializable () const
 Returns true if this module is 'Binary Serializable'. More...
 
virtual bool ImplementsFunction (const String &name, bool query_imports=false)
 Returns true if this module has a definition for a function of name. If query_imports is true, also search in any imported modules. More...
 
 TVM_DECLARE_FINAL_OBJECT_INFO (ModuleNode, Object)
 
- Public Member Functions inherited from tvm::runtime::Object
uint32_t type_index () const
 
std::string GetTypeKey () const
 
size_t GetTypeKeyHash () const
 
template<typename TargetType >
bool IsInstance () const
 
bool unique () const
 
 Object ()
 
 Object (const Object &other)
 
 Object (Object &&other)
 
Objectoperator= (const Object &other)
 
Objectoperator= (Object &&other)
 

Static Public Member Functions

static runtime::Module Load (const std::string &code, const runtime::Module lib)
 Load the saved VM executable. More...
 
- Static Public Member Functions inherited from tvm::runtime::Object
static std::string TypeIndex2Key (uint32_t tindex)
 Get the type key of the corresponding index from runtime. More...
 
static size_t TypeIndex2KeyHash (uint32_t tindex)
 Get the type key hash of the corresponding index from runtime. More...
 
static uint32_t TypeKey2Index (const std::string &key)
 Get the type index of the corresponding key from runtime. More...
 
static uint32_t _GetOrAllocRuntimeTypeIndex ()
 
static uint32_t RuntimeTypeIndex ()
 

Public Attributes

std::vector< std::pair< Device, std::string > > virtual_devices
 The (compile-time, virtual) devices corresponding to each device index. This vector contains a pair Device and its memory_scope. More...
 
int host_device_index = -1
 The device index corresponding to the 'host' device. That will hold and evaluate shape-related data and code. More...
 
std::vector< ObjectRefconstants
 The global constant array. More...
 
std::vector< Stringlate_bound_constant_names
 For each constant index the name of the late-bound constant, or null if constant is immediate. Only populated after loading executable but before loading late-bound constants. More...
 
std::unordered_map< std::string, Indexglobal_map
 A map from globals (as strings) to their index in the Relay function map. More...
 
std::unordered_map< std::string, Indexprimitive_map
 A mapping from the packed function's global name (as string) to the index that corresponds to the position of the packed_funcs list in a VirtualMachine object. More...
 
std::map< Index, Map< String, ObjectRef > > op_attrs
 The structural hashes of the operators in this function. More...
 
std::vector< VMFunctionfunctions
 The virtual machine's function table. More...
 
std::vector< Indexconst_device_indexes
 The index of the device holding each constant. More...
 

Additional Inherited Members

- Public Types inherited from tvm::runtime::Object
typedef void(* FDeleter) (Object *self)
 Object deleter. More...
 
using RefCounterType = std::atomic< int32_t >
 
- Static Public Attributes inherited from tvm::runtime::ModuleNode
static constexpr const uint32_t _type_index = TypeIndex::kRuntimeModule
 
static constexpr const char * _type_key = "runtime.Module"
 
- Static Public Attributes inherited from tvm::runtime::Object
static constexpr const char * _type_key = "runtime.Object"
 
static constexpr bool _type_final = false
 
static constexpr uint32_t _type_child_slots = 0
 
static constexpr bool _type_child_slots_can_overflow = true
 
static constexpr bool _type_has_method_visit_attrs = true
 
static constexpr bool _type_has_method_sequal_reduce = false
 
static constexpr bool _type_has_method_shash_reduce = false
 
static constexpr uint32_t _type_index = TypeIndex::kDynamic
 
- Protected Member Functions inherited from tvm::runtime::Object
void IncRef ()
 developer function, increases reference counter. More...
 
void DecRef ()
 developer function, decrease reference counter. More...
 
- Static Protected Member Functions inherited from tvm::runtime::Object
static uint32_t GetOrAllocRuntimeTypeIndex (const std::string &key, uint32_t static_tindex, uint32_t parent_tindex, uint32_t type_child_slots, bool type_child_slots_can_overflow)
 Get the type index using type key. More...
 
- Protected Attributes inherited from tvm::runtime::ModuleNode
std::vector< Moduleimports_
 The modules this module depend on. More...
 
- Protected Attributes inherited from tvm::runtime::Object
uint32_t type_index_ {0}
 Type index(tag) that indicates the type of the object. More...
 
RefCounterType ref_counter_ {0}
 The internal reference counter. More...
 
FDeleter deleter_ = nullptr
 deleter of this object to enable customized allocation. If the deleter is nullptr, no deletion will be performed. The creator of the object must always set the deleter field properly. More...
 

Detailed Description

The executable emitted by the VM compiler.

The executable contains information (e.g. data in different memory regions) to run in a virtual machine.

Constructor & Destructor Documentation

◆ ~Executable()

virtual tvm::runtime::vm::Executable::~Executable ( )
inlinevirtual

Member Function Documentation

◆ GetBytecode()

std::string tvm::runtime::vm::Executable::GetBytecode ( ) const

Get the serialized form of the functions. This is essentially bytecode serialization.

Returns
The serialized vm bytecode.
Note
The bytecode is in the following format: func_name reg_file_size num_instructions param1 param2 ... paramM instruction1 instruction2 ... instructionN

Each instruction is printed in the following format: opcode num_fields field1 ... fieldX # The text format.

Serializing an Instruction requires us to deal with the bytecode. Each line of the instructions could be serialized as the following format: hash, opcode, f1, f2, ..., fX, field with variable length

  1. hash: the hash of the instruction. This number will be used to help us validate if an instruction is well-formed during deserialization.
  2. opcode: the opcode code of the instruction.
  3. f1, f2, ..., fX. These fields together represent the fixed fields in an instruction, e.g., from and dst fields of a Move instruction. For example, DLDataType will be unpacked into three fields (code, bits, lanes).
  4. The rest of the line indicates the field with variable length, e.g., the shape of a tensor, the args used by an InvokPacked instruction, etc.

The field starting from # is only used for debugging. The serialized code doesn't contain it, therefore the deserializer doens't need to handle it.

◆ GetConstants()

std::string tvm::runtime::vm::Executable::GetConstants ( ) const

Returns a description of all the constants in the executable in human-readable format. Intended for debugging and diff-testing.

◆ GetFunctionArity()

int tvm::runtime::vm::Executable::GetFunctionArity ( std::string  func) const

Get the arity of the VMFunction.

Parameters
funcFunction name.
Returns
The number of parameters.

◆ GetFunctionParameterName()

std::string tvm::runtime::vm::Executable::GetFunctionParameterName ( std::string  func,
int  index 
) const

Get the parameter name given the function name and parameter index.

Parameters
funcFunction name.
indexParameter index.
Returns
The parameter name.

◆ GetLateBoundConstants()

Map<String, NDArray> tvm::runtime::vm::Executable::GetLateBoundConstants ( int64_t  byte_limit)

Get a map of all constants with larger that byte_limit in size.

◆ GetLib()

runtime::Module tvm::runtime::vm::Executable::GetLib ( ) const

Get the lib module in an executable. Users have the flexibility to call export_library from the frontend to save the library to disk.

Returns
The runtime module that contains the hardware dependent code.

◆ GetPrimitives()

std::string tvm::runtime::vm::Executable::GetPrimitives ( ) const

Returns a description of all the 'primitive' (ie PackedFuncs) in the executable in human-readable format. These correspond either to PrimFuncs we've compiled locally, or functions compiled by a BYOC external codegen. Intended for debugging and diff-testing.

◆ GetPropertyMask()

int tvm::runtime::vm::Executable::GetPropertyMask ( ) const
inlinefinalvirtual

Get the property of the runtime module .

Reimplemented from tvm::runtime::ModuleNode.

◆ GetVirtualDevices()

std::string tvm::runtime::vm::Executable::GetVirtualDevices ( ) const

Returns a description of all the (virtual) devices in the executable in human-readable format. Intended for debugging and diff-testing.

◆ GetVMFunctionWithName()

const VMFunction& tvm::runtime::vm::Executable::GetVMFunctionWithName ( const std::string &  func_name) const

Get VMFunction.

Parameters
func_nameThe function's name.
Returns
VMFunction.

◆ Load()

static runtime::Module tvm::runtime::vm::Executable::Load ( const std::string &  code,
const runtime::Module  lib 
)
static

Load the saved VM executable.

Late-bound constants (if any) must then be loaded by LoadLateBoundConstantsFromBinary.

Parameters
codeThe bytecode in string.
libThe compiled runtime library.
Returns
exe The constructed executable.

◆ LoadLateBoundConstantsFromFile()

void tvm::runtime::vm::Executable::LoadLateBoundConstantsFromFile ( const std::string &  path)

As for LoadLateBoundConstantsFromStream, but load from file at path.

◆ LoadLateBoundConstantsFromMap()

void tvm::runtime::vm::Executable::LoadLateBoundConstantsFromMap ( Map< String, NDArray map)

Restores the late-bound constants for the executable (if any) from given map.

Must be called after Load but before any other methods if MoveLateBoundConstantsToBinary was used when saving. Otherwise can be ignored.

◆ LoadLateBoundConstantsFromStream()

void tvm::runtime::vm::Executable::LoadLateBoundConstantsFromStream ( dmlc::Stream *  stream)

Restores the late-bound constants for the executable (if any) from given byte-stream.

Must be called after Load but before any other methods if MoveLateBoundConstantsToBinary was used when saving. Otherwise can be ignored.

◆ MoveLateBoundConstantsToFile()

void tvm::runtime::vm::Executable::MoveLateBoundConstantsToFile ( const std::string &  path,
int64_t  byte_limit 
)

As for MoveLateBoundConstantsToStream, but save to file at path.

◆ MoveLateBoundConstantsToStream()

void tvm::runtime::vm::Executable::MoveLateBoundConstantsToStream ( dmlc::Stream *  stream,
int64_t  byte_limit 
)

Returns the late-bound constants for the executable (if any) as a byte-stream. Leaves the executable's late-bound constants map empty. Only constants who's byte tensor size is greater than or equal to byte_limit are marked as late-bound. byte_limit may be zero.

Must be called before SaveToBinary and friends if late-bound constants are desired. Otherwise can be ignore.

◆ Save()

TVMByteArray tvm::runtime::vm::Executable::Save ( )

Serialize the executable into global section, constant section, and code section. This object must outlive the returned byte array.

Late-bound constants (if any) must have already been saved by MoveLateBoundConstantsToBinary.

Returns
The binary representation of the VM.

◆ SaveToBinary()

void tvm::runtime::vm::Executable::SaveToBinary ( dmlc::Stream *  stream)
finalvirtual

Write the Executable to the binary stream in serialized form.

Late-bound constants (if any) must have already been saved by MoveLateBoundConstantsToBinary.

Parameters
streamThe binary stream to save the executable to.

Reimplemented from tvm::runtime::ModuleNode.

◆ SaveToFile()

void tvm::runtime::vm::Executable::SaveToFile ( const String path,
const String format 
)
finalvirtual

Write the Executable to the provided path as a file containing its serialized content.

Late-bound constants (if any) must have already been saved by MoveLateBoundConstantsToBinary.

Parameters
pathThe path to write the serialized data to.
formatThe format of the serialized blob.

Reimplemented from tvm::runtime::ModuleNode.

◆ SetLib()

void tvm::runtime::vm::Executable::SetLib ( const runtime::Module lib)

Set the lib module in an executable.

This allows us to do partial initialization in the case of (de|ser)ialization cases. This method also ensures correct initialization of library ensuring we only Import a single library.

NB: This also provides some abstraction over how libraries are stored as there are plans to iterate on the way runtime::Module works in the backend of the compiler.

◆ Stats()

std::string tvm::runtime::vm::Executable::Stats ( ) const

Print the detailed statistics of the given code, i.e. number of globls and constants, etc.

◆ TVM_MODULE_VTABLE_BEGIN()

tvm::runtime::vm::Executable::TVM_MODULE_VTABLE_BEGIN ( "VMExecutable"  )

◆ TVM_MODULE_VTABLE_END()

tvm::runtime::vm::Executable::TVM_MODULE_VTABLE_END ( )

◆ TVM_MODULE_VTABLE_ENTRY() [1/14]

tvm::runtime::vm::Executable::TVM_MODULE_VTABLE_ENTRY ( "get_bytecode"  ,
&Executable::GetBytecode   
)

◆ TVM_MODULE_VTABLE_ENTRY() [2/14]

tvm::runtime::vm::Executable::TVM_MODULE_VTABLE_ENTRY ( "get_constants"  ,
&Executable::GetConstants   
)

◆ TVM_MODULE_VTABLE_ENTRY() [3/14]

tvm::runtime::vm::Executable::TVM_MODULE_VTABLE_ENTRY ( "get_function_arity"  ,
&Executable::GetFunctionArity   
)

◆ TVM_MODULE_VTABLE_ENTRY() [4/14]

tvm::runtime::vm::Executable::TVM_MODULE_VTABLE_ENTRY ( "get_function_param_name"  ,
&Executable::GetFunctionParameterName   
)

◆ TVM_MODULE_VTABLE_ENTRY() [5/14]

tvm::runtime::vm::Executable::TVM_MODULE_VTABLE_ENTRY ( "get_late_bound_consts"  ,
&Executable::GetLateBoundConstants   
)

◆ TVM_MODULE_VTABLE_ENTRY() [6/14]

tvm::runtime::vm::Executable::TVM_MODULE_VTABLE_ENTRY ( "get_lib"  ,
&Executable::GetLib   
)

◆ TVM_MODULE_VTABLE_ENTRY() [7/14]

tvm::runtime::vm::Executable::TVM_MODULE_VTABLE_ENTRY ( "get_primitives"  ,
&Executable::GetPrimitives   
)

◆ TVM_MODULE_VTABLE_ENTRY() [8/14]

tvm::runtime::vm::Executable::TVM_MODULE_VTABLE_ENTRY ( "get_stats"  ,
&Executable::Stats   
)

◆ TVM_MODULE_VTABLE_ENTRY() [9/14]

tvm::runtime::vm::Executable::TVM_MODULE_VTABLE_ENTRY ( "get_virtual_devices"  ,
&Executable::GetVirtualDevices   
)

◆ TVM_MODULE_VTABLE_ENTRY() [10/14]

tvm::runtime::vm::Executable::TVM_MODULE_VTABLE_ENTRY ( "load_late_bound_consts"  ,
&Executable::LoadLateBoundConstantsFromFile   
)

◆ TVM_MODULE_VTABLE_ENTRY() [11/14]

tvm::runtime::vm::Executable::TVM_MODULE_VTABLE_ENTRY ( "load_late_bound_consts_from_map"  ,
&Executable::LoadLateBoundConstantsFromMap   
)

◆ TVM_MODULE_VTABLE_ENTRY() [12/14]

tvm::runtime::vm::Executable::TVM_MODULE_VTABLE_ENTRY ( "move_late_bound_consts"  ,
&Executable::MoveLateBoundConstantsToFile   
)

◆ TVM_MODULE_VTABLE_ENTRY() [13/14]

tvm::runtime::vm::Executable::TVM_MODULE_VTABLE_ENTRY ( "save"  ,
&Executable::Save   
)

◆ TVM_MODULE_VTABLE_ENTRY() [14/14]

tvm::runtime::vm::Executable::TVM_MODULE_VTABLE_ENTRY ( "vm_load_executable"  ,
&Executable::VMLoadExecutable   
)

◆ VMLoadExecutable()

Module tvm::runtime::vm::Executable::VMLoadExecutable ( )

Creates a VM that loads this as the executable.

Member Data Documentation

◆ const_device_indexes

std::vector<Index> tvm::runtime::vm::Executable::const_device_indexes

The index of the device holding each constant.

◆ constants

std::vector<ObjectRef> tvm::runtime::vm::Executable::constants

The global constant array.

LoadConst instructions indexes are w.r.t. this vector. Late-bound constants are removed from this table after saving late-bound constants.

◆ functions

std::vector<VMFunction> tvm::runtime::vm::Executable::functions

The virtual machine's function table.

◆ global_map

std::unordered_map<std::string, Index> tvm::runtime::vm::Executable::global_map

A map from globals (as strings) to their index in the Relay function map.

◆ host_device_index

int tvm::runtime::vm::Executable::host_device_index = -1

The device index corresponding to the 'host' device. That will hold and evaluate shape-related data and code.

◆ late_bound_constant_names

std::vector<String> tvm::runtime::vm::Executable::late_bound_constant_names

For each constant index the name of the late-bound constant, or null if constant is immediate. Only populated after loading executable but before loading late-bound constants.

◆ op_attrs

std::map<Index, Map<String, ObjectRef> > tvm::runtime::vm::Executable::op_attrs

The structural hashes of the operators in this function.

◆ primitive_map

std::unordered_map<std::string, Index> tvm::runtime::vm::Executable::primitive_map

A mapping from the packed function's global name (as string) to the index that corresponds to the position of the packed_funcs list in a VirtualMachine object.

◆ virtual_devices

std::vector<std::pair<Device, std::string> > tvm::runtime::vm::Executable::virtual_devices

The (compile-time, virtual) devices corresponding to each device index. This vector contains a pair Device and its memory_scope.


The documentation for this class was generated from the following file: