|
tvm
|
This file serves as the entry point of Disco and defines key data structures and interfaces. More...
#include <tvm/runtime/container/shape_tuple.h>#include <tvm/runtime/object.h>#include <tvm/runtime/packed_func.h>#include <queue>#include <string>#include <utility>Go to the source code of this file.
Classes | |
| class | tvm::runtime::DRefObj |
| An object that exists on all workers. More... | |
| class | tvm::runtime::DRef |
| Managed reference to DRefObj. More... | |
| class | tvm::runtime::SessionObj |
| A Disco interactive session. It allows users to interact with the Disco command queue with various PackedFunc calling convention. More... | |
| class | tvm::runtime::Session |
| Managed reference to SessionObj. More... | |
| class | tvm::runtime::DiscoChannel |
| A bi-directional channel for controler-worker communication. This channel is primarily used to transfer control messages but not data. More... | |
| class | tvm::runtime::WorkerZeroData |
| A special communication channel between controler and worker-0, assuming they are always collocated in the same process. More... | |
Namespaces | |
| tvm | |
| runtime implementation for LibTorch/TorchScript. | |
| tvm::runtime | |
Enumerations | |
| enum class | tvm::runtime::DiscoAction : int32_t { tvm::runtime::kShutDown = 0 , tvm::runtime::kKillReg = 1 , tvm::runtime::kGetGlobalFunc = 2 , tvm::runtime::kCallPacked = 3 , tvm::runtime::kSyncWorker = 4 , tvm::runtime::kCopyFromWorker0 = 5 , tvm::runtime::kCopyToWorker0 = 6 , tvm::runtime::kDebugGetFromRemote = 7 , tvm::runtime::kDebugSetRegister = 8 } |
| All possible kinds of Disco commands. More... | |
Functions | |
| std::string | tvm::runtime::DiscoAction2String (DiscoAction action) |
Converts the enum class DiscoAction to string. More... | |
This file serves as the entry point of Disco and defines key data structures and interfaces.
Disco is a distributed runtime that consists of a controler and a cluster of workers. The controler is responsible for managing the workers by broadcasting commands to all the workers together, and the workers are responsible for executing the commands and. The controler and workers communicate with each other through a bi-directional channel.
Different from a generic system, Disco is designed to as "single-program-multiple-data" (SPMD) runtime, which means that all the workers execute the same instruction at the same time, but the data they are working on may be different. For example, in data parallelism, each worker may work on a different batches of the data, but they all execute the same set of instructions. Therefore, imagine there is a virtual machine that executes the program, the structures of workers' register files could be considered as "identical" (single program) although the values may differ (multiple data).
DRef. Following the design above, consider the program in SPMD in a virtual ISA, then each worker is a virtual machine instance to execute the ISA maintaining its own register file. The controler denotes each of their register files with a unique integer "register id", and the workers use this id to refer to the register file that resides on itself. DRef is a control-side object backed by such a register id. The data it contains is not assumed to be directly accessible by the controler, with an exception for worker-0, which is a special worker that is always co-located with the controler.
Worker-0. Worker-0 is a special worker that is always co-located with the controler. It is assumed that the controler can synchronize with and access the registers of worker-0. The Disco session provides multiple APIs to interact specifically with the worker-0. To shared data with other workers, a common paradigm in Disco is to copy data from the controler-side NDArray to the worker-0, and then copy it to other workers using primitives on the data plane, for example, broadcast and send.
Control plane. The controler broadcasts commands to all the workers as control signals. For example, the control may ask all workers to load a library or call a function respectively. Common control signals include: shutdown, retrievel a global PackedFunc, call packed function, etc. The controler is assumed to keep a message channel to each worker to implement the broadcast behavior, and the message channel may vary depends on usecases.
Data plane. The data channel is usually used to exchange data between workers, especially for tensor data which is usually large. For example, performing an allreduce operator for sharded matrix multiplication, or broadcasting for an input tensor. For efficiency, the data channel is usually backed by NCCL on NVIDIA GPUs, RCCL on AMD GPUs, or MPI on CPUs.
Session. A Disco session is a primary interface to interact with the Disco runtime, serving as a global context that manages the control and workers. It could be implemented as a multi-threaded with a pool of workers for single-node multi-gpu scenarios, or TCP sockets for workloads that span over a cluster of nodes.
Channel. Disco channel is a bi-directional communication channel between the controler and workers for exchanging control signals. It is no different from a generic RPC channel, but adopts TVM's PackedFunc calling convention to support polymorphic and variadic arguments.
1.9.1