.. DO NOT EDIT. THIS FILE WAS AUTOMATICALLY GENERATED BY .. TVM'S MONKEY-PATCHED VERSION OF SPHINX-GALLERY. TO MAKE .. CHANGES, EDIT THE SOURCE PYTHON FILE: .. "how_to/work_with_microtvm/micro_tvmc.py" .. only:: html .. note:: :class: sphx-glr-download-link-note This tutorial can be used interactively with Google Colab! You can also click :ref:`here ` to run the Jupyter notebook locally. .. image:: https://raw.githubusercontent.com/tlc-pack/web-data/main/images/utilities/colab_button.svg :align: center :target: https://colab.research.google.com/github/apache/tvm-site/blob/asf-site/docs/_downloads/6e511f5a8ddbf12f2fca2dfadc0cc4a9/micro_tvmc.ipynb :width: 300px .. rst-class:: sphx-glr-example-title .. _sphx_glr_how_to_work_with_microtvm_micro_tvmc.py: .. _tutorial-micro-cli-tool: 1. microTVM CLI Tool ==================== **Author**: `Mehrdad Hessar `_ This tutorial explains how to compile a tiny model for a micro device, build a program on Zephyr platform to execute this model, flash the program and run the model all using `tvmc micro` command. You need to install python and Zephyr dependencies before processing with this tutorial. .. GENERATED FROM PYTHON SOURCE LINES 34-36 .. include:: ../../../../gallery/how_to/work_with_microtvm/install_dependencies.rst .. GENERATED FROM PYTHON SOURCE LINES 39-41 .. include:: ../../../../gallery/how_to/work_with_microtvm/install_zephyr.rst .. GENERATED FROM PYTHON SOURCE LINES 45-72 Using TVMC Micro ########################################################### TVMC is a command-line tool which is installed as a part of TVM Python packages. Accessing this package varies based on your machine setup. In many cases, you can use the ``tvmc`` command directly. Alternatively, if you have TVM as a Python module on your ``$PYTHONPATH``, you can access this driver with ``python -m tvm.driver.tvmc`` command. This tutorial will use TVMC command as ``tvmc`` for simplicity. To check if you have TVMC command installed on your machine, you can run: .. code-block:: bash tvmc --help To compile a model for microtvm we use ``tvmc compile`` subcommand. The output of this command is used in next steps with ``tvmc micro`` subcommands. You can check the availability of TVMC Micro using: .. code-block:: bash tvmc micro --help The main tasks that you can perform using ``tvmc micro`` are ``create``, ``build`` and ``flash``. To read about specific options under a givern subcommand, use ``tvmc micro --help``. We will use each subcommand in this tutorial. .. GENERATED FROM PYTHON SOURCE LINES 74-86 Obtain a Tiny Model ########################################################### For this tutorial, we will use Micro Speech model from tflite micro. Micro Speech is a Depthwise Convolution Layer model to recognize keywords in speech. For this tutorial we will be using the model in tflite format. .. code-block:: bash wget https://github.com/tensorflow/tflite-micro/raw/a56087ffa2703b4d5632f024a8a4c899815c31bb/tensorflow/lite/micro/examples/micro_speech/micro_speech.tflite .. GENERATED FROM PYTHON SOURCE LINES 88-114 Compiling a TFLite model to a Model Library Format ########################################################### Model Library Format (MLF) is an output format that TVM provides for micro targets. MLF is a tarball containing a file for each piece of the TVM compiler output which can be used on micro targets outside TVM environment. Read more about :ref:`Model Library Format `. Here, we generate a MLF file for ``qemu_x86`` Zephyr board. You can chooses `aot` or `graph` executor type to run this tutorial, however, we recommend to use `aot` for microTVM targets since `aot` uses ahead of time compilation with static memory allocation. To generate MLF output for the ``micro_speech`` tflite model: .. code-block:: bash tvmc compile micro_speech.tflite \ --target='c -keys=cpu -model=host' \ --runtime=crt \ --runtime-crt-system-lib 1 \ --executor='aot' \ --output model.tar \ --output-format mlf \ --pass-config tir.disable_vectorize=1 This will generate a ``model.tar`` file which contains TVM compiler output files. To run this command for a different Zephyr device, you need to update ``target``. For instance, for ``nrf5340dk_nrf5340_cpuapp`` board the target is ``--target='c -keys=cpu -model=nrf5340dk'``. .. GENERATED FROM PYTHON SOURCE LINES 117-144 Create a Zephyr Project Using Model Library Format ########################################################### To generate a Zephyr project we use TVM Micro subcommand ``create``. We pass the MLF format and the path for the project to ``create`` subcommand along with project options. Project options for each platform (Zephyr/Arduino) are defined in their Project API server file. To build Zephyr project for a different Zephyr board, change ``zephyr_board`` project option. To generate Zephyr project, run: .. code-block:: bash tvmc micro create \ project \ model.tar \ zephyr \ --project-option project_type=host_driven board=qemu_x86 This will generate a ``Host-Driven`` Zephyr project for ``qemu_x86`` Zephyr board. In Host-Driven template project, the Graph Executor will run on host and perform the model execution on Zephyr device by issuing commands to the device using an RPC mechanism. Read more about `Host-Driven Execution `_. To get more information about TVMC Micro ``create`` subcommand: .. code-block:: bash tvmc micro create --help .. GENERATED FROM PYTHON SOURCE LINES 146-169 Build and Flash Zephyr Project Using TVMC Micro ########################################################### Next step is to build the Zephyr project which includes TVM generated code for running the tiny model, Zephyr template code to run a model in Host-Driven mode and TVM runtime source/header files. To build the project: .. code-block:: bash tvmc micro build \ project \ zephyr This will build the project in ``project`` directory and generates binary files under ``project/build``. Next, we flash the Zephyr binary file to Zephyr device. For ``qemu_x86`` Zephyr board this step does not actually perform any action since QEMU will be used, however you need this step for physical hardware. .. code-block:: bash tvmc micro flash \ project \ zephyr .. GENERATED FROM PYTHON SOURCE LINES 171-188 Run Tiny Model on Micro Target ########################################################### After flashing the device, the compiled model and TVM RPC server are programmed on the device. The Zephyr board is waiting for host to open a communication channel. MicroTVM devices typicall communicate using a serial communication (UART). To run the flashed model on the device using TVMC, we use ``tvmc run`` subcommand and pass ``--device micro`` to specify the device type. This command will open a communication channel, set input values using ``Graph Executor`` on host and run full model on the device. Then it gets output from the device. .. code-block:: bash tvmc run \ --device micro \ project \ --fill-mode ones \ --print-top 4 .. GENERATED FROM PYTHON SOURCE LINES 190-202 Specifically, this command sets the input of the model to all ones and shows the four values of the output with their indices. .. code-block:: bash # Output: # INFO:__main__:b'[100%] [QEMU] CPU: qemu32,+nx,+pae\n' # remote: microTVM Zephyr runtime - running # INFO:__main__:b'[100%] Built target run\n' # [[ 3 2 1 0] # [ 113 -120 -121 -128]] .. _sphx_glr_download_how_to_work_with_microtvm_micro_tvmc.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: micro_tvmc.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: micro_tvmc.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_