NNPACK Contrib Installation¶
NNPACK is an acceleration package for neural network computations, which can run on x86-64, ARMv7, or ARM64 architecture CPUs. Using NNPACK, higher-level libraries like _MXNet_ can speed up the execution on multi-core CPU computers, including laptops and mobile devices.
AS TVM already has natively tuned schedules, NNPACK is here mainly for reference and comparison purpose. For regular use prefer native tuned TVM implementation.
TVM supports NNPACK for forward propagation (inference only) in convolution, max-pooling, and fully-connected layers. In this document, we give a high level overview of how to use NNPACK with TVM.
The underlying implementation of NNPACK utilizes several acceleration methods, including fft and winograd. These algorithms work better on some special batch size, kernel size, and stride settings than on other, so depending on the context, not all convolution, max-pooling, or fully-connected layers can be powered by NNPACK. When favorable conditions for running NNPACKS are not met,
NNPACK only supports Linux and OS X systems. Windows is not supported at present.
If the trained model meets some conditions of using NNPACK, you can build TVM with NNPACK support. Follow these simple steps:
build NNPACK shared library with the following commands. TVM will link NNPACK dynamically.
Note: The following NNPACK installation instructions have been tested on Ubuntu 16.04.
NNPACK need a recent version of Ninja. So we need to install ninja from source.
git clone git://github.com/ninja-build/ninja.git cd ninja ./configure.py --bootstrap
Set the environment variable PATH to tell bash where to find the ninja executable. For example, assume we cloned ninja on the home directory ~. then we can added the following line in ~/.bashrc.
The new CMAKE version of NNPACK download Peach and other dependencies alone
Note: at least on OS X, running ninja install below will overwrite googletest libraries installed in /usr/local/lib. If you build googletest again to replace the nnpack copy, be sure to pass -DBUILD_SHARED_LIBS=ON to cmake.
git clone --recursive https://github.com/Maratyszcza/NNPACK.git cd NNPACK # Add PIC option in CFLAG and CXXFLAG to build NNPACK shared library sed -i "s|gnu99|gnu99 -fPIC|g" CMakeLists.txt sed -i "s|gnu++11|gnu++11 -fPIC|g" CMakeLists.txt mkdir build cd build # Generate ninja build rule and add shared library in configuration cmake -G Ninja -D BUILD_SHARED_LIBS=ON .. ninja sudo ninja install # Add NNPACK lib folder in your ldconfig echo "/usr/local/lib" > /etc/ld.so.conf.d/nnpack.conf sudo ldconfig
Build TVM with NNPACK support¶
git clone --recursive https://github.com/apache/tvm tvm
Set set(USE_NNPACK ON) in config.cmake.
Set NNPACK_PATH to the $(YOUR_NNPACK_INSTALL_PATH)
after configuration use make to build TVM