nnpack.md 3.85 KB
Newer Older
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
<!--- Licensed to the Apache Software Foundation (ASF) under one -->
<!--- or more contributor license agreements.  See the NOTICE file -->
<!--- distributed with this work for additional information -->
<!--- regarding copyright ownership.  The ASF licenses this file -->
<!--- to you under the Apache License, Version 2.0 (the -->
<!--- "License"); you may not use this file except in compliance -->
<!--- with the License.  You may obtain a copy of the License at -->

<!---   http://www.apache.org/licenses/LICENSE-2.0 -->

<!--- Unless required by applicable law or agreed to in writing, -->
<!--- software distributed under the License is distributed on an -->
<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
<!--- KIND, either express or implied.  See the License for the -->
<!--- specific language governing permissions and limitations -->
<!--- under the License. -->

18
# NNPACK Contrib Installation
19

20 21 22 23 24
[NNPACK](https://github.com/Maratyszcza/NNPACK) is an acceleration package
for neural network computations, which can run on x86-64, ARMv7, or ARM64 architecture CPUs.
Using NNPACK, higher-level libraries like _MXNet_ can speed up
the execution on multi-core CPU computers, including laptops and mobile devices.

25
***Note***: AS TVM already has natively tuned schedules, NNPACK is here mainly for reference and comparison purpose.
26 27 28 29 30
For regular use prefer native tuned TVM implementation.

_TVM_ supports NNPACK for forward propagation (inference only) in convolution, max-pooling, and fully-connected layers.
In this document, we give a high level overview of how to use NNPACK with _TVM_.

31
## Conditions
32 33 34 35 36 37 38 39
The underlying implementation of NNPACK utilizes several acceleration methods,
including [fft](https://arxiv.org/abs/1312.5851) and [winograd](https://arxiv.org/abs/1509.09308).
These algorithms work better on some special `batch size`, `kernel size`, and `stride` settings than on other,
so depending on the context, not all convolution, max-pooling, or fully-connected layers can be powered by NNPACK.
When favorable conditions for running NNPACKS are not met,

NNPACK only supports Linux and OS X systems. Windows is not supported at present.

40
## Build/Install NNPACK
41 42 43

If the trained model meets some conditions of using NNPACK,
you can build TVM with NNPACK support.
44
Follow these simple steps:
45 46 47 48
* Build NNPACK shared library with the following commands. _TVM_ will link NNPACK dynamically.

Note: The following NNPACK installation instructions have been tested on Ubuntu 16.04.

49
### Build [Ninja](https://ninja-build.org/)
50 51 52 53 54 55 56 57

NNPACK need a recent version of Ninja. So we need to install ninja from source.
```bash
git clone git://github.com/ninja-build/ninja.git
cd ninja
./configure.py --bootstrap
```

58
Set the environment variable PATH to tell bash where to find the ninja executable. For example, assume we cloned ninja on the home directory ~. then we can added the following line in ~/.bashrc.
59 60 61 62
```bash
export PATH="${PATH}:~/ninja"
```

63
### Build [NNPACK](https://github.com/Maratyszcza/NNPACK)
64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84

The new CMAKE version of NNPACK download [Peach](https://github.com/Maratyszcza/PeachPy) and other dependencies alone

```bash
git clone --recursive https://github.com/Maratyszcza/NNPACK.git
cd NNPACK
# Add PIC option in CFLAG and CXXFLAG to build NNPACK shared library
sed -i "s|gnu99|gnu99 -fPIC|g" CMakeLists.txt
sed -i "s|gnu++11|gnu++11 -fPIC|g" CMakeLists.txt
mkdir build
cd build
# Generate ninja build rule and add shared library in configuration
cmake -G Ninja -D BUILD_SHARED_LIBS=ON ..
ninja
sudo ninja install

# Add NNPACK lib folder in your ldconfig
echo "/usr/local/lib" > /etc/ld.so.conf.d/nnpack.conf
sudo ldconfig
```

85
## Build TVM with NNPACK support
86 87

```bash
88
git clone --recursive https://github.com/apache/incubator-tvm tvm
89 90
```

91
* Set `set(USE_NNPACK ON)` in config.cmake.
92 93 94 95 96 97 98
* Set `NNPACK_PATH` to the $(YOUR_NNPACK_INSTALL_PATH)

after configuration use `make` to build TVM

```bash
make
```