Commit 6d4cf448 by Siva Committed by Tianqi Chen

[DOCS] Neural network Deployment Guide with System Module Mode #1523 (#1533)

parent 41d4dd6e
......@@ -116,3 +116,60 @@ int main()
return 0;
}
```
## Deploy as System Module
C++ additionally support deployment as system module.
This process need few additional options as given below to NNVM build.
- For target llvm append --system-lib as ```target=llvm --system-lib```
- For a GPU build (or non llvm) the additional option should be given to targat_host as ```target_host=llvm --system-lib```
Module export require additional options for not to compile but save as ```lib.export_library (path, fcompile=False)```
The output of above API is a tar compressed file containing object file ```(lib.o)``` and cpp source file ```(devc.cc)``` which embeds device blob. Thease two files should be compiled along with other files or objects while building c++ application.
Please refer to [Makefile](https://github.com/dmlc/tvm/tree/master/apps/howto_deploy/Makefile#L32) for a reference.
The c++ code to load this system module require the below change.
```cpp
// tvm module for compiled functions
tvm::runtime::Module mod_syslib = (*tvm::runtime::Registry::Get("module._GetSystemLib"))();
```
Based on the build environment the system object, device blob source should be included in the final executable. An example with bazel build is given below.
```bash
cc_library(
name = "host_module",
srcs = ["lib.o"],
alwayslink=1
)
cc_library(
name = "device_module",
srcs = ["devc.cc"],
alwayslink=1
)
cc_library(
name = "tvm_runtime",
srcs = ["libtvm_runtime_pack.cc"],
)
cc_binary(
name = "bazel_deploy",
srcs = ["cpp_deploy.cc"],
deps = [
":tvm_runtime", ":host_module", ":device_module"
],
linkopts = [ "-lpthread -ldl" ]
)
```
This build directive creates
- new library ```host_module``` out of ```lib.o```
- new library ```device_module``` out of ```devc.cc```
These intermediate modules can be used as a dependency to final deploy application.
In bazel ```alwayslink=1``` enforce embedding entire lib into application (even though it doesn't call any API from this module).
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment