README.md 2.41 KB
Newer Older
tqchen committed
1
# NNVM: Build deep learning system by parts
Tianqi Chen committed
2

Tianqi Chen committed
3 4 5
[![Build Status](https://travis-ci.org/dmlc/nnvm.svg?branch=master)](https://travis-ci.org/dmlc/nnvm)
[![GitHub license](http://dmlc.github.io/img/apache2.svg)](./LICENSE)

Tianqi Chen committed
6 7
NNVM is not a deep learning library. It is a modular,
decentralized and lightweight part to help build deep learning libraries.
Tianqi Chen committed
8

tqchen committed
9
## What is it
Tianqi Chen committed
10

tqchen committed
11
While most deep learning systems offer end to end solutions,
Tianqi Chen committed
12 13
it is interesting to  assemble a deep learning system by parts.
The goal is to enable user to customize optimizations, target platforms and set of operators they care about.
tqchen committed
14
We believe that the decentralized modular system is an interesting direction.
15

tqchen committed
16 17
The hope is that effective parts can be assembled together just like you assemble your own desktops.
So the customized deep learning solution can be minimax, minimum in terms of dependencies,
Wei Wu committed
18
while maximizing the users' need.
tqchen committed
19

20 21
NNVM offers one such part, it provides a generic way to do
computation graph optimization such as memory reduction, device allocation and more
Wei Wu committed
22
while being agnostic to the operator interface definition and how operators are executed.
Tianqi Chen committed
23
NNVM is inspired by LLVM, aiming to be a high level intermediate representation library
tqchen committed
24
for neural nets and computation graphs generation and optimizations.
tqchen committed
25

Tianqi Chen committed
26 27
See [Overview](docs/overview.md) for an introduction on what it provides.

Tianqi Chen committed
28 29 30
## Example
See [TinyFlow](https://github.com/tqchen/tinyflow) on how you can build a TensorFlow API with NNVM and Torch.
 
Tianqi Chen committed
31 32 33
## Why build learning system by parts

This is essentially ***Unix philosophy*** applied to machine learning system.
34 35

- Essential parts can be assembled in minimum way for embedding systems.
Tianqi Chen committed
36
- Developers can hack the parts they need and compose with other well defined parts.
37
- Decentralized modules enable new extensions creators to own their project
Wei Wu committed
38
  without creating a monolithic version.
39

Tianqi Chen committed
40
Deep learning system itself is not necessary one part, for example
Tianqi Chen committed
41
here are some relative independent parts that can be isolated
tqchen committed
42 43 44 45 46 47 48 49 50 51 52 53

- Computation graph definition, manipulation.
- Computation graph intermediate optimization.
- Computation graph execution.
- Operator kernel libraries.
- Imperative task scheduling and parallel task coordination.

We hope that there will be more modular parts in the future,
so system building can be fun and rewarding.

## Links

Tianqi Chen committed
54
[MXNet](https://github.com/dmlc/mxnet) is moving to NNVM as its intermediate
tqchen committed
55
representation layer for symbolic graphs.