nnvm_top.rst 9.37 KB
Newer Older
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
..  Licensed to the Apache Software Foundation (ASF) under one
    or more contributor license agreements.  See the NOTICE file
    distributed with this work for additional information
    regarding copyright ownership.  The ASF licenses this file
    to you under the Apache License, Version 2.0 (the
    "License"); you may not use this file except in compliance
    with the License.  You may obtain a copy of the License at

..    http://www.apache.org/licenses/LICENSE-2.0

..  Unless required by applicable law or agreed to in writing,
    software distributed under the License is distributed on an
    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    KIND, either express or implied.  See the License for the
    specific language governing permissions and limitations
    under the License.

18 19
NNVM Core Tensor Operators
==========================
20

Tianqi Chen committed
21
This page contains the list of core tensor operator primitives pre-defined in NNVM.
22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
The core tensor operator primitives(``nnvm.top``) covers typical workloads in deep learning.
They can represent workloads in front-end frameworks, and provide basic building blocks for optimization.
Since deep learning is a fast evolving field and it is that possible to have operators that are not in here.
NNVM is designed for this problem and can easily new operators without changing the core library.

.. note::

   Each operator node in the graph IR contains the following two kinds of parameters.

   - inputs: positional list of input tensors
   - attrs: attributes about operator(e.g. kernel_size in conv2d)

   This document lists both inputs and attributes in the parameter field.  You can distinguish them by the marked type. The inputs are of type Tensor, while the rest parameters are attributes.
   To construct the graph with NNVM python API, a user can pass in the input Tensors as positional arguments, and attributes as keyword arguments.


Overview of Operators
---------------------
40
**Level 1: Basic Operators**
41

42
This level enables fully connected multi-layer perceptron.
43 44 45 46 47 48

.. autosummary::
   :nosignatures:

   nnvm.symbol.dense
   nnvm.symbol.relu
49
   nnvm.symbol.prelu
50 51 52 53
   nnvm.symbol.tanh
   nnvm.symbol.sigmoid
   nnvm.symbol.exp
   nnvm.symbol.log
54
   nnvm.symbol.sqrt
55 56 57
   nnvm.symbol.logical_and
   nnvm.symbol.logical_or
   nnvm.symbol.logical_not
58 59 60 61
   nnvm.symbol.elemwise_add
   nnvm.symbol.elemwise_sub
   nnvm.symbol.elemwise_mul
   nnvm.symbol.elemwise_div
62
   nnvm.symbol.elemwise_sum
63 64
   nnvm.symbol.elemwise_mod
   nnvm.symbol.elemwise_pow
65 66
   nnvm.symbol.flatten
   nnvm.symbol.concatenate
67
   nnvm.symbol.expand_dims
Xingjian Shi committed
68
   nnvm.symbol.squeeze
69 70 71 72 73
   nnvm.symbol.split
   nnvm.symbol.dropout
   nnvm.symbol.batch_norm
   nnvm.symbol.softmax
   nnvm.symbol.log_softmax
Yuwei Hu committed
74
   nnvm.symbol.pad
75
   nnvm.symbol.block_grad
76 77 78 79 80 81 82 83
   nnvm.symbol.matmul
   nnvm.symbol.resize
   nnvm.symbol.upsampling
   nnvm.symbol.take
   nnvm.symbol.l2_normalize
   nnvm.symbol.flip
   nnvm.symbol.lrn
   nnvm.symbol.where
84
   nnvm.symbol.gather_nd
85 86 87 88


**Level 2: Convolutions**

89 90
This level enables typical convnet models.

91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109
.. autosummary::
   :nosignatures:

   nnvm.symbol.conv2d
   nnvm.symbol.conv2d_transpose
   nnvm.symbol.max_pool2d
   nnvm.symbol.avg_pool2d
   nnvm.symbol.global_max_pool2d
   nnvm.symbol.global_avg_pool2d


**Level 3: Additional Tensor Ops**

.. autosummary::
   :nosignatures:

   nnvm.symbol.reshape
   nnvm.symbol.copy
   nnvm.symbol.negative
110 111 112 113
   nnvm.symbol.floor
   nnvm.symbol.ceil
   nnvm.symbol.round
   nnvm.symbol.trunc
114
   nnvm.symbol.abs
115 116 117 118 119 120 121 122 123
   nnvm.symbol.leaky_relu
   nnvm.symbol.__add_scalar__
   nnvm.symbol.__sub_scalar__
   nnvm.symbol.__rsub_scalar__
   nnvm.symbol.__mul_scalar__
   nnvm.symbol.__div_scalar__
   nnvm.symbol.__rdiv_scalar__
   nnvm.symbol.__pow_scalar__
   nnvm.symbol.__rpow_scalar__
124 125
   nnvm.symbol.__lshift_scalar__
   nnvm.symbol.__rshift_scalar__
126

127

128 129 130 131 132 133 134 135 136 137
**Level 4: Broadcast and Reductions**

.. autosummary::
   :nosignatures:

   nnvm.symbol.transpose
   nnvm.symbol.broadcast_to
   nnvm.symbol.sum
   nnvm.symbol.min
   nnvm.symbol.max
138 139
   nnvm.symbol.mean
   nnvm.symbol.prod
140 141 142 143
   nnvm.symbol.broadcast_add
   nnvm.symbol.broadcast_sub
   nnvm.symbol.broadcast_mul
   nnvm.symbol.broadcast_div
Yao Wang committed
144 145 146 147 148 149 150 151 152 153 154
   nnvm.symbol.clip
   nnvm.symbol.greater
   nnvm.symbol.less
   nnvm.symbol.expand_like
   nnvm.symbol.reshape_like
   nnvm.symbol.full
   nnvm.symbol.full_like
   nnvm.symbol.ones
   nnvm.symbol.ones_like
   nnvm.symbol.zeros
   nnvm.symbol.zeros_like
155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183
   nnvm.symbol.slice_like
   nnvm.symbol.strided_slice
   nnvm.symbol.argmax
   nnvm.symbol.argmin
   nnvm.symbol.collapse_sum
   nnvm.symbol.broadcast_equal
   nnvm.symbol.broadcast_greater_equal
   nnvm.symbol.broadcast_greater
   nnvm.symbol.broadcast_left_shift
   nnvm.symbol.broadcast_less_equal
   nnvm.symbol.broadcast_less
   nnvm.symbol.broadcast_max
   nnvm.symbol.broadcast_min
   nnvm.symbol.broadcast_mod
   nnvm.symbol.broadcast_not_equal
   nnvm.symbol.broadcast_pow
   nnvm.symbol.broadcast_right_shift


**Level 5: Vision Operators**

.. autosummary::
   :nosignatures:

   nnvm.symbol.multibox_prior
   nnvm.symbol.multibox_transform_loc
   nnvm.symbol.nms
   nnvm.symbol.yolo_region
   nnvm.symbol.yolo_reorg
184

185 186
Detailed Definitions
--------------------
187 188
.. autofunction:: nnvm.symbol.dense
.. autofunction:: nnvm.symbol.relu
189
.. autofunction:: nnvm.symbol.prelu
190 191 192 193
.. autofunction:: nnvm.symbol.tanh
.. autofunction:: nnvm.symbol.sigmoid
.. autofunction:: nnvm.symbol.exp
.. autofunction:: nnvm.symbol.log
194
.. autofunction:: nnvm.symbol.sqrt
195 196 197
.. autofunction:: nnvm.symbol.logical_and
.. autofunction:: nnvm.symbol.logical_or
.. autofunction:: nnvm.symbol.logical_not
198 199 200 201
.. autofunction:: nnvm.symbol.elemwise_add
.. autofunction:: nnvm.symbol.elemwise_sub
.. autofunction:: nnvm.symbol.elemwise_mul
.. autofunction:: nnvm.symbol.elemwise_div
202
.. autofunction:: nnvm.symbol.elemwise_sum
203 204
.. autofunction:: nnvm.symbol.elemwise_mod
.. autofunction:: nnvm.symbol.elemwise_pow
205 206
.. autofunction:: nnvm.symbol.flatten
.. autofunction:: nnvm.symbol.concatenate
207
.. autofunction:: nnvm.symbol.expand_dims
Xingjian Shi committed
208
.. autofunction:: nnvm.symbol.squeeze
209 210 211 212 213
.. autofunction:: nnvm.symbol.split
.. autofunction:: nnvm.symbol.dropout
.. autofunction:: nnvm.symbol.batch_norm
.. autofunction:: nnvm.symbol.softmax
.. autofunction:: nnvm.symbol.log_softmax
Yuwei Hu committed
214
.. autofunction:: nnvm.symbol.pad
215
.. autofunction:: nnvm.symbol.block_grad
216 217 218 219 220 221 222 223
.. autofunction:: nnvm.symbol.matmul
.. autofunction:: nnvm.symbol.resize
.. autofunction:: nnvm.symbol.upsampling
.. autofunction:: nnvm.symbol.take
.. autofunction:: nnvm.symbol.l2_normalize
.. autofunction:: nnvm.symbol.flip
.. autofunction:: nnvm.symbol.lrn
.. autofunction:: nnvm.symbol.where
224
.. autofunction:: nnvm.symbol.gather_nd
225 226 227 228 229 230 231 232 233 234 235

.. autofunction:: nnvm.symbol.conv2d
.. autofunction:: nnvm.symbol.conv2d_transpose
.. autofunction:: nnvm.symbol.max_pool2d
.. autofunction:: nnvm.symbol.avg_pool2d
.. autofunction:: nnvm.symbol.global_max_pool2d
.. autofunction:: nnvm.symbol.global_avg_pool2d

.. autofunction:: nnvm.symbol.reshape
.. autofunction:: nnvm.symbol.copy
.. autofunction:: nnvm.symbol.negative
236 237 238 239
.. autofunction:: nnvm.symbol.floor
.. autofunction:: nnvm.symbol.ceil
.. autofunction:: nnvm.symbol.round
.. autofunction:: nnvm.symbol.trunc
240
.. autofunction:: nnvm.symbol.abs
241 242 243 244 245 246 247 248 249
.. autofunction:: nnvm.symbol.leaky_relu
.. autofunction:: nnvm.symbol.__add_scalar__
.. autofunction:: nnvm.symbol.__sub_scalar__
.. autofunction:: nnvm.symbol.__rsub_scalar__
.. autofunction:: nnvm.symbol.__mul_scalar__
.. autofunction:: nnvm.symbol.__div_scalar__
.. autofunction:: nnvm.symbol.__rdiv_scalar__
.. autofunction:: nnvm.symbol.__pow_scalar__
.. autofunction:: nnvm.symbol.__rpow_scalar__
250 251
.. autofunction:: nnvm.symbol.__lshift_scalar__
.. autofunction:: nnvm.symbol.__rshift_scalar__
252 253 254 255 256 257

.. autofunction:: nnvm.symbol.transpose
.. autofunction:: nnvm.symbol.broadcast_to
.. autofunction:: nnvm.symbol.sum
.. autofunction:: nnvm.symbol.min
.. autofunction:: nnvm.symbol.max
258 259
.. autofunction:: nnvm.symbol.mean
.. autofunction:: nnvm.symbol.prod
260 261 262 263
.. autofunction:: nnvm.symbol.broadcast_add
.. autofunction:: nnvm.symbol.broadcast_sub
.. autofunction:: nnvm.symbol.broadcast_mul
.. autofunction:: nnvm.symbol.broadcast_div
Yao Wang committed
264 265 266 267 268 269 270 271 272 273 274
.. autofunction:: nnvm.symbol.clip
.. autofunction:: nnvm.symbol.greater
.. autofunction:: nnvm.symbol.less
.. autofunction:: nnvm.symbol.expand_like
.. autofunction:: nnvm.symbol.reshape_like
.. autofunction:: nnvm.symbol.full
.. autofunction:: nnvm.symbol.full_like
.. autofunction:: nnvm.symbol.ones
.. autofunction:: nnvm.symbol.ones_like
.. autofunction:: nnvm.symbol.zeros
.. autofunction:: nnvm.symbol.zeros_like
275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297
.. autofunction:: nnvm.symbol.slice_like
.. autofunction:: nnvm.symbol.strided_slice
.. autofunction:: nnvm.symbol.argmax
.. autofunction:: nnvm.symbol.argmin
.. autofunction:: nnvm.symbol.collapse_sum
.. autofunction:: nnvm.symbol.broadcast_equal
.. autofunction:: nnvm.symbol.broadcast_greater_equal
.. autofunction:: nnvm.symbol.broadcast_greater
.. autofunction:: nnvm.symbol.broadcast_left_shift
.. autofunction:: nnvm.symbol.broadcast_less_equal
.. autofunction:: nnvm.symbol.broadcast_less
.. autofunction:: nnvm.symbol.broadcast_max
.. autofunction:: nnvm.symbol.broadcast_min
.. autofunction:: nnvm.symbol.broadcast_mod
.. autofunction:: nnvm.symbol.broadcast_not_equal
.. autofunction:: nnvm.symbol.broadcast_pow
.. autofunction:: nnvm.symbol.broadcast_right_shift

.. autofunction:: nnvm.symbol.multibox_prior
.. autofunction:: nnvm.symbol.multibox_transform_loc
.. autofunction:: nnvm.symbol.nms
.. autofunction:: nnvm.symbol.yolo_region
.. autofunction:: nnvm.symbol.yolo_reorg