relay_op.rst 8.9 KB
Newer Older
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
..  Licensed to the Apache Software Foundation (ASF) under one
    or more contributor license agreements.  See the NOTICE file
    distributed with this work for additional information
    regarding copyright ownership.  The ASF licenses this file
    to you under the Apache License, Version 2.0 (the
    "License"); you may not use this file except in compliance
    with the License.  You may obtain a copy of the License at

..    http://www.apache.org/licenses/LICENSE-2.0

..  Unless required by applicable law or agreed to in writing,
    software distributed under the License is distributed on an
    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    KIND, either express or implied.  See the License for the
    specific language governing permissions and limitations
    under the License.

18 19 20 21
Relay Core Tensor Operators
===========================

This page contains the list of core tensor operator primitives pre-defined in tvm.relay.
22 23 24
The core tensor operator primitives cover typical workloads in deep learning.
They can represent workloads in front-end frameworks and provide basic building blocks for optimization.
Since deep learning is a fast evolving field, it is possible to have operators that are not in here.
25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43


.. note::

   This document will directly list the function signature of
   these operators in the python frontend.


Overview of Operators
---------------------
**Level 1: Basic Operators**

This level enables fully connected multi-layer perceptron.

.. autosummary::
   :nosignatures:

   tvm.relay.log
   tvm.relay.sqrt
44
   tvm.relay.rsqrt
45
   tvm.relay.exp
46
   tvm.relay.sigmoid
47
   tvm.relay.add
48 49 50 51 52
   tvm.relay.subtract
   tvm.relay.multiply
   tvm.relay.divide
   tvm.relay.mod
   tvm.relay.tanh
53 54 55 56
   tvm.relay.concatenate
   tvm.relay.expand_dims
   tvm.relay.nn.softmax
   tvm.relay.nn.log_softmax
雾雨魔理沙 committed
57
   tvm.relay.nn.relu
58 59
   tvm.relay.nn.dropout
   tvm.relay.nn.batch_norm
60 61
   tvm.relay.nn.bias_add

62 63 64 65 66 67 68 69 70

**Level 2: Convolutions**

This level enables typical convnet models.

.. autosummary::
   :nosignatures:

   tvm.relay.nn.conv2d
71
   tvm.relay.nn.conv2d_transpose
72
   tvm.relay.nn.dense
73 74 75 76 77 78
   tvm.relay.nn.max_pool2d
   tvm.relay.nn.avg_pool2d
   tvm.relay.nn.global_max_pool2d
   tvm.relay.nn.global_avg_pool2d
   tvm.relay.nn.upsampling
   tvm.relay.nn.batch_flatten
79
   tvm.relay.nn.pad
80 81
   tvm.relay.nn.lrn
   tvm.relay.nn.l2_normalize
82 83
   tvm.relay.nn.contrib_conv2d_winograd_without_weight_transform
   tvm.relay.nn.contrib_conv2d_winograd_weight_transform
84 85 86 87


**Level 3: Additional Math And Transform Operators**

88 89
This level enables additional math and transform operators.

90 91 92
.. autosummary::
   :nosignatures:

93
   tvm.relay.nn.leaky_relu
Siju committed
94
   tvm.relay.nn.prelu
95
   tvm.relay.reshape
Siju committed
96
   tvm.relay.reshape_like
97 98
   tvm.relay.copy
   tvm.relay.transpose
99
   tvm.relay.squeeze
100 101
   tvm.relay.floor
   tvm.relay.ceil
102
   tvm.relay.sign
103
   tvm.relay.trunc
104
   tvm.relay.clip
105 106 107
   tvm.relay.round
   tvm.relay.abs
   tvm.relay.negative
Siva committed
108
   tvm.relay.take
109 110 111 112
   tvm.relay.zeros
   tvm.relay.zeros_like
   tvm.relay.ones
   tvm.relay.ones_like
113
   tvm.relay.gather_nd
114 115
   tvm.relay.full
   tvm.relay.full_like
116
   tvm.relay.cast
Siva committed
117
   tvm.relay.split
118
   tvm.relay.arange
119
   tvm.relay.stack
120 121
   tvm.relay.repeat
   tvm.relay.tile
122
   tvm.relay.reverse
123

雾雨魔理沙 committed
124

125 126
**Level 4: Broadcast and Reductions**

127 128 129
.. autosummary::
   :nosignatures:

130
   tvm.relay.right_shift
131
   tvm.relay.left_shift
132 133 134 135 136 137
   tvm.relay.equal
   tvm.relay.not_equal
   tvm.relay.greater
   tvm.relay.greater_equal
   tvm.relay.less
   tvm.relay.less_equal
138
   tvm.relay.all
139 140 141
   tvm.relay.logical_and
   tvm.relay.logical_or
   tvm.relay.logical_not
142
   tvm.relay.maximum
143
   tvm.relay.minimum
144
   tvm.relay.power
Zhi committed
145
   tvm.relay.where
146 147
   tvm.relay.argmax
   tvm.relay.argmin
148 149 150 151 152
   tvm.relay.sum
   tvm.relay.max
   tvm.relay.min
   tvm.relay.mean
   tvm.relay.prod
153
   tvm.relay.strided_slice
154
   tvm.relay.broadcast_to
155

雾雨魔理沙 committed
156

157 158
**Level 5: Vision/Image Operators**

159 160 161 162
.. autosummary::
   :nosignatures:

   tvm.relay.image.resize
163 164 165
   tvm.relay.vision.multibox_prior
   tvm.relay.vision.multibox_transform_loc
   tvm.relay.vision.nms
Siju committed
166
   tvm.relay.vision.yolo_reorg
167

168

169 170 171 172 173 174
**Level 6: Algorithm Operators**

.. autosummary::
   :nosignatures:

   tvm.relay.argsort
175
   tvm.relay.topk
176 177


178 179 180 181 182 183 184 185 186
**Level 10: Temporary Operators**

This level support backpropagation of broadcast operators. It is temporary.

.. autosummary::
   :nosignatures:

   tvm.relay.broadcast_to_like
   tvm.relay.collapse_sum_like
187
   tvm.relay.slice_like
188
   tvm.relay.shape_of
189
   tvm.relay.layout_transform
190 191
   tvm.relay.device_copy
   tvm.relay.annotation.on_device
192
   tvm.relay.reverse_reshape
193
   tvm.relay.nn.batch_matmul
194 195
   tvm.relay.contrib.adaptive_max_pool2d
   tvm.relay.contrib.adaptive_avg_pool2d
196 197


198 199 200 201
Level 1 Definitions
-------------------
.. autofunction:: tvm.relay.log
.. autofunction:: tvm.relay.sqrt
202
.. autofunction:: tvm.relay.rsqrt
203
.. autofunction:: tvm.relay.exp
204
.. autofunction:: tvm.relay.sigmoid
205
.. autofunction:: tvm.relay.add
206 207 208 209 210 211
.. autofunction:: tvm.relay.subtract
.. autofunction:: tvm.relay.multiply
.. autofunction:: tvm.relay.divide
.. autofunction:: tvm.relay.mod
.. autofunction:: tvm.relay.tanh
.. autofunction:: tvm.relay.concatenate
212
.. autofunction:: tvm.relay.expand_dims
213
.. autofunction:: tvm.relay.nn.softmax
214
.. autofunction:: tvm.relay.nn.log_softmax
雾雨魔理沙 committed
215
.. autofunction:: tvm.relay.nn.relu
216 217 218
.. autofunction:: tvm.relay.nn.dropout
.. autofunction:: tvm.relay.nn.batch_norm
.. autofunction:: tvm.relay.nn.bias_add
219 220 221 222 223


Level 2 Definitions
-------------------
.. autofunction:: tvm.relay.nn.conv2d
224
.. autofunction:: tvm.relay.nn.conv2d_transpose
225
.. autofunction:: tvm.relay.nn.dense
226 227 228 229 230 231
.. autofunction:: tvm.relay.nn.max_pool2d
.. autofunction:: tvm.relay.nn.avg_pool2d
.. autofunction:: tvm.relay.nn.global_max_pool2d
.. autofunction:: tvm.relay.nn.global_avg_pool2d
.. autofunction:: tvm.relay.nn.upsampling
.. autofunction:: tvm.relay.nn.batch_flatten
232
.. autofunction:: tvm.relay.nn.pad
233 234
.. autofunction:: tvm.relay.nn.lrn
.. autofunction:: tvm.relay.nn.l2_normalize
235 236
.. autofunction:: tvm.relay.nn.contrib_conv2d_winograd_without_weight_transform
.. autofunction:: tvm.relay.nn.contrib_conv2d_winograd_weight_transform
237

238 239
Level 3 Definitions
-------------------
240
.. autofunction:: tvm.relay.nn.leaky_relu
Siju committed
241
.. autofunction:: tvm.relay.nn.prelu
242 243 244 245 246
.. autofunction:: tvm.relay.reshape
.. autofunction:: tvm.relay.reshape_like
.. autofunction:: tvm.relay.copy
.. autofunction:: tvm.relay.transpose
.. autofunction:: tvm.relay.squeeze
247 248
.. autofunction:: tvm.relay.floor
.. autofunction:: tvm.relay.ceil
249
.. autofunction:: tvm.relay.sign
250
.. autofunction:: tvm.relay.trunc
251
.. autofunction:: tvm.relay.clip
252 253 254
.. autofunction:: tvm.relay.round
.. autofunction:: tvm.relay.abs
.. autofunction:: tvm.relay.negative
Siva committed
255
.. autofunction:: tvm.relay.take
256
.. autofunction:: tvm.relay.zeros
雾雨魔理沙 committed
257
.. autofunction:: tvm.relay.zeros_like
258
.. autofunction:: tvm.relay.ones
雾雨魔理沙 committed
259
.. autofunction:: tvm.relay.ones_like
260
.. autofunction:: tvm.relay.gather_nd
261 262 263
.. autofunction:: tvm.relay.full
.. autofunction:: tvm.relay.full_like
.. autofunction:: tvm.relay.cast
Siva committed
264
.. autofunction:: tvm.relay.split
265
.. autofunction:: tvm.relay.arange
266
.. autofunction:: tvm.relay.stack
267 268
.. autofunction:: tvm.relay.repeat
.. autofunction:: tvm.relay.tile
269
.. autofunction:: tvm.relay.reverse
雾雨魔理沙 committed
270 271


272 273 274
Level 4 Definitions
-------------------
.. autofunction:: tvm.relay.right_shift
275
.. autofunction:: tvm.relay.left_shift
276 277 278 279 280 281
.. autofunction:: tvm.relay.equal
.. autofunction:: tvm.relay.not_equal
.. autofunction:: tvm.relay.greater
.. autofunction:: tvm.relay.greater_equal
.. autofunction:: tvm.relay.less
.. autofunction:: tvm.relay.less_equal
282
.. autofunction:: tvm.relay.all
283 284 285
.. autofunction:: tvm.relay.logical_and
.. autofunction:: tvm.relay.logical_or
.. autofunction:: tvm.relay.logical_not
286 287
.. autofunction:: tvm.relay.maximum
.. autofunction:: tvm.relay.minimum
288
.. autofunction:: tvm.relay.power
Zhi committed
289
.. autofunction:: tvm.relay.where
290 291
.. autofunction:: tvm.relay.argmax
.. autofunction:: tvm.relay.argmin
292 293 294 295 296
.. autofunction:: tvm.relay.sum
.. autofunction:: tvm.relay.max
.. autofunction:: tvm.relay.min
.. autofunction:: tvm.relay.mean
.. autofunction:: tvm.relay.prod
297
.. autofunction:: tvm.relay.strided_slice
298
.. autofunction:: tvm.relay.broadcast_to
299

300

301 302 303
Level 5 Definitions
-------------------
.. autofunction:: tvm.relay.image.resize
304 305 306
.. autofunction:: tvm.relay.vision.multibox_prior
.. autofunction:: tvm.relay.vision.multibox_transform_loc
.. autofunction:: tvm.relay.vision.nms
Siju committed
307
.. autofunction:: tvm.relay.vision.yolo_reorg
308 309


310 311 312
Level 6 Definitions
-------------------
.. autofunction:: tvm.relay.argsort
313
.. autofunction:: tvm.relay.topk
314 315


316 317 318 319
Level 10 Definitions
--------------------
.. autofunction:: tvm.relay.broadcast_to_like
.. autofunction:: tvm.relay.collapse_sum_like
320
.. autofunction:: tvm.relay.slice_like
321
.. autofunction:: tvm.relay.shape_of
322 323
.. autofunction:: tvm.relay.layout_transform
.. autofunction:: tvm.relay.device_copy
324 325
.. autofunction:: tvm.relay.annotation.on_device
.. autofunction:: tvm.relay.reverse_reshape
326
.. autofunction:: tvm.relay.nn.batch_matmul
327 328
.. autofunction:: tvm.relay.contrib.adaptive_max_pool2d
.. autofunction:: tvm.relay.contrib.adaptive_avg_pool2d