relay_op.rst 6.25 KB
Newer Older
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
Relay Core Tensor Operators
===========================

This page contains the list of core tensor operator primitives pre-defined in tvm.relay.
The core tensor operator primitives covers typical workloads in deep learning.
They can represent workloads in front-end frameworks, and provide basic building blocks for optimization.
Since deep learning is a fast evolving field and it is that possible to have operators that are not in here.


.. note::

   This document will directly list the function signature of
   these operators in the python frontend.


Overview of Operators
---------------------
**Level 1: Basic Operators**

This level enables fully connected multi-layer perceptron.

.. autosummary::
   :nosignatures:

   tvm.relay.log
   tvm.relay.sqrt
   tvm.relay.exp
28
   tvm.relay.sigmoid
29
   tvm.relay.add
30
   tvm.relay.expand_dims
31
   tvm.relay.concatenate
32
   tvm.relay.nn.softmax
33
   tvm.relay.nn.log_softmax
34 35 36 37 38
   tvm.relay.subtract
   tvm.relay.multiply
   tvm.relay.divide
   tvm.relay.mod
   tvm.relay.tanh
雾雨魔理沙 committed
39
   tvm.relay.nn.relu
40 41
   tvm.relay.nn.dropout
   tvm.relay.nn.batch_norm
42 43
   tvm.relay.nn.bias_add

44

45

46 47 48 49 50 51 52 53
**Level 2: Convolutions**

This level enables typical convnet models.

.. autosummary::
   :nosignatures:

   tvm.relay.nn.conv2d
54
   tvm.relay.nn.conv2d_transpose
55
   tvm.relay.nn.dense
56 57 58 59 60 61
   tvm.relay.nn.max_pool2d
   tvm.relay.nn.avg_pool2d
   tvm.relay.nn.global_max_pool2d
   tvm.relay.nn.global_avg_pool2d
   tvm.relay.nn.upsampling
   tvm.relay.nn.batch_flatten
62
   tvm.relay.nn.pad
63 64
   tvm.relay.nn.lrn
   tvm.relay.nn.l2_normalize
65 66 67 68


**Level 3: Additional Math And Transform Operators**

69 70
This level enables additional math and transform operators.

71 72 73
.. autosummary::
   :nosignatures:

74
   tvm.relay.zeros
75
   tvm.relay.nn.leaky_relu
Siju committed
76
   tvm.relay.nn.prelu
77
   tvm.relay.zeros_like
78
   tvm.relay.ones
79
   tvm.relay.ones_like
80
   tvm.relay.reshape
Siju committed
81
   tvm.relay.reshape_like
82 83
   tvm.relay.copy
   tvm.relay.transpose
84
   tvm.relay.squeeze
85 86 87 88 89 90
   tvm.relay.floor
   tvm.relay.ceil
   tvm.relay.trunc
   tvm.relay.round
   tvm.relay.abs
   tvm.relay.negative
Siva committed
91
   tvm.relay.take
92 93 94 95
   tvm.relay.zeros
   tvm.relay.zeros_like
   tvm.relay.ones
   tvm.relay.ones_like
96 97
   tvm.relay.full
   tvm.relay.full_like
98
   tvm.relay.cast
Siva committed
99
   tvm.relay.split
100

雾雨魔理沙 committed
101

102 103
**Level 4: Broadcast and Reductions**

104 105 106
.. autosummary::
   :nosignatures:

107
   tvm.relay.right_shift
108
   tvm.relay.left_shift
109 110 111 112 113 114
   tvm.relay.equal
   tvm.relay.not_equal
   tvm.relay.greater
   tvm.relay.greater_equal
   tvm.relay.less
   tvm.relay.less_equal
115
   tvm.relay.maximum
116
   tvm.relay.minimum
117
   tvm.relay.power
Zhi committed
118
   tvm.relay.where
119 120
   tvm.relay.argmax
   tvm.relay.argmin
121 122 123 124 125
   tvm.relay.sum
   tvm.relay.max
   tvm.relay.min
   tvm.relay.mean
   tvm.relay.prod
126
   tvm.relay.strided_slice
127
   tvm.relay.broadcast_to
128

雾雨魔理沙 committed
129

130 131
**Level 5: Vision/Image Operators**

132 133 134 135
.. autosummary::
   :nosignatures:

   tvm.relay.image.resize
136 137 138
   tvm.relay.vision.multibox_prior
   tvm.relay.vision.multibox_transform_loc
   tvm.relay.vision.nms
139

140

141 142 143 144 145 146 147 148 149
**Level 10: Temporary Operators**

This level support backpropagation of broadcast operators. It is temporary.

.. autosummary::
   :nosignatures:

   tvm.relay.broadcast_to_like
   tvm.relay.collapse_sum_like
150
   tvm.relay.slice_like
151 152


153 154 155 156 157
Level 1 Definitions
-------------------
.. autofunction:: tvm.relay.log
.. autofunction:: tvm.relay.sqrt
.. autofunction:: tvm.relay.exp
158
.. autofunction:: tvm.relay.sigmoid
159
.. autofunction:: tvm.relay.add
160 161 162 163 164 165
.. autofunction:: tvm.relay.subtract
.. autofunction:: tvm.relay.multiply
.. autofunction:: tvm.relay.divide
.. autofunction:: tvm.relay.mod
.. autofunction:: tvm.relay.tanh
.. autofunction:: tvm.relay.concatenate
166
.. autofunction:: tvm.relay.expand_dims
167
.. autofunction:: tvm.relay.nn.softmax
168
.. autofunction:: tvm.relay.nn.log_softmax
雾雨魔理沙 committed
169
.. autofunction:: tvm.relay.nn.relu
170 171 172
.. autofunction:: tvm.relay.nn.dropout
.. autofunction:: tvm.relay.nn.batch_norm
.. autofunction:: tvm.relay.nn.bias_add
173 174 175 176 177


Level 2 Definitions
-------------------
.. autofunction:: tvm.relay.nn.conv2d
178
.. autofunction:: tvm.relay.nn.conv2d_transpose
179
.. autofunction:: tvm.relay.nn.dense
180 181 182 183 184 185
.. autofunction:: tvm.relay.nn.max_pool2d
.. autofunction:: tvm.relay.nn.avg_pool2d
.. autofunction:: tvm.relay.nn.global_max_pool2d
.. autofunction:: tvm.relay.nn.global_avg_pool2d
.. autofunction:: tvm.relay.nn.upsampling
.. autofunction:: tvm.relay.nn.batch_flatten
186 187
.. autofunction:: tvm.relay.nn.lrn
.. autofunction:: tvm.relay.nn.l2_normalize
188

189

190 191
Level 3 Definitions
-------------------
192
.. autofunction:: tvm.relay.nn.leaky_relu
Siju committed
193
.. autofunction:: tvm.relay.nn.prelu
194 195 196 197 198 199 200
.. autofunction:: tvm.relay.floor
.. autofunction:: tvm.relay.ceil
.. autofunction:: tvm.relay.trunc
.. autofunction:: tvm.relay.round
.. autofunction:: tvm.relay.abs
.. autofunction:: tvm.relay.negative
.. autofunction:: tvm.relay.reshape
Siju committed
201
.. autofunction:: tvm.relay.reshape_like
202
.. autofunction:: tvm.relay.copy
203
.. autofunction:: tvm.relay.squeeze
204
.. autofunction:: tvm.relay.transpose
Siva committed
205
.. autofunction:: tvm.relay.take
206
.. autofunction:: tvm.relay.zeros
雾雨魔理沙 committed
207
.. autofunction:: tvm.relay.zeros_like
208
.. autofunction:: tvm.relay.ones
雾雨魔理沙 committed
209
.. autofunction:: tvm.relay.ones_like
210 211 212
.. autofunction:: tvm.relay.full
.. autofunction:: tvm.relay.full_like
.. autofunction:: tvm.relay.cast
Siva committed
213
.. autofunction:: tvm.relay.split
雾雨魔理沙 committed
214 215


216 217 218
Level 4 Definitions
-------------------
.. autofunction:: tvm.relay.right_shift
219
.. autofunction:: tvm.relay.left_shift
220 221 222 223 224 225
.. autofunction:: tvm.relay.equal
.. autofunction:: tvm.relay.not_equal
.. autofunction:: tvm.relay.greater
.. autofunction:: tvm.relay.greater_equal
.. autofunction:: tvm.relay.less
.. autofunction:: tvm.relay.less_equal
226 227
.. autofunction:: tvm.relay.maximum
.. autofunction:: tvm.relay.minimum
228
.. autofunction:: tvm.relay.power
Zhi committed
229
.. autofunction:: tvm.relay.where
230 231
.. autofunction:: tvm.relay.argmax
.. autofunction:: tvm.relay.argmin
232 233 234 235 236
.. autofunction:: tvm.relay.sum
.. autofunction:: tvm.relay.max
.. autofunction:: tvm.relay.min
.. autofunction:: tvm.relay.mean
.. autofunction:: tvm.relay.prod
237
.. autofunction:: tvm.relay.strided_slice
238

239

240 241 242
Level 5 Definitions
-------------------
.. autofunction:: tvm.relay.image.resize
243 244 245
.. autofunction:: tvm.relay.vision.multibox_prior
.. autofunction:: tvm.relay.vision.multibox_transform_loc
.. autofunction:: tvm.relay.vision.nms
246 247 248 249 250 251


Level 10 Definitions
--------------------
.. autofunction:: tvm.relay.broadcast_to_like
.. autofunction:: tvm.relay.collapse_sum_like
252
.. autofunction:: tvm.relay.slice_like