* use pip3 for python3

* make python3 as default

* [Relay] [Quantization] WIP - Common files for the qauntization work.

* [Relay] [Quantization] WIP - Prototyping requantize op.

* Requantize operator implementation.

Requantize converts one quantized tensor representation to another quantized
representation. The PR has following implementation features

- Requantize operator defined in qnn namespace - relay.qnn.requantize
- Lowering of the requantize to exisiting Relay operators
- Integer fixed point implementation of requantize
    - Two rounding modes - FE_UPWARDS (round towards infinity) and
    FE_AWAY_FROM_ZERO (std::round behavior)
- Floating point implementation as well, that can act as reference or can be
used for devices when FP32 computation is not used.
- Unit test cases

Relevant Issue - https://github.com/dmlc/tvm/issues/2351

Credit to TFLite and GemmLowp to provide reference implementations.

* Typo and lint fixes.

* Doc fix.

* Uncommenting the lint script (fixing mistake).

* Modifying the unit tests.

* Moving C++ files into src/relay/qnn

* Moving python files to python/tvm/relay/qnn. Some minor fixes.

* Moving the attrs.h inside the include directory.

* Pushing files that I forgot earlier. Changing util location.

* Incorporating comments. API change. Lint fixes.

* Modifying the GetFixedPointMultiplierShift API as per comments.

* Forgot the dialect change.

* Changing rewrite to qnn_lower.

* Renaming Quantize to Qnn for clarity.

* Remove use_int_domain.

* Incorportaing review comments.

* Adding API doc for QNN dialect.

* Move the qnn_lower pass to transform namespace.

* Moving from expr to module. Adding namespace in C++.

* Minor sentence rewrites. Added qnn namespace.

* Added the API doc.

* Chanding default out_dtype to int8. Adding a test with in/out_dtype as uint8.

* Style fixes. Better error messages.

* Adding documentation.

* More documentation fixes.

* Adding out dtype check for requantize.

* Adding corner case for FP32 to fixed point conversion.

* Adding extra line.

* Documentation fix.

* Adding static inline.

* Incorporating jackwish comment. Removed idtype from requantize lowering.

* Removing Quantize/Dequantize code. Restricting Requantize to (u)int8/int32.

* Style fixes.

* Fix the docs.

* Move to Legalize API.

* Add LayerNorm op

* update

* fix

* Add mean_std and mean_variance

* add std and update doc

* add license

* x

* lint

* x

* fix

* fix doc

* relay vm serialization

* fix lint

* load params, fix stream

* lint

* fix typo

* initial compilation script for chisel-vta;

* replace tabs with spaces;

* compile script for de10-nano;

* remove generated verilog source code;

* remove `altsource_probe`, `debounce`, `edge_detect` ip;

* replace quartus project files with a single tcl script;

* Update install.md

* improved makefile-based compilation script;

* complete makefile-based compilation of chisel-vta for de10-nano;

* install quartus;

* conversion to .rbf file;

* document chisel-vta compilation process for de10-nano;

* rename generated bitstream file;

* download and extract custom ip for de10-nano;

* minor change

* minor change

* fix indentation;

* bug fix;

* improved robustness in makefile;

* clean up;

* add `.sdc .ipx .qsys` allowance in jenkins;

* add ASF header;

* add ASF header;

* remove IntelShell.scala, update vta_hw.tcl, clean up Makefile & soc_system.qsys;

* add ASF header;

* keep sources compact;

* keep sources compact;

* it's not necessary now

* AXI4LiteClient -> AXI3Client for IntelShell

* remove connection to fpga_only_master;

* a few important bug fix: wire reset pin, and set host_r_last to high

* remove intel specific interface definition;

* add NO_DSP option in Makefile;

* AXI4Lite is not used in IntelShell;

* minor fix: disable dsp and use logic instead;

* quartus version change: 18.0 -> 18.1

* remove altera related statement;

* compose compile_design.tcl

* initial tcl script for soc_system generation;

* remove .qsys file;

* remove unused;

* .qsys can be generated by tcl script;

* remove hps_io and shrink size of soc_system;

* integrate into makefile;

* version change: 18.0 -> 18.1

* add sample config file for de10-nano;

* parameterize DEVICE and PROJECT_NAME

* remove extra lines;

* brief description on flashing sd card image for de10-nano

* docs on building additional components

* parameterize DEVICE and DEVICE_FAMILY

* parameterize DEVICE and DEVICE_FAMILY

* parameterize DEVICE and DEVICE_FAMILY

* de10-nano -> de10nano

* minor change

* add comment in code and document in order to address review comments;

* hardware refactor for increased FPGA coverage, small optimizations

* fix header

* cleaning up parameters that won't be needed for now

* streamlining makefile, and simplifying tcl scripts

* moving parameter derivation into pkg_config.py, keeping tcl scripts lightweight

* refactoring tcl script to avoid global variables

* deriving AXI signals in pkg_config.py

* unifying address map definition for hardware and software drivers

* single channel design for ultra96 to simplify build

* enable alu by default, no mul opcode for now

* hardware fix

* new bitstream; vta version

* avoid error when env variable is not set

* ultra96 cleanup

* further cleaning up tcl script for bitstream generation

* preliminary rpc server support on ultra96

* rpc server tracker scripts

* ultra96 ldflag

* ultra96 support

* ultra96 support

* cleanup line

* cmake support for ultra96

* simplify memory instantiation

* cleaning up IP parameter initialization

* fix queue instantiation

* 2019.1 transition

* fix macro def

* removing bus width from config

* cleanup

* fix

* turning off testing for now

* cleanup ultra96 ps insantiation

* minor refactor

* adding comments

* upgrading to tophub v0.6

* model used in TVM target now refers to a specific version of VTA for better autoTVM scheduling

* revert change due to bug

* rename driver files to be for zynq-type devices

* streamlining address mapping

* unifying register map offset values between driver and hardware generator

* rely on cma library for cache flush/invalidation

* coherence management

* not make buffer packing depend on data types that can be wider than 64bits

* refactor config derivation to minimize free parameters

* fix environment/pkg config interaction

* adding cfg dump property to pkgconfig:

* fix rpc reconfig

* fix spacing

* cleanup

* fix spacing

* long line fix

* fix spacing and lint

* fix line length

* cmake fix

* environment fix

* renaming after pynq since the driver stack relies on the pynq library - see pynq.io

* update doc

* adding parameterization to  name

* space

* removing reg width

* vta RPC

* update doc on how to edit vta_config.json

* fix path

* fix path

* Add USE_GTEST as a CMake variable

* Add GTest section in installation docs

* Incorporate feedback

= Motivation

It's useful to expose the tvm::reinterpret functionality to Relay/TOPI users, as
this allows them to build (fused) operators leveraging the bitwise
reinterpretation of an operator. An example is approximate transcendental
functions, which can be implemented similar to:

```.py
    def C(x):
        return relay.expr.const(x, "float32")

    def approx_exp(x):
        x = relay.minimum(relay.maximum(x, C(-88.0)), C(88.0))
        x = C(127.0) + x * C(1.44269504)
        xf = relay.floor(x)
        i = relay.cast(xf, "int32")
        x = x - xf
        Y = C(0.99992522) + x * (C(0.69583354) + x * (C(0.22606716) + x * C(0.078024523)))
        exponent = relay.left_shift(i, relay.expr.const(23, "int32"))
        exponent = relay.reinterpret(exponent, "float32")
        return exponent * Y

    def approx_sigmoid(x):
        # <2.0e-5 absolute error over [-5, 5]
        y = approx_exp(x)
        return y / (y + C(1.0))

    def approx_tanh(x):
        # <4.0e-5 absolute error over [-5, 5]
        x = x * C(2.0)
        y = approx_exp(x)
        return (y - C(1.0)) / (y + C(1.0))
```

See unit tests for implementations of these approximate transendentals.

* Update the Relay adding pass doc to reference the new pass infrastructure

* Correct pass name

Co-Authored-By: Zhi <5145158+zhiics@users.noreply.github.com>

* Align header equals signs

* [docs] Add a tutorial for the pass manager

* address comment

* address more comments

* retrigger ci

* address steven's comments

* address comments

* retrigger ci

* Update docs/dev/relay_pass_infra.rst

Co-Authored-By: Steven S. Lyubomirsky <slyubomirsky@gmail.com>

* Update docs/dev/relay_pass_infra.rst

Co-Authored-By: Steven S. Lyubomirsky <slyubomirsky@gmail.com>

* Update docs/dev/relay_pass_infra.rst

Co-Authored-By: Steven S. Lyubomirsky <slyubomirsky@gmail.com>

* Update docs/dev/relay_pass_infra.rst

Co-Authored-By: Steven S. Lyubomirsky <slyubomirsky@gmail.com>

* Update docs/dev/relay_pass_infra.rst

Co-Authored-By: Steven S. Lyubomirsky <slyubomirsky@gmail.com>

* Update docs/dev/relay_pass_infra.rst

Co-Authored-By: Logan Weber <36520469+weberlo@users.noreply.github.com>

* Update docs/dev/relay_pass_infra.rst

Co-Authored-By: Logan Weber <36520469+weberlo@users.noreply.github.com>

* Add sequence_mask

use exactly the same arguments as mxnet

fix

* fix lint

* fix lint

* add mxnet conversion + relay

* update

* update doc

* fix pylint

* fix doc

* address comment

* try to address comments

* try to enable shape check for valid_length

* fix

* try to fix

* fix bug

* try to fix

* address comment

* address comment

* init impl for topk

* Fix cpu for topk

* init cuda impl for topk

* Add cuda for topk

* fix

* Add doc

* update doc

* lint

* lint

* lint

* x

* fix warning

* [Relay] Add TopK in tf converter

* Add frontend converter

* fix

* [Relay][Docs] Add parser dependency install instructions.

See https://discuss.tvm.ai/t/trouble-enabling-antlr/2783.

* Add a word.

* Update since the parser will now be committed to the repo.

* revert b/c adding the parser doesn't fix this

* [Bugfix][VTA] PkgConfig cause crash in PYNQ board due to link library
not exist.

Symptom:
When run vta_get_started.py with pynq board, host crash and
complain "cannot find -lsds_lib" and "cannot find -l:libdma.so"

Reproduce:
At pynq board, delete the ./build/vta_config.json, then run rpc
server.
In host machine run vta_get_started.py, issue would reproduce.

Analysis:
This issue caused by 'PkgConfig' function  still using pynq2.1
library which not exist in pynq2.4 anymore, when a "reconfig_runtime"
logic of rpc_server.py get triggered , the compile would failed due to
link library not exist.

Solution:
change the link library to libcma.so.

* [Document Change][VTA] Change pynq version from 2.3 into 2.4.

Issue:
pynq 2.3 image not available anymore from pynq download page and pynq
2.4 is the current latest image which available in the said website, after
verification, currently VTA work good with pynq 2.4 image, hence update
related document from pynq 2.3 to 2.4.

* [Doc][Relay] Add VM doc

* Add Apache header

* Apply suggestions from code review

Co-Authored-By: Steven S. Lyubomirsky <slyubomirsky@gmail.com>
Co-Authored-By: 雾雨魔理沙 <lolisa@marisa.moe>
Co-Authored-By: Logan Weber <36520469+weberlo@users.noreply.github.com>
Co-Authored-By: Zhi <5145158+zhiics@users.noreply.github.com>

* Junru's comment

* More fix

* More fix

* More fix

* last fix

* Apply suggestions from code review

Co-Authored-By: 雾雨魔理沙 <lolisa@marisa.moe>

* Apply suggestions from code review

Co-Authored-By: Logan Weber <36520469+weberlo@users.noreply.github.com>

* Add code links

* Remove unused bp

* Update docs/dev/virtual_machine.rst

Co-Authored-By: Logan Weber <36520469+weberlo@users.noreply.github.com>

* Explain TODO

* Yong's comment

Co-Authored-By: Yong Wu <55wuyong@163.com>

* Comment

* [Relay][TOPI] operator All

* Update tests/python/frontend/tensorflow/test_forward.py

Co-Authored-By: yongwww <55wuyong@163.com>

* fix comments

* change to level 4

* Add topi adaptive_pool

* Use adaptive_pool to compute global_pool

* Add relay adaptive pool2d

* Fix lint

* Fix typo

* Minor change

* Change support level to 10

* Add contrib

* Remove global pool schedule

* Add contrib module

* Fix lint

* Update doc

* Update doc

* Developer documentation for InferBound pass.

* Correct spelling of 'inavlid'

* [DOC] correct spelling of 'schdule'.

* [DOC] clean up use of abbreviation "interop"

* [DOC] capitalize API abbreviation consistently

* [DOC] correct spelling of 'peformed'.

* [DOC] correct spelling of 'intermidiate'

* Remove trailing white space.

* Correct spelling of 'parametrization'.

* [DOC] minor improvements to Range documentation.

* ssd gluoncv gpu op updated

* ssd gluoncv gpu op updated

* tutorials and testes modified

* tutorials and testes modified

* fix lint

* fix lint

* address comment

* multibox bug fixed

* space line added

* use less threads per block

* use less threads per block

* less threads per block for get valid count

* less threads per block for get valid count

* merge with master

* Revert "less threads per block for get valid count"

This reverts commit 08896cfccc34b0b2a1646d01d01ea4cad73941c4.

* Revert "less threads per block for get valid count"

This reverts commit 08896cfccc34b0b2a1646d01d01ea4cad73941c4.

* typo fixed

* elem length made to a variable

* fix lint error

* fix lint error

* lint fixed

* bug fixed

* bug fixed

* lint fixed

* error fixed

* error fixed

* test ci

* test ci

* seperate argsort to be an independent op

* seperate argsort to be an independent op

* fix lint

* fix lint

* remove unsupported models

* typo fixed

* argsort added to realy

* solve conflicts with master

* fix lint

* fix lint

* test push

* Revert "test push"

This reverts commit 6db00883fab6cc06bddf564c926bb27c874397d8.

* fix lint error

* fix more lint

* cpu test_sort udpated

* debug ci

* nms fixed

* expose argsort to relay frontend

* test ci

* fix lint

* sort register error fixed

* fix nnvm

* nms type fixed

* adaptive pooling added to relay

* Revert "adaptive pooling added to relay"

This reverts commit 1119f1f2c055753e0cc5611627597749134c5c8c.

* fix lint

* expose argsort op

* fix lint

* fix lint

* fix lint

* sort test updated

* sort bug fixed

* nnvm error fixed

* fix argsort default data type returned to be float insteaf of int

* fix lint

* fix lint

* test fixed

* fix valid count

* fix titanx bug

* tutorial add both targets

* titanx error fixed

* try to fix CI old gpu error

* try to solve CI GPU error

* get_valid_count added

* reverse get_valid_count

* get valid count optimized

* address comments

* fix ci error

* remove unessesary block sync

* add back one sync

* address comments

* address more comments

* more comments

* move sort to be indepent algorithm

* typo fixed

* more typos

* comments addressed

* doc updated

* fix pylint

* address final comments

* apache license added

* Add Relay compiler pass tutorial docs

* Add Python API hook wrapping step

* Incorporate feedback

* More doc iteration

* Mooooore iteration

* Rewrite `runtime.md` in rst

* Add gradient operator tutorial docs

* Incorporate Steven's and Ziheng's feedback

* Remove TODO about `collapse_sum_like`

* Add more examples

* [HEADER] ASF header dir=include

* [HEADER] ASF Header dir=src

* [HEADER] ASF Header -dir=python

* [HEADER] ASF header dir=topi

* [HEADER] ASF Header dir=nnvm

* [HEADER] ASF Header -dir=tutorials

* [HEADER] ASF Header dir=tests

* [HEADER] ASF Header -dir=docker

* fix whitespace

* [HEADER] ASF Header -dir=jvm

* [HEADER] ASF Header -dir=web

* [HEADER] ASF Header --dir=apps

* [HEADER] ASF Header --dir=vta

* [HEADER] ASF Header -dir=go

* temp

* [HEADER] ASF Header --dir=rust

* [HEADER] Add ASF Header --dir=cmake

* [HEADER] ASF Header --dir=docs

* [HEADER] Header for Jenkinsfile

* [HEADER] ASF Header to toml and md

* [HEADER] ASF Header to gradle

* Finalize rat cleanup

* Fix permission

* Fix java test

* temporary remove nnvm onnx test

* gather_nd added

* gather_nd test added

* more test added

* fix lint

* fix build error

* fix lint

* comments addressed