.. note::
    :class: sphx-glr-download-link-note

    Click :ref:`here <sphx_glr_download_how_to_deploy_models_deploy_ssd_gluoncv.py>` to download the full example code
.. rst-class:: sphx-glr-example-title

.. _sphx_glr_how_to_deploy_models_deploy_ssd_gluoncv.py:


Deploy Single Shot Multibox Detector(SSD) model
===============================================
**Author**: `Yao Wang <https://github.com/kevinthesun>`_
`Leyuan Wang <https://github.com/Laurawly>`_

This article is an introductory tutorial to deploy SSD models with TVM.
We will use GluonCV pre-trained SSD model and convert it to Relay IR


.. code-block:: default

    import tvm
    from tvm import te

    from matplotlib import pyplot as plt
    from tvm import relay
    from tvm.contrib import graph_executor
    from tvm.contrib.download import download_testdata
    from gluoncv import model_zoo, data, utils






.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    /usr/local/lib/python3.6/dist-packages/gluoncv/__init__.py:40: UserWarning: Both `mxnet==1.6.0` and `torch==1.7.0` are installed. You might encounter increased GPU memory footprint if both framework are used at the same time.
      warnings.warn(f'Both `mxnet=={mx.__version__}` and `torch=={torch.__version__}` are installed. '



Preliminary and Set parameters
------------------------------
.. note::

  We support compiling SSD on both CPUs and GPUs now.

  To get best inference performance on CPU, change
  target argument according to your device and
  follow the :ref:`tune_relay_x86` to tune x86 CPU and
  :ref:`tune_relay_arm` for arm CPU.

  To get best inference performance on Intel graphics,
  change target argument to :code:`opencl -device=intel_graphics`.
  But when using Intel graphics on Mac, target needs to
  be set to `opencl` only for the reason that Intel subgroup
  extension is not supported on Mac.

  To get best inference performance on CUDA-based GPUs,
  change the target argument to :code:`cuda`; and for
  OPENCL-based GPUs, change target argument to
  :code:`opencl` followed by device argument according
  to your device.


.. code-block:: default


    supported_model = [
        "ssd_512_resnet50_v1_voc",
        "ssd_512_resnet50_v1_coco",
        "ssd_512_resnet101_v2_voc",
        "ssd_512_mobilenet1.0_voc",
        "ssd_512_mobilenet1.0_coco",
        "ssd_300_vgg16_atrous_voc" "ssd_512_vgg16_atrous_coco",
    ]

    model_name = supported_model[0]
    dshape = (1, 3, 512, 512)







Download and pre-process demo image


.. code-block:: default


    im_fname = download_testdata(
        "https://github.com/dmlc/web-data/blob/main/" + "gluoncv/detection/street_small.jpg?raw=true",
        "street_small.jpg",
        module="data",
    )
    x, img = data.transforms.presets.ssd.load_test(im_fname, short=512)







Convert and compile model for CPU.


.. code-block:: default


    block = model_zoo.get_model(model_name, pretrained=True)


    def build(target):
        mod, params = relay.frontend.from_mxnet(block, {"data": dshape})
        with tvm.transform.PassContext(opt_level=3):
            lib = relay.build(mod, target, params=params)
        return lib






.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    /usr/local/lib/python3.6/dist-packages/mxnet/gluon/block.py:1389: UserWarning: Cannot decide type for the following arguments. Consider providing them as input:
            data: None
      input_sym_arg_type = in_param.infer_type()[0]



Create TVM runtime and do inference
.. note::

  Use target = "cuda -libs" to enable thrust based sort, if you
  enabled thrust during cmake by -DUSE_THRUST=ON.


.. code-block:: default



    def run(lib, dev):
        # Build TVM runtime
        m = graph_executor.GraphModule(lib["default"](dev))
        tvm_input = tvm.nd.array(x.asnumpy(), device=dev)
        m.set_input("data", tvm_input)
        # execute
        m.run()
        # get outputs
        class_IDs, scores, bounding_boxs = m.get_output(0), m.get_output(1), m.get_output(2)
        return class_IDs, scores, bounding_boxs


    for target in ["llvm", "cuda"]:
        dev = tvm.device(target, 0)
        if dev.exist:
            lib = build(target)
            class_IDs, scores, bounding_boxs = run(lib, dev)







Display result


.. code-block:: default


    ax = utils.viz.plot_bbox(
        img,
        bounding_boxs.numpy()[0],
        scores.numpy()[0],
        class_IDs.numpy()[0],
        class_names=block.classes,
    )
    plt.show()



.. image:: /how_to/deploy_models/images/sphx_glr_deploy_ssd_gluoncv_001.png
    :class: sphx-glr-single-img





.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 3 minutes  12.207 seconds)


.. _sphx_glr_download_how_to_deploy_models_deploy_ssd_gluoncv.py:


.. only :: html

 .. container:: sphx-glr-footer
    :class: sphx-glr-footer-example



  .. container:: sphx-glr-download

     :download:`Download Python source code: deploy_ssd_gluoncv.py <deploy_ssd_gluoncv.py>`



  .. container:: sphx-glr-download

     :download:`Download Jupyter notebook: deploy_ssd_gluoncv.ipynb <deploy_ssd_gluoncv.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_