Compile PaddlePaddle Models

Author: Ziyuan Ma

This article is an introductory tutorial to deploy PaddlePaddle models with Relay. For us to begin with, PaddlePaddle>=2.1.3 is required to be installed. A quick solution is

pip install paddlepaddle -i https://mirror.baidu.com/pypi/simple

or please refer to official site. https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/install/pip/linux-pip.html

import tarfile
import paddle
import numpy as np
import tvm
from tvm import relay
from tvm.contrib.download import download_testdata

Out:

WARNING: OMP_NUM_THREADS set to 4, not 1. The computation speed will not be optimized if you use data parallel. It will fail if this PaddlePaddle binary is compiled with OpenBlas since OpenBlas does not support multi-threads.
PLEASE USE OMP_NUM_THREADS WISELY.

Load pretrained ResNet50 model

We load a pretrained ResNet50 provided by PaddlePaddle.

url = "https://bj.bcebos.com/x2paddle/models/paddle_resnet50.tar"
model_path = download_testdata(url, "paddle_resnet50.tar", module="model")

with tarfile.open(model_path) as tar:
    names = tar.getnames()
    for name in names:
        tar.extract(name, "./")

model = paddle.jit.load("./paddle_resnet50/model")

Load a test image

A single cat dominates the examples!

from PIL import Image
import paddle.vision.transforms as T


transforms = T.Compose(
    [
        T.Resize((256, 256)),
        T.CenterCrop(224),
        T.ToTensor(),
        T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
    ]
)

img_url = "https://github.com/dmlc/mxnet.js/blob/main/data/cat.png?raw=true"
img_path = download_testdata(img_url, "cat.png", module="data")
img = Image.open(img_path).resize((224, 224))

img = transforms(img)
img = np.expand_dims(img, axis=0)

Compile the model with relay

target = "llvm"
shape_dict = {"inputs": img.shape}
mod, params = relay.frontend.from_paddle(model, shape_dict)

with tvm.transform.PassContext(opt_level=3):
    executor = relay.build_module.create_executor(
        "graph", mod, tvm.cpu(0), target, params
    ).evaluate()

Execute on TVM

dtype = "float32"
tvm_output = executor(tvm.nd.array(img.astype(dtype))).numpy()

Look up synset name

Look up prediction top 1 index in 1000 class synset.

synset_url = "".join(
    [
        "https://gist.githubusercontent.com/zhreshold/",
        "4d0b62f3d01426887599d4f7ede23ee5/raw/",
        "596b27d23537e5a1b5751d2b0481ef172f58b539/",
        "imagenet1000_clsid_to_human.txt",
    ]
)
synset_name = "imagenet1000_clsid_to_human.txt"
synset_path = download_testdata(synset_url, synset_name, module="data")
with open(synset_path) as f:
    synset = f.readlines()

top1 = np.argmax(tvm_output[0])
print(f"TVM prediction top-1 id: {top1}, class name: {synset[top1]}")

Out:

TVM prediction top-1 id: 282, class name:  282: 'tiger cat',