Tuning High Performance Convolution on NVIDIA GPUs

Author: Lianmin Zheng

This is an advanced tutorial for writing high performance tunable template for NVIDIA GPU. By running auto-tuner on this template, we can outperform the vendor provided library CuDNN in many cases.

Note that this tutorial will not run on Windows or recent versions of macOS. To get it to run, you will need to wrap the body of this tutorial in a if __name__ == "__main__": block.

Install dependencies

To use autotvm package in tvm, we need to install some extra dependencies. (change “3” to “2” if you use python2):

pip3 install --user psutil xgboost tornado cloudpickle

To make TVM run faster in tuning, it is recommended to use cython as FFI of tvm. In the root directory of tvm, execute

pip3 install --user cython
sudo make cython3

Now return to python code. Import packages.

import logging
import sys
import numpy as np

import tvm
from tvm import te, topi, testing
from tvm.topi.testing import conv2d_nchw_python
import tvm.testing

from tvm import autotvm

Step 1: Define the search space

There are plenty of useful schedule primitives in tvm. You can also find some tutorials that describe them in more details, such as (1). How to optimize convolution on GPU (2). Optimizing DepthwiseConv on NVIDIA GPU

However, their implementations are manually tuned for some special input shapes. In this section, we build a large enough space to cover the techniques used in these tutorials. Then we rely on the efficient auto-tuner to search through this space and pick some good configurations.

If you are familiar with writing cuda schedule, you can find the following template is very general. Actually this template can be easily modified to tune other operators such as depthwise convolution and gemm. In order to fully understand this template, you should be familiar with the schedule primitives and auto tuning API. You can refer to the above tutorials and autotvm tutorial

It is worth noting that the search space for a conv2d operator can be very large (at the level of 10^9 for some input shapes)

@autotvm.template("tutorial/conv2d_no_batching")
def conv2d_no_batching(N, H, W, CO, CI, KH, KW, stride, padding):
    assert N == 1, "Only consider batch_size = 1 in this template"

    data = te.placeholder((N, CI, H, W), name="data")
    kernel = te.placeholder((CO, CI, KH, KW), name="kernel")
    conv = topi.nn.conv2d_nchw(data, kernel, stride, padding, dilation=1, out_dtype="float32")
    s = te.create_schedule([conv.op])

    ##### space definition begin #####
    n, f, y, x = s[conv].op.axis
    rc, ry, rx = s[conv].op.reduce_axis

    cfg = autotvm.get_config()
    cfg.define_split("tile_f", f, num_outputs=4)
    cfg.define_split("tile_y", y, num_outputs=4)
    cfg.define_split("tile_x", x, num_outputs=4)
    cfg.define_split("tile_rc", rc, num_outputs=3)
    cfg.define_split("tile_ry", ry, num_outputs=3)
    cfg.define_split("tile_rx", rx, num_outputs=3)
    cfg.define_knob("auto_unroll_max_step", [0, 512, 1500])
    cfg.define_knob("unroll_explicit", [0, 1])
    ##### space definition end #####

    # inline padding
    pad_data = s[conv].op.input_tensors[0]
    s[pad_data].compute_inline()
    data, raw_data = pad_data, data

    output = conv
    OL = s.cache_write(conv, "local")

    # create cache stage
    AA = s.cache_read(data, "shared", [OL])
    WW = s.cache_read(kernel, "shared", [OL])
    AL = s.cache_read(AA, "local", [OL])
    WL = s.cache_read(WW, "local", [OL])

    # tile and bind spatial axes
    n, f, y, x = s[output].op.axis
    bf, vf, tf, fi = cfg["tile_f"].apply(s, output, f)
    by, vy, ty, yi = cfg["tile_y"].apply(s, output, y)
    bx, vx, tx, xi = cfg["tile_x"].apply(s, output, x)
    kernel_scope = n  # this is the scope to attach global config inside this kernel

    s[output].bind(bf, te.thread_axis("blockIdx.z"))
    s[output].bind(by, te.thread_axis("blockIdx.y"))
    s[output].bind(bx, te.thread_axis("blockIdx.x"))
    s[output].bind(vf, te.thread_axis("vthread"))
    s[output].bind(vy, te.thread_axis("vthread"))
    s[output].bind(vx, te.thread_axis("vthread"))
    s[output].bind(tf, te.thread_axis("threadIdx.z"))
    s[output].bind(ty, te.thread_axis("threadIdx.y"))
    s[output].bind(tx, te.thread_axis("threadIdx.x"))
    s[output].reorder(n, bf, by, bx, vf, vy, vx, tf, ty, tx, fi, yi, xi)
    s[OL].compute_at(s[output], tx)

    # tile reduction axes
    n, f, y, x = s[OL].op.axis
    rc, ry, rx = s[OL].op.reduce_axis
    rco, rcm, rci = cfg["tile_rc"].apply(s, OL, rc)
    ryo, rym, ryi = cfg["tile_rx"].apply(s, OL, ry)
    rxo, rxm, rxi = cfg["tile_ry"].apply(s, OL, rx)
    s[OL].reorder(rco, ryo, rxo, rcm, rym, rxm, rci, ryi, rxi, n, f, y, x)

    s[AA].compute_at(s[OL], rxo)
    s[WW].compute_at(s[OL], rxo)
    s[AL].compute_at(s[OL], rxm)
    s[WL].compute_at(s[OL], rxm)

    # cooperative fetching
    for load in [AA, WW]:
        n, f, y, x = s[load].op.axis
        fused = s[load].fuse(n, f, y, x)
        tz, fused = s[load].split(fused, nparts=cfg["tile_f"].size[2])
        ty, fused = s[load].split(fused, nparts=cfg["tile_y"].size[2])
        tx, fused = s[load].split(fused, nparts=cfg["tile_x"].size[2])
        s[load].bind(tz, te.thread_axis("threadIdx.z"))
        s[load].bind(ty, te.thread_axis("threadIdx.y"))
        s[load].bind(tx, te.thread_axis("threadIdx.x"))

    # tune unroll
    s[output].pragma(kernel_scope, "auto_unroll_max_step", cfg["auto_unroll_max_step"].val)
    s[output].pragma(kernel_scope, "unroll_explicit", cfg["unroll_explicit"].val)

    return s, [raw_data, kernel, conv]

Step 2: Search through the space

We pick the last layer on resnet as test case. Since our space is very large, XGBoostTuner is most suitable for our case. Here we only do 20 trials for demonstration. In practice, making 1000 trials usually can find some good kernels for this template

# logging config (for printing tuning log to screen)
logging.getLogger("autotvm").setLevel(logging.DEBUG)
logging.getLogger("autotvm").addHandler(logging.StreamHandler(sys.stdout))

# the last layer in resnet
N, H, W, CO, CI, KH, KW, strides, padding = 1, 7, 7, 512, 512, 3, 3, (1, 1), (1, 1)
task = autotvm.task.create(
    "tutorial/conv2d_no_batching", args=(N, H, W, CO, CI, KH, KW, strides, padding), target="cuda"
)
print(task.config_space)

# Use local gpu, measure 10 times for every config to reduce variance
# The timeout of compiling a program is 10 seconds, the timeout for running is 4 seconds
measure_option = autotvm.measure_option(
    builder=autotvm.LocalBuilder(),
    runner=autotvm.LocalRunner(repeat=3, min_repeat_ms=100, timeout=4),
)

# Begin tuning, log records to file `conv2d.log`
# During tuning we will also try many invalid configs, so you are expected to
# see many error reports. As long as you can see non-zero GFLOPS, it is okay.
tuner = autotvm.tuner.XGBTuner(task)
tuner.tune(
    n_trial=20,
    measure_option=measure_option,
    callbacks=[autotvm.callback.log_to_file("conv2d.log")],
)

Out:

ConfigSpace (len=10454400, space_map=
   0 tile_f: Split(policy=factors, product=512, num_outputs=4) len=220
   1 tile_y: Split(policy=factors, product=7, num_outputs=4) len=4
   2 tile_x: Split(policy=factors, product=7, num_outputs=4) len=4
   3 tile_rc: Split(policy=factors, product=512, num_outputs=3) len=55
   4 tile_ry: Split(policy=factors, product=3, num_outputs=3) len=3
   5 tile_rx: Split(policy=factors, product=3, num_outputs=3) len=3
   6 auto_unroll_max_step: OtherOption([0, 512, 1500]) len=3
   7 unroll_explicit: OtherOption([0, 1]) len=2
)
Get devices for measurement successfully!
No: 1   GFLOPS: 0.00/0.00       result: MeasureResult(costs=(InstantiationError('Traceback (most recent call last):\n  24: TVMFuncCall\n        at /workspace/src/runtime/c_runtime_api.cc:474\n  23: tvm::runtime::PackedFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /workspace/include/tvm/runtime/packed_func.h:1151\n  22: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  21: operator()\n        at /workspace/include/tvm/runtime/packed_func.h:1480\n  20: unpack_call<tvm::IRModule, 5, tvm::<lambda(tvm::te::Schedule, const tvm::runtime::Array<tvm::runtime::ObjectRef>&, const tvm::runtime::String&, const tvm::runtime::Map<tvm::te::Tensor, tvm::tir::Buffer>&, bool)> >\n        at /workspace/include/tvm/runtime/packed_func.h:1421\n  19: run<>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  18: run<tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  17: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  16: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  15: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  14: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1397\n  13: operator()\n        at /workspace/src/driver/driver_api.cc:369\n  12: tvm::LowerSchedule(tvm::te::Schedule, tvm::runtime::Array<tvm::runtime::ObjectRef, void> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unordered_map<tvm::te::Tensor, tvm::tir::Buffer, std::hash<tvm::te::Tensor>, std::equal_to<tvm::te::Tensor>, std::allocator<std::pair<tvm::te::Tensor const, tvm::tir::Buffer> > > const&, bool)\n        at /workspace/src/driver/driver_api.cc:356\n  11: tvm::LowerWithPassList(tvm::IRModule, tvm::runtime::Array<tvm::transform::Pass, void>)\n        at /workspace/src/driver/driver_api.cc:265\n  10: tvm::transform::Pass::operator()(tvm::IRModule) const\n        at /workspace/src/ir/transform.cc:255\n  9: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  8: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:481\n  7: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  6: tvm::tir::transform::PrimFuncPassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/tir/ir/transform.cc:100\n  5: tvm::runtime::TypedPackedFunc<tvm::tir::PrimFunc (tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext)>::operator()(tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext) const\n        at /workspace/include/tvm/runtime/packed_func.h:1498\n  4: tvm::tir::PrimFunc tvm::runtime::detail::typed_packed_call_dispatcher<tvm::tir::PrimFunc>::run<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::runtime::PackedFunc const&, tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&)\n        at /workspace/include/tvm/runtime/packed_func.h:1444\n  3: tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&) const\n        at /workspace/include/tvm/runtime/packed_func.h:1369\n  2: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  1: _M_invoke\n        at /usr/include/c++/7/bits/std_function.h:316\n  0: operator()\n        at /workspace/src/runtime/c_runtime_api.cc:525\n  File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback\n  File "/workspace/docs/../python/tvm/autotvm/measure/measure_methods.py", line 786, in verify_pass\n    raise InstantiationError("Skipped because of invalid gpu kernel")\ntvm.autotvm.task.space.InstantiationError: Skipped because of invalid gpu kernel',),), error_no=1, all_cost=0.10827493667602539, timestamp=1627516438.3094618)    [('tile_f', [-1, 4, 4, 2]), ('tile_y', [-1, 7, 1, 1]), ('tile_x', [-1, 1, 7, 1]), ('tile_rc', [-1, 128, 2]), ('tile_ry', [-1, 1, 3]), ('tile_rx', [-1, 3, 1]), ('auto_unroll_max_step', 512), ('unroll_explicit', 1)],None,7999494
No: 2   GFLOPS: 0.00/0.00       result: MeasureResult(costs=(InstantiationError('Traceback (most recent call last):\n  24: TVMFuncCall\n        at /workspace/src/runtime/c_runtime_api.cc:474\n  23: tvm::runtime::PackedFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /workspace/include/tvm/runtime/packed_func.h:1151\n  22: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  21: operator()\n        at /workspace/include/tvm/runtime/packed_func.h:1480\n  20: unpack_call<tvm::IRModule, 5, tvm::<lambda(tvm::te::Schedule, const tvm::runtime::Array<tvm::runtime::ObjectRef>&, const tvm::runtime::String&, const tvm::runtime::Map<tvm::te::Tensor, tvm::tir::Buffer>&, bool)> >\n        at /workspace/include/tvm/runtime/packed_func.h:1421\n  19: run<>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  18: run<tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  17: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  16: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  15: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  14: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1397\n  13: operator()\n        at /workspace/src/driver/driver_api.cc:369\n  12: tvm::LowerSchedule(tvm::te::Schedule, tvm::runtime::Array<tvm::runtime::ObjectRef, void> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unordered_map<tvm::te::Tensor, tvm::tir::Buffer, std::hash<tvm::te::Tensor>, std::equal_to<tvm::te::Tensor>, std::allocator<std::pair<tvm::te::Tensor const, tvm::tir::Buffer> > > const&, bool)\n        at /workspace/src/driver/driver_api.cc:356\n  11: tvm::LowerWithPassList(tvm::IRModule, tvm::runtime::Array<tvm::transform::Pass, void>)\n        at /workspace/src/driver/driver_api.cc:265\n  10: tvm::transform::Pass::operator()(tvm::IRModule) const\n        at /workspace/src/ir/transform.cc:255\n  9: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  8: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:481\n  7: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  6: tvm::tir::transform::PrimFuncPassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/tir/ir/transform.cc:100\n  5: tvm::runtime::TypedPackedFunc<tvm::tir::PrimFunc (tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext)>::operator()(tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext) const\n        at /workspace/include/tvm/runtime/packed_func.h:1498\n  4: tvm::tir::PrimFunc tvm::runtime::detail::typed_packed_call_dispatcher<tvm::tir::PrimFunc>::run<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::runtime::PackedFunc const&, tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&)\n        at /workspace/include/tvm/runtime/packed_func.h:1444\n  3: tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&) const\n        at /workspace/include/tvm/runtime/packed_func.h:1369\n  2: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  1: _M_invoke\n        at /usr/include/c++/7/bits/std_function.h:316\n  0: operator()\n        at /workspace/src/runtime/c_runtime_api.cc:525\n  File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback\n  File "/workspace/docs/../python/tvm/autotvm/measure/measure_methods.py", line 786, in verify_pass\n    raise InstantiationError("Skipped because of invalid gpu kernel")\ntvm.autotvm.task.space.InstantiationError: Skipped because of invalid gpu kernel',),), error_no=1, all_cost=0.10191011428833008, timestamp=1627516438.461217)     [('tile_f', [-1, 1, 8, 2]), ('tile_y', [-1, 1, 7, 1]), ('tile_x', [-1, 1, 7, 1]), ('tile_rc', [-1, 1, 64]), ('tile_ry', [-1, 1, 3]), ('tile_rx', [-1, 1, 3]), ('auto_unroll_max_step', 1500), ('unroll_explicit', 0)],None,5194279
No: 3   GFLOPS: 0.00/0.00       result: MeasureResult(costs=(InstantiationError('Traceback (most recent call last):\n  24: TVMFuncCall\n        at /workspace/src/runtime/c_runtime_api.cc:474\n  23: tvm::runtime::PackedFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /workspace/include/tvm/runtime/packed_func.h:1151\n  22: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  21: operator()\n        at /workspace/include/tvm/runtime/packed_func.h:1480\n  20: unpack_call<tvm::IRModule, 5, tvm::<lambda(tvm::te::Schedule, const tvm::runtime::Array<tvm::runtime::ObjectRef>&, const tvm::runtime::String&, const tvm::runtime::Map<tvm::te::Tensor, tvm::tir::Buffer>&, bool)> >\n        at /workspace/include/tvm/runtime/packed_func.h:1421\n  19: run<>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  18: run<tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  17: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  16: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  15: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  14: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1397\n  13: operator()\n        at /workspace/src/driver/driver_api.cc:369\n  12: tvm::LowerSchedule(tvm::te::Schedule, tvm::runtime::Array<tvm::runtime::ObjectRef, void> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unordered_map<tvm::te::Tensor, tvm::tir::Buffer, std::hash<tvm::te::Tensor>, std::equal_to<tvm::te::Tensor>, std::allocator<std::pair<tvm::te::Tensor const, tvm::tir::Buffer> > > const&, bool)\n        at /workspace/src/driver/driver_api.cc:356\n  11: tvm::LowerWithPassList(tvm::IRModule, tvm::runtime::Array<tvm::transform::Pass, void>)\n        at /workspace/src/driver/driver_api.cc:265\n  10: tvm::transform::Pass::operator()(tvm::IRModule) const\n        at /workspace/src/ir/transform.cc:255\n  9: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  8: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:481\n  7: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  6: tvm::tir::transform::PrimFuncPassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/tir/ir/transform.cc:100\n  5: tvm::runtime::TypedPackedFunc<tvm::tir::PrimFunc (tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext)>::operator()(tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext) const\n        at /workspace/include/tvm/runtime/packed_func.h:1498\n  4: tvm::tir::PrimFunc tvm::runtime::detail::typed_packed_call_dispatcher<tvm::tir::PrimFunc>::run<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::runtime::PackedFunc const&, tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&)\n        at /workspace/include/tvm/runtime/packed_func.h:1444\n  3: tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&) const\n        at /workspace/include/tvm/runtime/packed_func.h:1369\n  2: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  1: _M_invoke\n        at /usr/include/c++/7/bits/std_function.h:316\n  0: operator()\n        at /workspace/src/runtime/c_runtime_api.cc:525\n  File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback\n  File "/workspace/docs/../python/tvm/autotvm/measure/measure_methods.py", line 786, in verify_pass\n    raise InstantiationError("Skipped because of invalid gpu kernel")\ntvm.autotvm.task.space.InstantiationError: Skipped because of invalid gpu kernel',),), error_no=1, all_cost=0.12183070182800293, timestamp=1627516438.5174346)    [('tile_f', [-1, 8, 32, 1]), ('tile_y', [-1, 1, 1, 7]), ('tile_x', [-1, 1, 7, 1]), ('tile_rc', [-1, 2, 64]), ('tile_ry', [-1, 3, 1]), ('tile_rx', [-1, 1, 1]), ('auto_unroll_max_step', 1500), ('unroll_explicit', 1)],None,9069983
No: 4   GFLOPS: 0.00/0.00       result: MeasureResult(costs=(InstantiationError('Traceback (most recent call last):\n  24: TVMFuncCall\n        at /workspace/src/runtime/c_runtime_api.cc:474\n  23: tvm::runtime::PackedFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /workspace/include/tvm/runtime/packed_func.h:1151\n  22: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  21: operator()\n        at /workspace/include/tvm/runtime/packed_func.h:1480\n  20: unpack_call<tvm::IRModule, 5, tvm::<lambda(tvm::te::Schedule, const tvm::runtime::Array<tvm::runtime::ObjectRef>&, const tvm::runtime::String&, const tvm::runtime::Map<tvm::te::Tensor, tvm::tir::Buffer>&, bool)> >\n        at /workspace/include/tvm/runtime/packed_func.h:1421\n  19: run<>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  18: run<tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  17: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  16: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  15: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  14: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1397\n  13: operator()\n        at /workspace/src/driver/driver_api.cc:369\n  12: tvm::LowerSchedule(tvm::te::Schedule, tvm::runtime::Array<tvm::runtime::ObjectRef, void> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unordered_map<tvm::te::Tensor, tvm::tir::Buffer, std::hash<tvm::te::Tensor>, std::equal_to<tvm::te::Tensor>, std::allocator<std::pair<tvm::te::Tensor const, tvm::tir::Buffer> > > const&, bool)\n        at /workspace/src/driver/driver_api.cc:356\n  11: tvm::LowerWithPassList(tvm::IRModule, tvm::runtime::Array<tvm::transform::Pass, void>)\n        at /workspace/src/driver/driver_api.cc:265\n  10: tvm::transform::Pass::operator()(tvm::IRModule) const\n        at /workspace/src/ir/transform.cc:255\n  9: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  8: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:481\n  7: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  6: tvm::tir::transform::PrimFuncPassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/tir/ir/transform.cc:100\n  5: tvm::runtime::TypedPackedFunc<tvm::tir::PrimFunc (tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext)>::operator()(tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext) const\n        at /workspace/include/tvm/runtime/packed_func.h:1498\n  4: tvm::tir::PrimFunc tvm::runtime::detail::typed_packed_call_dispatcher<tvm::tir::PrimFunc>::run<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::runtime::PackedFunc const&, tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&)\n        at /workspace/include/tvm/runtime/packed_func.h:1444\n  3: tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&) const\n        at /workspace/include/tvm/runtime/packed_func.h:1369\n  2: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  1: _M_invoke\n        at /usr/include/c++/7/bits/std_function.h:316\n  0: operator()\n        at /workspace/src/runtime/c_runtime_api.cc:525\n  File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback\n  File "/workspace/docs/../python/tvm/autotvm/measure/measure_methods.py", line 786, in verify_pass\n    raise InstantiationError("Skipped because of invalid gpu kernel")\ntvm.autotvm.task.space.InstantiationError: Skipped because of invalid gpu kernel',),), error_no=1, all_cost=0.09532999992370605, timestamp=1627516438.5573344)    [('tile_f', [-1, 16, 16, 1]), ('tile_y', [-1, 7, 1, 1]), ('tile_x', [-1, 7, 1, 1]), ('tile_rc', [-1, 16, 32]), ('tile_ry', [-1, 1, 1]), ('tile_rx', [-1, 3, 1]), ('auto_unroll_max_step', 0), ('unroll_explicit', 0)],None,736818
No: 5   GFLOPS: 0.00/0.00       result: MeasureResult(costs=(InstantiationError('Traceback (most recent call last):\n  24: TVMFuncCall\n        at /workspace/src/runtime/c_runtime_api.cc:474\n  23: tvm::runtime::PackedFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /workspace/include/tvm/runtime/packed_func.h:1151\n  22: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  21: operator()\n        at /workspace/include/tvm/runtime/packed_func.h:1480\n  20: unpack_call<tvm::IRModule, 5, tvm::<lambda(tvm::te::Schedule, const tvm::runtime::Array<tvm::runtime::ObjectRef>&, const tvm::runtime::String&, const tvm::runtime::Map<tvm::te::Tensor, tvm::tir::Buffer>&, bool)> >\n        at /workspace/include/tvm/runtime/packed_func.h:1421\n  19: run<>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  18: run<tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  17: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  16: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  15: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  14: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1397\n  13: operator()\n        at /workspace/src/driver/driver_api.cc:369\n  12: tvm::LowerSchedule(tvm::te::Schedule, tvm::runtime::Array<tvm::runtime::ObjectRef, void> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unordered_map<tvm::te::Tensor, tvm::tir::Buffer, std::hash<tvm::te::Tensor>, std::equal_to<tvm::te::Tensor>, std::allocator<std::pair<tvm::te::Tensor const, tvm::tir::Buffer> > > const&, bool)\n        at /workspace/src/driver/driver_api.cc:356\n  11: tvm::LowerWithPassList(tvm::IRModule, tvm::runtime::Array<tvm::transform::Pass, void>)\n        at /workspace/src/driver/driver_api.cc:265\n  10: tvm::transform::Pass::operator()(tvm::IRModule) const\n        at /workspace/src/ir/transform.cc:255\n  9: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  8: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:481\n  7: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  6: tvm::tir::transform::PrimFuncPassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/tir/ir/transform.cc:100\n  5: tvm::runtime::TypedPackedFunc<tvm::tir::PrimFunc (tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext)>::operator()(tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext) const\n        at /workspace/include/tvm/runtime/packed_func.h:1498\n  4: tvm::tir::PrimFunc tvm::runtime::detail::typed_packed_call_dispatcher<tvm::tir::PrimFunc>::run<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::runtime::PackedFunc const&, tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&)\n        at /workspace/include/tvm/runtime/packed_func.h:1444\n  3: tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&) const\n        at /workspace/include/tvm/runtime/packed_func.h:1369\n  2: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  1: _M_invoke\n        at /usr/include/c++/7/bits/std_function.h:316\n  0: operator()\n        at /workspace/src/runtime/c_runtime_api.cc:525\n  File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback\n  File "/workspace/docs/../python/tvm/autotvm/measure/measure_methods.py", line 786, in verify_pass\n    raise InstantiationError("Skipped because of invalid gpu kernel")\ntvm.autotvm.task.space.InstantiationError: Skipped because of invalid gpu kernel',),), error_no=1, all_cost=0.06326556205749512, timestamp=1627516440.2049036)    [('tile_f', [-1, 4, 4, 32]), ('tile_y', [-1, 1, 1, 7]), ('tile_x', [-1, 1, 7, 1]), ('tile_rc', [-1, 1, 128]), ('tile_ry', [-1, 1, 3]), ('tile_rx', [-1, 3, 1]), ('auto_unroll_max_step', 512), ('unroll_explicit', 0)],None,2885496
No: 6   GFLOPS: 32.89/32.89     result: MeasureResult(costs=(0.007039407533333333,), error_no=0, all_cost=1.7559926509857178, timestamp=1627516442.2039375)     [('tile_f', [-1, 1, 1, 1]), ('tile_y', [-1, 1, 1, 1]), ('tile_x', [-1, 1, 7, 1]), ('tile_rc', [-1, 4, 4]), ('tile_ry', [-1, 3, 1]), ('tile_rx', [-1, 1, 1]), ('auto_unroll_max_step', 1500), ('unroll_explicit', 0)],None,3754080
No: 7   GFLOPS: 0.00/32.89      result: MeasureResult(costs=(InstantiationError('Traceback (most recent call last):\n  24: TVMFuncCall\n        at /workspace/src/runtime/c_runtime_api.cc:474\n  23: tvm::runtime::PackedFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /workspace/include/tvm/runtime/packed_func.h:1151\n  22: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  21: operator()\n        at /workspace/include/tvm/runtime/packed_func.h:1480\n  20: unpack_call<tvm::IRModule, 5, tvm::<lambda(tvm::te::Schedule, const tvm::runtime::Array<tvm::runtime::ObjectRef>&, const tvm::runtime::String&, const tvm::runtime::Map<tvm::te::Tensor, tvm::tir::Buffer>&, bool)> >\n        at /workspace/include/tvm/runtime/packed_func.h:1421\n  19: run<>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  18: run<tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  17: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  16: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  15: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  14: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1397\n  13: operator()\n        at /workspace/src/driver/driver_api.cc:369\n  12: tvm::LowerSchedule(tvm::te::Schedule, tvm::runtime::Array<tvm::runtime::ObjectRef, void> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unordered_map<tvm::te::Tensor, tvm::tir::Buffer, std::hash<tvm::te::Tensor>, std::equal_to<tvm::te::Tensor>, std::allocator<std::pair<tvm::te::Tensor const, tvm::tir::Buffer> > > const&, bool)\n        at /workspace/src/driver/driver_api.cc:356\n  11: tvm::LowerWithPassList(tvm::IRModule, tvm::runtime::Array<tvm::transform::Pass, void>)\n        at /workspace/src/driver/driver_api.cc:265\n  10: tvm::transform::Pass::operator()(tvm::IRModule) const\n        at /workspace/src/ir/transform.cc:255\n  9: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  8: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:481\n  7: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  6: tvm::tir::transform::PrimFuncPassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/tir/ir/transform.cc:100\n  5: tvm::runtime::TypedPackedFunc<tvm::tir::PrimFunc (tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext)>::operator()(tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext) const\n        at /workspace/include/tvm/runtime/packed_func.h:1498\n  4: tvm::tir::PrimFunc tvm::runtime::detail::typed_packed_call_dispatcher<tvm::tir::PrimFunc>::run<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::runtime::PackedFunc const&, tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&)\n        at /workspace/include/tvm/runtime/packed_func.h:1444\n  3: tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&) const\n        at /workspace/include/tvm/runtime/packed_func.h:1369\n  2: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  1: _M_invoke\n        at /usr/include/c++/7/bits/std_function.h:316\n  0: operator()\n        at /workspace/src/runtime/c_runtime_api.cc:525\n  File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback\n  File "/workspace/docs/../python/tvm/autotvm/measure/measure_methods.py", line 786, in verify_pass\n    raise InstantiationError("Skipped because of invalid gpu kernel")\ntvm.autotvm.task.space.InstantiationError: Skipped because of invalid gpu kernel',),), error_no=1, all_cost=0.08770942687988281, timestamp=1627516441.114616)     [('tile_f', [-1, 1, 16, 32]), ('tile_y', [-1, 1, 1, 1]), ('tile_x', [-1, 1, 7, 1]), ('tile_rc', [-1, 256, 1]), ('tile_ry', [-1, 1, 3]), ('tile_rx', [-1, 3, 1]), ('auto_unroll_max_step', 0), ('unroll_explicit', 1)],None,6225319
No: 8   GFLOPS: 0.00/32.89      result: MeasureResult(costs=(InstantiationError('Traceback (most recent call last):\n  24: TVMFuncCall\n        at /workspace/src/runtime/c_runtime_api.cc:474\n  23: tvm::runtime::PackedFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /workspace/include/tvm/runtime/packed_func.h:1151\n  22: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  21: operator()\n        at /workspace/include/tvm/runtime/packed_func.h:1480\n  20: unpack_call<tvm::IRModule, 5, tvm::<lambda(tvm::te::Schedule, const tvm::runtime::Array<tvm::runtime::ObjectRef>&, const tvm::runtime::String&, const tvm::runtime::Map<tvm::te::Tensor, tvm::tir::Buffer>&, bool)> >\n        at /workspace/include/tvm/runtime/packed_func.h:1421\n  19: run<>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  18: run<tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  17: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  16: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  15: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  14: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1397\n  13: operator()\n        at /workspace/src/driver/driver_api.cc:369\n  12: tvm::LowerSchedule(tvm::te::Schedule, tvm::runtime::Array<tvm::runtime::ObjectRef, void> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unordered_map<tvm::te::Tensor, tvm::tir::Buffer, std::hash<tvm::te::Tensor>, std::equal_to<tvm::te::Tensor>, std::allocator<std::pair<tvm::te::Tensor const, tvm::tir::Buffer> > > const&, bool)\n        at /workspace/src/driver/driver_api.cc:356\n  11: tvm::LowerWithPassList(tvm::IRModule, tvm::runtime::Array<tvm::transform::Pass, void>)\n        at /workspace/src/driver/driver_api.cc:265\n  10: tvm::transform::Pass::operator()(tvm::IRModule) const\n        at /workspace/src/ir/transform.cc:255\n  9: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  8: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:481\n  7: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  6: tvm::tir::transform::PrimFuncPassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/tir/ir/transform.cc:100\n  5: tvm::runtime::TypedPackedFunc<tvm::tir::PrimFunc (tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext)>::operator()(tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext) const\n        at /workspace/include/tvm/runtime/packed_func.h:1498\n  4: tvm::tir::PrimFunc tvm::runtime::detail::typed_packed_call_dispatcher<tvm::tir::PrimFunc>::run<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::runtime::PackedFunc const&, tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&)\n        at /workspace/include/tvm/runtime/packed_func.h:1444\n  3: tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&) const\n        at /workspace/include/tvm/runtime/packed_func.h:1369\n  2: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  1: _M_invoke\n        at /usr/include/c++/7/bits/std_function.h:316\n  0: operator()\n        at /workspace/src/runtime/c_runtime_api.cc:525\n  File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback\n  File "/workspace/docs/../python/tvm/autotvm/measure/measure_methods.py", line 786, in verify_pass\n    raise InstantiationError("Skipped because of invalid gpu kernel")\ntvm.autotvm.task.space.InstantiationError: Skipped because of invalid gpu kernel',),), error_no=1, all_cost=0.07125639915466309, timestamp=1627516441.1147218)    [('tile_f', [-1, 2, 1, 32]), ('tile_y', [-1, 1, 1, 1]), ('tile_x', [-1, 1, 1, 1]), ('tile_rc', [-1, 8, 64]), ('tile_ry', [-1, 3, 1]), ('tile_rx', [-1, 3, 1]), ('auto_unroll_max_step', 0), ('unroll_explicit', 0)],None,943546
No: 9   GFLOPS: 0.00/32.89      result: MeasureResult(costs=(InstantiationError('Traceback (most recent call last):\n  24: TVMFuncCall\n        at /workspace/src/runtime/c_runtime_api.cc:474\n  23: tvm::runtime::PackedFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /workspace/include/tvm/runtime/packed_func.h:1151\n  22: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  21: operator()\n        at /workspace/include/tvm/runtime/packed_func.h:1480\n  20: unpack_call<tvm::IRModule, 5, tvm::<lambda(tvm::te::Schedule, const tvm::runtime::Array<tvm::runtime::ObjectRef>&, const tvm::runtime::String&, const tvm::runtime::Map<tvm::te::Tensor, tvm::tir::Buffer>&, bool)> >\n        at /workspace/include/tvm/runtime/packed_func.h:1421\n  19: run<>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  18: run<tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  17: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  16: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  15: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  14: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1397\n  13: operator()\n        at /workspace/src/driver/driver_api.cc:369\n  12: tvm::LowerSchedule(tvm::te::Schedule, tvm::runtime::Array<tvm::runtime::ObjectRef, void> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unordered_map<tvm::te::Tensor, tvm::tir::Buffer, std::hash<tvm::te::Tensor>, std::equal_to<tvm::te::Tensor>, std::allocator<std::pair<tvm::te::Tensor const, tvm::tir::Buffer> > > const&, bool)\n        at /workspace/src/driver/driver_api.cc:356\n  11: tvm::LowerWithPassList(tvm::IRModule, tvm::runtime::Array<tvm::transform::Pass, void>)\n        at /workspace/src/driver/driver_api.cc:265\n  10: tvm::transform::Pass::operator()(tvm::IRModule) const\n        at /workspace/src/ir/transform.cc:255\n  9: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  8: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:481\n  7: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  6: tvm::tir::transform::PrimFuncPassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/tir/ir/transform.cc:100\n  5: tvm::runtime::TypedPackedFunc<tvm::tir::PrimFunc (tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext)>::operator()(tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext) const\n        at /workspace/include/tvm/runtime/packed_func.h:1498\n  4: tvm::tir::PrimFunc tvm::runtime::detail::typed_packed_call_dispatcher<tvm::tir::PrimFunc>::run<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::runtime::PackedFunc const&, tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&)\n        at /workspace/include/tvm/runtime/packed_func.h:1444\n  3: tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&) const\n        at /workspace/include/tvm/runtime/packed_func.h:1369\n  2: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  1: _M_invoke\n        at /usr/include/c++/7/bits/std_function.h:316\n  0: operator()\n        at /workspace/src/runtime/c_runtime_api.cc:525\n  File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback\n  File "/workspace/docs/../python/tvm/autotvm/measure/measure_methods.py", line 786, in verify_pass\n    raise InstantiationError("Skipped because of invalid gpu kernel")\ntvm.autotvm.task.space.InstantiationError: Skipped because of invalid gpu kernel',),), error_no=1, all_cost=0.08841848373413086, timestamp=1627516443.4757838)    [('tile_f', [-1, 4, 16, 4]), ('tile_y', [-1, 1, 1, 7]), ('tile_x', [-1, 1, 1, 7]), ('tile_rc', [-1, 16, 32]), ('tile_ry', [-1, 1, 3]), ('tile_rx', [-1, 3, 1]), ('auto_unroll_max_step', 512), ('unroll_explicit', 0)],None,2868708
No: 10  GFLOPS: 0.00/32.89      result: MeasureResult(costs=(TimeoutError(),), error_no=6, all_cost=10, timestamp=1627516453.343513)    [('tile_f', [-1, 32, 2, 4]), ('tile_y', [-1, 1, 7, 1]), ('tile_x', [-1, 1, 1, 7]), ('tile_rc', [-1, 4, 2]), ('tile_ry', [-1, 1, 1]), ('tile_rx', [-1, 1, 3]), ('auto_unroll_max_step', 1500), ('unroll_explicit', 0)],None,4691833
No: 11  GFLOPS: 0.00/32.89      result: MeasureResult(costs=(InstantiationError('Traceback (most recent call last):\n  24: TVMFuncCall\n        at /workspace/src/runtime/c_runtime_api.cc:474\n  23: tvm::runtime::PackedFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /workspace/include/tvm/runtime/packed_func.h:1151\n  22: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  21: operator()\n        at /workspace/include/tvm/runtime/packed_func.h:1480\n  20: unpack_call<tvm::IRModule, 5, tvm::<lambda(tvm::te::Schedule, const tvm::runtime::Array<tvm::runtime::ObjectRef>&, const tvm::runtime::String&, const tvm::runtime::Map<tvm::te::Tensor, tvm::tir::Buffer>&, bool)> >\n        at /workspace/include/tvm/runtime/packed_func.h:1421\n  19: run<>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  18: run<tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  17: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  16: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  15: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  14: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1397\n  13: operator()\n        at /workspace/src/driver/driver_api.cc:369\n  12: tvm::LowerSchedule(tvm::te::Schedule, tvm::runtime::Array<tvm::runtime::ObjectRef, void> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unordered_map<tvm::te::Tensor, tvm::tir::Buffer, std::hash<tvm::te::Tensor>, std::equal_to<tvm::te::Tensor>, std::allocator<std::pair<tvm::te::Tensor const, tvm::tir::Buffer> > > const&, bool)\n        at /workspace/src/driver/driver_api.cc:356\n  11: tvm::LowerWithPassList(tvm::IRModule, tvm::runtime::Array<tvm::transform::Pass, void>)\n        at /workspace/src/driver/driver_api.cc:265\n  10: tvm::transform::Pass::operator()(tvm::IRModule) const\n        at /workspace/src/ir/transform.cc:255\n  9: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  8: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:481\n  7: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  6: tvm::tir::transform::PrimFuncPassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/tir/ir/transform.cc:100\n  5: tvm::runtime::TypedPackedFunc<tvm::tir::PrimFunc (tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext)>::operator()(tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext) const\n        at /workspace/include/tvm/runtime/packed_func.h:1498\n  4: tvm::tir::PrimFunc tvm::runtime::detail::typed_packed_call_dispatcher<tvm::tir::PrimFunc>::run<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::runtime::PackedFunc const&, tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&)\n        at /workspace/include/tvm/runtime/packed_func.h:1444\n  3: tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&) const\n        at /workspace/include/tvm/runtime/packed_func.h:1369\n  2: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  1: _M_invoke\n        at /usr/include/c++/7/bits/std_function.h:316\n  0: operator()\n        at /workspace/src/runtime/c_runtime_api.cc:525\n  File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback\n  File "/workspace/docs/../python/tvm/autotvm/measure/measure_methods.py", line 786, in verify_pass\n    raise InstantiationError("Skipped because of invalid gpu kernel")\ntvm.autotvm.task.space.InstantiationError: Skipped because of invalid gpu kernel',),), error_no=1, all_cost=0.09370756149291992, timestamp=1627516453.3436956)    [('tile_f', [-1, 1, 2, 64]), ('tile_y', [-1, 1, 1, 1]), ('tile_x', [-1, 1, 1, 1]), ('tile_rc', [-1, 4, 4]), ('tile_ry', [-1, 1, 3]), ('tile_rx', [-1, 3, 1]), ('auto_unroll_max_step', 0), ('unroll_explicit', 0)],None,1042124
No: 12  GFLOPS: 0.00/32.89      result: MeasureResult(costs=(InstantiationError('Traceback (most recent call last):\n  24: TVMFuncCall\n        at /workspace/src/runtime/c_runtime_api.cc:474\n  23: tvm::runtime::PackedFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /workspace/include/tvm/runtime/packed_func.h:1151\n  22: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  21: operator()\n        at /workspace/include/tvm/runtime/packed_func.h:1480\n  20: unpack_call<tvm::IRModule, 5, tvm::<lambda(tvm::te::Schedule, const tvm::runtime::Array<tvm::runtime::ObjectRef>&, const tvm::runtime::String&, const tvm::runtime::Map<tvm::te::Tensor, tvm::tir::Buffer>&, bool)> >\n        at /workspace/include/tvm/runtime/packed_func.h:1421\n  19: run<>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  18: run<tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  17: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  16: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  15: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  14: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1397\n  13: operator()\n        at /workspace/src/driver/driver_api.cc:369\n  12: tvm::LowerSchedule(tvm::te::Schedule, tvm::runtime::Array<tvm::runtime::ObjectRef, void> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unordered_map<tvm::te::Tensor, tvm::tir::Buffer, std::hash<tvm::te::Tensor>, std::equal_to<tvm::te::Tensor>, std::allocator<std::pair<tvm::te::Tensor const, tvm::tir::Buffer> > > const&, bool)\n        at /workspace/src/driver/driver_api.cc:356\n  11: tvm::LowerWithPassList(tvm::IRModule, tvm::runtime::Array<tvm::transform::Pass, void>)\n        at /workspace/src/driver/driver_api.cc:265\n  10: tvm::transform::Pass::operator()(tvm::IRModule) const\n        at /workspace/src/ir/transform.cc:255\n  9: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  8: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:481\n  7: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  6: tvm::tir::transform::PrimFuncPassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/tir/ir/transform.cc:100\n  5: tvm::runtime::TypedPackedFunc<tvm::tir::PrimFunc (tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext)>::operator()(tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext) const\n        at /workspace/include/tvm/runtime/packed_func.h:1498\n  4: tvm::tir::PrimFunc tvm::runtime::detail::typed_packed_call_dispatcher<tvm::tir::PrimFunc>::run<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::runtime::PackedFunc const&, tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&)\n        at /workspace/include/tvm/runtime/packed_func.h:1444\n  3: tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&) const\n        at /workspace/include/tvm/runtime/packed_func.h:1369\n  2: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  1: _M_invoke\n        at /usr/include/c++/7/bits/std_function.h:316\n  0: operator()\n        at /workspace/src/runtime/c_runtime_api.cc:525\n  File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback\n  File "/workspace/docs/../python/tvm/autotvm/measure/measure_methods.py", line 786, in verify_pass\n    raise InstantiationError("Skipped because of invalid gpu kernel")\ntvm.autotvm.task.space.InstantiationError: Skipped because of invalid gpu kernel',),), error_no=1, all_cost=0.09113740921020508, timestamp=1627516453.343805)     [('tile_f', [-1, 32, 1, 4]), ('tile_y', [-1, 1, 1, 7]), ('tile_x', [-1, 1, 7, 1]), ('tile_rc', [-1, 32, 16]), ('tile_ry', [-1, 1, 1]), ('tile_rx', [-1, 1, 3]), ('auto_unroll_max_step', 1500), ('unroll_explicit', 1)],None,10013405
No: 13  GFLOPS: 0.00/32.89      result: MeasureResult(costs=(InstantiationError('Traceback (most recent call last):\n  24: TVMFuncCall\n        at /workspace/src/runtime/c_runtime_api.cc:474\n  23: tvm::runtime::PackedFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /workspace/include/tvm/runtime/packed_func.h:1151\n  22: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  21: operator()\n        at /workspace/include/tvm/runtime/packed_func.h:1480\n  20: unpack_call<tvm::IRModule, 5, tvm::<lambda(tvm::te::Schedule, const tvm::runtime::Array<tvm::runtime::ObjectRef>&, const tvm::runtime::String&, const tvm::runtime::Map<tvm::te::Tensor, tvm::tir::Buffer>&, bool)> >\n        at /workspace/include/tvm/runtime/packed_func.h:1421\n  19: run<>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  18: run<tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  17: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  16: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  15: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  14: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1397\n  13: operator()\n        at /workspace/src/driver/driver_api.cc:369\n  12: tvm::LowerSchedule(tvm::te::Schedule, tvm::runtime::Array<tvm::runtime::ObjectRef, void> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unordered_map<tvm::te::Tensor, tvm::tir::Buffer, std::hash<tvm::te::Tensor>, std::equal_to<tvm::te::Tensor>, std::allocator<std::pair<tvm::te::Tensor const, tvm::tir::Buffer> > > const&, bool)\n        at /workspace/src/driver/driver_api.cc:356\n  11: tvm::LowerWithPassList(tvm::IRModule, tvm::runtime::Array<tvm::transform::Pass, void>)\n        at /workspace/src/driver/driver_api.cc:265\n  10: tvm::transform::Pass::operator()(tvm::IRModule) const\n        at /workspace/src/ir/transform.cc:255\n  9: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  8: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:481\n  7: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  6: tvm::tir::transform::PrimFuncPassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/tir/ir/transform.cc:100\n  5: tvm::runtime::TypedPackedFunc<tvm::tir::PrimFunc (tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext)>::operator()(tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext) const\n        at /workspace/include/tvm/runtime/packed_func.h:1498\n  4: tvm::tir::PrimFunc tvm::runtime::detail::typed_packed_call_dispatcher<tvm::tir::PrimFunc>::run<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::runtime::PackedFunc const&, tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&)\n        at /workspace/include/tvm/runtime/packed_func.h:1444\n  3: tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&) const\n        at /workspace/include/tvm/runtime/packed_func.h:1369\n  2: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  1: _M_invoke\n        at /usr/include/c++/7/bits/std_function.h:316\n  0: operator()\n        at /workspace/src/runtime/c_runtime_api.cc:525\n  File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback\n  File "/workspace/docs/../python/tvm/autotvm/measure/measure_methods.py", line 786, in verify_pass\n    raise InstantiationError("Skipped because of invalid gpu kernel")\ntvm.autotvm.task.space.InstantiationError: Skipped because of invalid gpu kernel',),), error_no=1, all_cost=0.06317591667175293, timestamp=1627516454.895323)     [('tile_f', [-1, 8, 8, 2]), ('tile_y', [-1, 1, 1, 1]), ('tile_x', [-1, 1, 7, 1]), ('tile_rc', [-1, 4, 32]), ('tile_ry', [-1, 3, 1]), ('tile_rx', [-1, 1, 3]), ('auto_unroll_max_step', 0), ('unroll_explicit', 1)],None,6732082
No: 14  GFLOPS: 0.00/32.89      result: MeasureResult(costs=(InstantiationError('Traceback (most recent call last):\n  24: TVMFuncCall\n        at /workspace/src/runtime/c_runtime_api.cc:474\n  23: tvm::runtime::PackedFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /workspace/include/tvm/runtime/packed_func.h:1151\n  22: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  21: operator()\n        at /workspace/include/tvm/runtime/packed_func.h:1480\n  20: unpack_call<tvm::IRModule, 5, tvm::<lambda(tvm::te::Schedule, const tvm::runtime::Array<tvm::runtime::ObjectRef>&, const tvm::runtime::String&, const tvm::runtime::Map<tvm::te::Tensor, tvm::tir::Buffer>&, bool)> >\n        at /workspace/include/tvm/runtime/packed_func.h:1421\n  19: run<>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  18: run<tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  17: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  16: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  15: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  14: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1397\n  13: operator()\n        at /workspace/src/driver/driver_api.cc:369\n  12: tvm::LowerSchedule(tvm::te::Schedule, tvm::runtime::Array<tvm::runtime::ObjectRef, void> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unordered_map<tvm::te::Tensor, tvm::tir::Buffer, std::hash<tvm::te::Tensor>, std::equal_to<tvm::te::Tensor>, std::allocator<std::pair<tvm::te::Tensor const, tvm::tir::Buffer> > > const&, bool)\n        at /workspace/src/driver/driver_api.cc:356\n  11: tvm::LowerWithPassList(tvm::IRModule, tvm::runtime::Array<tvm::transform::Pass, void>)\n        at /workspace/src/driver/driver_api.cc:265\n  10: tvm::transform::Pass::operator()(tvm::IRModule) const\n        at /workspace/src/ir/transform.cc:255\n  9: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  8: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:481\n  7: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  6: tvm::tir::transform::PrimFuncPassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/tir/ir/transform.cc:100\n  5: tvm::runtime::TypedPackedFunc<tvm::tir::PrimFunc (tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext)>::operator()(tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext) const\n        at /workspace/include/tvm/runtime/packed_func.h:1498\n  4: tvm::tir::PrimFunc tvm::runtime::detail::typed_packed_call_dispatcher<tvm::tir::PrimFunc>::run<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::runtime::PackedFunc const&, tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&)\n        at /workspace/include/tvm/runtime/packed_func.h:1444\n  3: tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&) const\n        at /workspace/include/tvm/runtime/packed_func.h:1369\n  2: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  1: _M_invoke\n        at /usr/include/c++/7/bits/std_function.h:316\n  0: operator()\n        at /workspace/src/runtime/c_runtime_api.cc:525\n  File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback\n  File "/workspace/docs/../python/tvm/autotvm/measure/measure_methods.py", line 786, in verify_pass\n    raise InstantiationError("Skipped because of invalid gpu kernel")\ntvm.autotvm.task.space.InstantiationError: Skipped because of invalid gpu kernel',),), error_no=1, all_cost=0.11940932273864746, timestamp=1627516455.0261977)    [('tile_f', [-1, 2, 4, 32]), ('tile_y', [-1, 7, 1, 1]), ('tile_x', [-1, 1, 1, 1]), ('tile_rc', [-1, 4, 128]), ('tile_ry', [-1, 1, 3]), ('tile_rx', [-1, 1, 1]), ('auto_unroll_max_step', 512), ('unroll_explicit', 1)],None,7536735
No: 15  GFLOPS: 0.00/32.89      result: MeasureResult(costs=(InstantiationError('Traceback (most recent call last):\n  24: TVMFuncCall\n        at /workspace/src/runtime/c_runtime_api.cc:474\n  23: tvm::runtime::PackedFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /workspace/include/tvm/runtime/packed_func.h:1151\n  22: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  21: operator()\n        at /workspace/include/tvm/runtime/packed_func.h:1480\n  20: unpack_call<tvm::IRModule, 5, tvm::<lambda(tvm::te::Schedule, const tvm::runtime::Array<tvm::runtime::ObjectRef>&, const tvm::runtime::String&, const tvm::runtime::Map<tvm::te::Tensor, tvm::tir::Buffer>&, bool)> >\n        at /workspace/include/tvm/runtime/packed_func.h:1421\n  19: run<>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  18: run<tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  17: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  16: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  15: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  14: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1397\n  13: operator()\n        at /workspace/src/driver/driver_api.cc:369\n  12: tvm::LowerSchedule(tvm::te::Schedule, tvm::runtime::Array<tvm::runtime::ObjectRef, void> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unordered_map<tvm::te::Tensor, tvm::tir::Buffer, std::hash<tvm::te::Tensor>, std::equal_to<tvm::te::Tensor>, std::allocator<std::pair<tvm::te::Tensor const, tvm::tir::Buffer> > > const&, bool)\n        at /workspace/src/driver/driver_api.cc:356\n  11: tvm::LowerWithPassList(tvm::IRModule, tvm::runtime::Array<tvm::transform::Pass, void>)\n        at /workspace/src/driver/driver_api.cc:265\n  10: tvm::transform::Pass::operator()(tvm::IRModule) const\n        at /workspace/src/ir/transform.cc:255\n  9: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  8: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:481\n  7: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  6: tvm::tir::transform::PrimFuncPassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/tir/ir/transform.cc:100\n  5: tvm::runtime::TypedPackedFunc<tvm::tir::PrimFunc (tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext)>::operator()(tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext) const\n        at /workspace/include/tvm/runtime/packed_func.h:1498\n  4: tvm::tir::PrimFunc tvm::runtime::detail::typed_packed_call_dispatcher<tvm::tir::PrimFunc>::run<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::runtime::PackedFunc const&, tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&)\n        at /workspace/include/tvm/runtime/packed_func.h:1444\n  3: tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&) const\n        at /workspace/include/tvm/runtime/packed_func.h:1369\n  2: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  1: _M_invoke\n        at /usr/include/c++/7/bits/std_function.h:316\n  0: operator()\n        at /workspace/src/runtime/c_runtime_api.cc:525\n  File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback\n  File "/workspace/docs/../python/tvm/autotvm/measure/measure_methods.py", line 786, in verify_pass\n    raise InstantiationError("Skipped because of invalid gpu kernel")\ntvm.autotvm.task.space.InstantiationError: Skipped because of invalid gpu kernel',),), error_no=1, all_cost=0.08919262886047363, timestamp=1627516455.0976448)    [('tile_f', [-1, 2, 1, 4]), ('tile_y', [-1, 1, 1, 7]), ('tile_x', [-1, 1, 1, 7]), ('tile_rc', [-1, 128, 4]), ('tile_ry', [-1, 1, 3]), ('tile_rx', [-1, 1, 1]), ('auto_unroll_max_step', 0), ('unroll_explicit', 0)],None,482121
No: 16  GFLOPS: 0.00/32.89      result: MeasureResult(costs=(InstantiationError('Traceback (most recent call last):\n  24: TVMFuncCall\n        at /workspace/src/runtime/c_runtime_api.cc:474\n  23: tvm::runtime::PackedFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /workspace/include/tvm/runtime/packed_func.h:1151\n  22: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  21: operator()\n        at /workspace/include/tvm/runtime/packed_func.h:1480\n  20: unpack_call<tvm::IRModule, 5, tvm::<lambda(tvm::te::Schedule, const tvm::runtime::Array<tvm::runtime::ObjectRef>&, const tvm::runtime::String&, const tvm::runtime::Map<tvm::te::Tensor, tvm::tir::Buffer>&, bool)> >\n        at /workspace/include/tvm/runtime/packed_func.h:1421\n  19: run<>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  18: run<tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  17: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  16: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  15: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  14: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1397\n  13: operator()\n        at /workspace/src/driver/driver_api.cc:369\n  12: tvm::LowerSchedule(tvm::te::Schedule, tvm::runtime::Array<tvm::runtime::ObjectRef, void> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unordered_map<tvm::te::Tensor, tvm::tir::Buffer, std::hash<tvm::te::Tensor>, std::equal_to<tvm::te::Tensor>, std::allocator<std::pair<tvm::te::Tensor const, tvm::tir::Buffer> > > const&, bool)\n        at /workspace/src/driver/driver_api.cc:356\n  11: tvm::LowerWithPassList(tvm::IRModule, tvm::runtime::Array<tvm::transform::Pass, void>)\n        at /workspace/src/driver/driver_api.cc:265\n  10: tvm::transform::Pass::operator()(tvm::IRModule) const\n        at /workspace/src/ir/transform.cc:255\n  9: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  8: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:481\n  7: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  6: tvm::tir::transform::PrimFuncPassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/tir/ir/transform.cc:100\n  5: tvm::runtime::TypedPackedFunc<tvm::tir::PrimFunc (tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext)>::operator()(tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext) const\n        at /workspace/include/tvm/runtime/packed_func.h:1498\n  4: tvm::tir::PrimFunc tvm::runtime::detail::typed_packed_call_dispatcher<tvm::tir::PrimFunc>::run<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::runtime::PackedFunc const&, tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&)\n        at /workspace/include/tvm/runtime/packed_func.h:1444\n  3: tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&) const\n        at /workspace/include/tvm/runtime/packed_func.h:1369\n  2: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  1: _M_invoke\n        at /usr/include/c++/7/bits/std_function.h:316\n  0: operator()\n        at /workspace/src/runtime/c_runtime_api.cc:525\n  File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback\n  File "/workspace/docs/../python/tvm/autotvm/measure/measure_methods.py", line 786, in verify_pass\n    raise InstantiationError("Skipped because of invalid gpu kernel")\ntvm.autotvm.task.space.InstantiationError: Skipped because of invalid gpu kernel',),), error_no=1, all_cost=0.09257125854492188, timestamp=1627516455.1806688)    [('tile_f', [-1, 2, 1, 16]), ('tile_y', [-1, 1, 7, 1]), ('tile_x', [-1, 7, 1, 1]), ('tile_rc', [-1, 32, 8]), ('tile_ry', [-1, 1, 3]), ('tile_rx', [-1, 3, 1]), ('auto_unroll_max_step', 512), ('unroll_explicit', 0)],None,2824525
No: 17  GFLOPS: 0.00/32.89      result: MeasureResult(costs=(InstantiationError('Traceback (most recent call last):\n  24: TVMFuncCall\n        at /workspace/src/runtime/c_runtime_api.cc:474\n  23: tvm::runtime::PackedFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /workspace/include/tvm/runtime/packed_func.h:1151\n  22: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  21: operator()\n        at /workspace/include/tvm/runtime/packed_func.h:1480\n  20: unpack_call<tvm::IRModule, 5, tvm::<lambda(tvm::te::Schedule, const tvm::runtime::Array<tvm::runtime::ObjectRef>&, const tvm::runtime::String&, const tvm::runtime::Map<tvm::te::Tensor, tvm::tir::Buffer>&, bool)> >\n        at /workspace/include/tvm/runtime/packed_func.h:1421\n  19: run<>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  18: run<tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  17: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  16: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  15: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  14: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1397\n  13: operator()\n        at /workspace/src/driver/driver_api.cc:369\n  12: tvm::LowerSchedule(tvm::te::Schedule, tvm::runtime::Array<tvm::runtime::ObjectRef, void> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unordered_map<tvm::te::Tensor, tvm::tir::Buffer, std::hash<tvm::te::Tensor>, std::equal_to<tvm::te::Tensor>, std::allocator<std::pair<tvm::te::Tensor const, tvm::tir::Buffer> > > const&, bool)\n        at /workspace/src/driver/driver_api.cc:356\n  11: tvm::LowerWithPassList(tvm::IRModule, tvm::runtime::Array<tvm::transform::Pass, void>)\n        at /workspace/src/driver/driver_api.cc:265\n  10: tvm::transform::Pass::operator()(tvm::IRModule) const\n        at /workspace/src/ir/transform.cc:255\n  9: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  8: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:481\n  7: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  6: tvm::tir::transform::PrimFuncPassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/tir/ir/transform.cc:100\n  5: tvm::runtime::TypedPackedFunc<tvm::tir::PrimFunc (tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext)>::operator()(tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext) const\n        at /workspace/include/tvm/runtime/packed_func.h:1498\n  4: tvm::tir::PrimFunc tvm::runtime::detail::typed_packed_call_dispatcher<tvm::tir::PrimFunc>::run<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::runtime::PackedFunc const&, tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&)\n        at /workspace/include/tvm/runtime/packed_func.h:1444\n  3: tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&) const\n        at /workspace/include/tvm/runtime/packed_func.h:1369\n  2: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  1: _M_invoke\n        at /usr/include/c++/7/bits/std_function.h:316\n  0: operator()\n        at /workspace/src/runtime/c_runtime_api.cc:525\n  File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback\n  File "/workspace/docs/../python/tvm/autotvm/measure/measure_methods.py", line 786, in verify_pass\n    raise InstantiationError("Skipped because of invalid gpu kernel")\ntvm.autotvm.task.space.InstantiationError: Skipped because of invalid gpu kernel',),), error_no=1, all_cost=0.08941364288330078, timestamp=1627516456.6708798)    [('tile_f', [-1, 64, 1, 1]), ('tile_y', [-1, 1, 1, 1]), ('tile_x', [-1, 7, 1, 1]), ('tile_rc', [-1, 8, 8]), ('tile_ry', [-1, 1, 3]), ('tile_rx', [-1, 3, 1]), ('auto_unroll_max_step', 1500), ('unroll_explicit', 0)],None,4559286
No: 18  GFLOPS: 0.00/32.89      result: MeasureResult(costs=(InstantiationError('Traceback (most recent call last):\n  24: TVMFuncCall\n        at /workspace/src/runtime/c_runtime_api.cc:474\n  23: tvm::runtime::PackedFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /workspace/include/tvm/runtime/packed_func.h:1151\n  22: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  21: operator()\n        at /workspace/include/tvm/runtime/packed_func.h:1480\n  20: unpack_call<tvm::IRModule, 5, tvm::<lambda(tvm::te::Schedule, const tvm::runtime::Array<tvm::runtime::ObjectRef>&, const tvm::runtime::String&, const tvm::runtime::Map<tvm::te::Tensor, tvm::tir::Buffer>&, bool)> >\n        at /workspace/include/tvm/runtime/packed_func.h:1421\n  19: run<>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  18: run<tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  17: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  16: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  15: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1382\n  14: run<tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_, tvm::runtime::TVMMovableArgValueWithContext_>\n        at /workspace/include/tvm/runtime/packed_func.h:1397\n  13: operator()\n        at /workspace/src/driver/driver_api.cc:369\n  12: tvm::LowerSchedule(tvm::te::Schedule, tvm::runtime::Array<tvm::runtime::ObjectRef, void> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unordered_map<tvm::te::Tensor, tvm::tir::Buffer, std::hash<tvm::te::Tensor>, std::equal_to<tvm::te::Tensor>, std::allocator<std::pair<tvm::te::Tensor const, tvm::tir::Buffer> > > const&, bool)\n        at /workspace/src/driver/driver_api.cc:356\n  11: tvm::LowerWithPassList(tvm::IRModule, tvm::runtime::Array<tvm::transform::Pass, void>)\n        at /workspace/src/driver/driver_api.cc:265\n  10: tvm::transform::Pass::operator()(tvm::IRModule) const\n        at /workspace/src/ir/transform.cc:255\n  9: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  8: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:481\n  7: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/ir/transform.cc:267\n  6: tvm::tir::transform::PrimFuncPassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const\n        at /workspace/src/tir/ir/transform.cc:100\n  5: tvm::runtime::TypedPackedFunc<tvm::tir::PrimFunc (tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext)>::operator()(tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext) const\n        at /workspace/include/tvm/runtime/packed_func.h:1498\n  4: tvm::tir::PrimFunc tvm::runtime::detail::typed_packed_call_dispatcher<tvm::tir::PrimFunc>::run<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::runtime::PackedFunc const&, tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&)\n        at /workspace/include/tvm/runtime/packed_func.h:1444\n  3: tvm::runtime::TVMRetValue tvm::runtime::PackedFunc::operator()<tvm::tir::PrimFunc, tvm::IRModule, tvm::transform::PassContext>(tvm::tir::PrimFunc&&, tvm::IRModule&&, tvm::transform::PassContext&&) const\n        at /workspace/include/tvm/runtime/packed_func.h:1369\n  2: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  1: _M_invoke\n        at /usr/include/c++/7/bits/std_function.h:316\n  0: operator()\n        at /workspace/src/runtime/c_runtime_api.cc:525\n  File "tvm/_ffi/_cython/./packed_func.pxi", line 56, in tvm._ffi._cy3.core.tvm_callback\n  File "/workspace/docs/../python/tvm/autotvm/measure/measure_methods.py", line 786, in verify_pass\n    raise InstantiationError("Skipped because of invalid gpu kernel")\ntvm.autotvm.task.space.InstantiationError: Skipped because of invalid gpu kernel',),), error_no=1, all_cost=0.11762499809265137, timestamp=1627516456.820258)     [('tile_f', [-1, 1, 32, 16]), ('tile_y', [-1, 1, 1, 1]), ('tile_x', [-1, 7, 1, 1]), ('tile_rc', [-1, 1, 512]), ('tile_ry', [-1, 3, 1]), ('tile_rx', [-1, 3, 1]), ('auto_unroll_max_step', 1500), ('unroll_explicit', 1)],None,9677544
No: 19  GFLOPS: 0.00/32.89      result: MeasureResult(costs=(RuntimeError('Traceback (most recent call last):\n  11: TVMFuncCall\n        at /workspace/src/runtime/c_runtime_api.cc:474\n  10: tvm::runtime::PackedFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /workspace/include/tvm/runtime/packed_func.h:1151\n  9: std::function<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)>::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /usr/include/c++/7/bits/std_function.h:706\n  8: _M_invoke\n        at /usr/include/c++/7/bits/std_function.h:316\n  7: operator()\n        at /workspace/src/runtime/rpc/rpc_module.cc:278\n  6: tvm::runtime::RPCWrappedFunc::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const\n        at /workspace/src/runtime/rpc/rpc_module.cc:127\n  5: tvm::runtime::RPCClientSession::CallFunc(void*, TVMValue const*, int const*, int, std::function<void (tvm::runtime::TVMArgs)> const&)\n        at /workspace/src/runtime/rpc/rpc_endpoint.cc:1005\n  4: tvm::runtime::RPCEndpoint::CallFunc(void*, TVMValue ',),), error_no=4, all_cost=4.246142387390137, timestamp=1627516461.4920328)  [('tile_f', [-1, 8, 2, 16]), ('tile_y', [-1, 7, 1, 1]), ('tile_x', [-1, 7, 1, 1]), ('tile_rc', [-1, 1, 1]), ('tile_ry', [-1, 1, 1]), ('tile_rx', [-1, 1, 3]), ('auto_unroll_max_step', 0), ('unroll_explicit', 1)],None,6390073
No: 20  GFLOPS: 136.42/136.42   result: MeasureResult(costs=(0.001697000573770492,), error_no=0, all_cost=1.6595425605773926, timestamp=1627516462.6660373)     [('tile_f', [-1, 1, 4, 1]), ('tile_y', [-1, 1, 1, 1]), ('tile_x', [-1, 7, 1, 1]), ('tile_rc', [-1, 4, 1]), ('tile_ry', [-1, 1, 1]), ('tile_rx', [-1, 1, 3]), ('auto_unroll_max_step', 1500), ('unroll_explicit', 1)],None,9881539

Finally we can inspect the best config from log file, check correctness, and measure running time.

# inspect the best config
dispatch_context = autotvm.apply_history_best("conv2d.log")
best_config = dispatch_context.query(task.target, task.workload)
print("\nBest config:")
print(best_config)

# apply history best from log file
with autotvm.apply_history_best("conv2d.log"):
    with tvm.target.Target("cuda"):
        s, arg_bufs = conv2d_no_batching(N, H, W, CO, CI, KH, KW, strides, padding)
        func = tvm.build(s, arg_bufs)

# check correctness
a_np = np.random.uniform(size=(N, CI, H, W)).astype(np.float32)
w_np = np.random.uniform(size=(CO, CI, KH, KW)).astype(np.float32)
c_np = conv2d_nchw_python(a_np, w_np, strides, padding)

dev = tvm.cuda()
a_tvm = tvm.nd.array(a_np, device=dev)
w_tvm = tvm.nd.array(w_np, device=dev)
c_tvm = tvm.nd.empty(c_np.shape, device=dev)
func(a_tvm, w_tvm, c_tvm)

tvm.testing.assert_allclose(c_np, c_tvm.numpy(), rtol=1e-2)

# Evaluate running time. Here we choose a large repeat number (400) to reduce the noise
# and the overhead of kernel launch. You can also use nvprof to validate the result.
evaluator = func.time_evaluator(func.entry_name, dev, number=400)
print("Time cost of this operator: %f" % evaluator(a_tvm, w_tvm, c_tvm).mean)

Out:

Best config:
[('tile_f', [-1, 1, 4, 1]), ('tile_y', [-1, 1, 1, 1]), ('tile_x', [-1, 7, 1, 1]), ('tile_rc', [-1, 4, 1]), ('tile_ry', [-1, 1, 1]), ('tile_rx', [-1, 1, 3]), ('auto_unroll_max_step', 1500), ('unroll_explicit', 1)],None,9881539
Time cost of this operator: 0.001913

Gallery generated by Sphinx-Gallery