為x86 CPU自動排程神經網路

阿新 • • 發佈：2020-12-25

為x86 CPU自動排程神經網路
對特定裝置和工作負載進行自動除錯對於獲得最佳效能至關重要。這是有關如何使用自動排程器為x86 CPU除錯整個神經網路的文件。
為了自動除錯神經網路，將網路劃分為小的子圖，並對其進行獨立除錯。每個子圖被視為一個搜尋任務。任務排程程式可以對時間進行分片，併為這些任務動態分配時間資源。任務排程程式可以預測每個任務對端到端執行時間的影響，並優先排程可以最大程度地減少執行時間的任務。
對於每個子圖，使用compute宣告tvm/python/topi獲取張量表達式形式的計算DAG。然後，使用自動排程器來構造此DAG的搜尋空間，並搜尋良好的排程（低階優化）。

與依靠手動模板定義搜尋空間的基於模板的autotvm不同，自動排程程式不需要任何排程模板。換句話說，自動排程程式僅在tvm/python/topi中使用計算宣告，而不使用現有的排程模板。
注意，本文無法在Windows或最新版本的macOS上執行。要使其執行，需要將本文的內容包裝在一個塊中。if name == “main”:
import numpy as np

import tvm
from tvm import relay, auto_scheduler
import tvm.relay.testing
from tvm.contrib import graph_runtime
定義網路
首先，需要使用中繼前端API定義網路。可以載入一些預定義的網路tvm.relay.testing。還可以從MXNet，ONNX，PyTorch和TensorFlow載入模型。

對於卷積神經網路，儘管自動排程程式可以在任何佈局下正常工作，但使用NHWC佈局通常可以實現最佳效能。還使用自動排程程式對NHWC佈局實施了更多優化。因此，建議將模型轉換為NHWC佈局以使用自動排程程式。可以在TVM中使用ConvertLayout pass進行佈局轉換。
def get_network(name, batch_size, layout=“NHWC”, dtype=“float32”):
“”“Get the symbol definition and random weight of a network”""

# auto-scheduler prefers NHWC layout
if layout == "NHWC":
    image_shape = (224, 224, 3)
elif layout == "NCHW":
    image_shape = (3, 224, 224)
else:
    raise ValueError("Invalid layout: " + layout)

input_shape = (batch_size,) + image_shape
output_shape = (batch_size, 1000)

if name.startswith("resnet-"):
    n_layer = int(name.split("-")[1])
    mod, params = relay.testing.resnet.get_workload(
        num_layers=n_layer,
        batch_size=batch_size,
        layout=layout,
        dtype=dtype,
        image_shape=image_shape,
    )
elif name.startswith("resnet3d-"):
    n_layer = int(name.split("-")[1])
    mod, params = relay.testing.resnet.get_workload(
        num_layers=n_layer,
        batch_size=batch_size,
        layout=layout,
        dtype=dtype,
        image_shape=image_shape,
    )
elif name == "mobilenet":
    mod, params = relay.testing.mobilenet.get_workload(
        batch_size=batch_size, layout=layout, dtype=dtype, image_shape=image_shape
    )
elif name == "squeezenet_v1.1":
    assert layout == "NCHW", "squeezenet_v1.1 only supports NCHW layout"
    mod, params = relay.testing.squeezenet.get_workload(
        version="1.1",
        batch_size=batch_size,
        dtype=dtype,
        image_shape=image_shape,
    )
elif name == "inception_v3":
    input_shape = (batch_size, 3, 299, 299) if layout == "NCHW" else (batch_size, 299, 299, 3)
    mod, params = relay.testing.inception_v3.get_workload(batch_size=batch_size, dtype=dtype)
elif name == "mxnet":
    # an example for mxnet model
    from mxnet.gluon.model_zoo.vision import get_model

    assert layout == "NCHW"

    block = get_model("resnet50_v1", pretrained=True)
    mod, params = relay.frontend.from_mxnet(block, shape={"data": input_shape}, dtype=dtype)
    net = mod["main"]
    net = relay.Function(
        net.params, relay.nn.softmax(net.body), None, net.type_params, net.attrs
    )
    mod = tvm.IRModule.from_expr(net)

return mod, params, input_shape, output_shape

# Define the neural network and compilation target.
# If the target machine supports avx512 instructions, replace the
# “llvm -mcpu=core-avx2” with "llvm -mcpu=skylake-avx512"
network = “resnet-50”
batch_size = 1
layout = “NHWC”
target = tvm.target.Target(“llvm -mcpu=core-avx2”)
dtype = “float32”
log_file = “%s-%s-B%d-%s.json” % (network, layout, batch_size, target.kind.name)
提取搜尋任務
接下來，從網路中提取搜尋任務及其權重。任務的權重是整個網路中任務子圖的出現次數。通過使用權重，可以將網路的端到端延遲近似為sum(latency[t] * weight[t])，其中latency[t]是任務的延遲，weight[t]是任務的權重。任務排程程式只會優化此目標。
# Extract tasks from the network
print(“Extract tasks…”)
mod, params, input_shape, output_shape = get_network(network, batch_size, layout, dtype=dtype)
tasks, task_weights = auto_scheduler.extract_tasks(mod[“main”], params, target)

for idx, task in enumerate(tasks):
print("========== Task %d (workload key: %s) ==========" % (idx, task.workload_key))
print(task.compute_dag)
出：
Extract tasks…
========== Task 0 (workload key: [“b32ed43fb351136894c322ee49097a1a”]) ==========
placeholder = PLACEHOLDER [1, 1000]
T_softmax_maxelem(i0) max= placeholder[i0, k]
T_softmax_exp(i0, i1) = tir.exp((placeholder[i0, i1] - T_softmax_maxelem[i0]))
T_softmax_expsum(i0) += T_softmax_exp[i0, k]
T_softmax_norm(i0, i1) = (T_softmax_exp[i0, i1]/T_softmax_expsum[i0])

========== Task 1 (workload key: [“6129df1a3d5f6326c8393a8d17160199”]) ==========
placeholder = PLACEHOLDER [1, 2048]
placeholder = PLACEHOLDER [1000, 2048]
compute(z, y, x) += (placeholder[z, ((k*16) + x)]placeholder[y, ((k16) + x)])
compute(y, x) += compute[y, x, kk]
placeholder = PLACEHOLDER [1000]
T_add(ax0, ax1) = (compute[ax0, ax1] + placeholder[ax1])

========== Task 2 (workload key: [“36ee2798ed60bae3bcd1bb89a0285fe8”]) ==========
placeholder = PLACEHOLDER [1, 7, 7, 2048]
tensor(ax0, ax1, ax2, ax3) += placeholder[ax0, ((ax17) + rv0), ((ax27) + rv1), ax3]
tensor(ax0, ax1, ax2, ax3) = (tensor[ax0, ax1, ax2, ax3]/(float32((select((bool)1, ((ax1 + 1)*7), (((ax1 + 1)7) + 1)) - (ax17)))*float32((select((bool)1, ((ax2 + 1)*7), (((ax2 + 1)7) + 1)) - (ax27)))))

========== Task 3 (workload key: [“dcf6fcf5f56fa614bf9aef0c82382caf”]) ==========
placeholder = PLACEHOLDER [1, 7, 7, 512]
PaddedInput(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
placeholder = PLACEHOLDER [1, 1, 512, 2048]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, (yy + ry), (xx + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 7, 7, 2048]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, ax1, ax2, ax3])
placeholder = PLACEHOLDER [1, 1, 1, 2048]
T_multiply(ax0, ax1, ax2, ax3) = (T_add[ax0, ax1, ax2, ax3]*placeholder[ax0, 0, 0, ax3])
placeholder = PLACEHOLDER [1, 1, 1, 2048]
T_add(ax0, ax1, ax2, ax3) = (T_multiply[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)

========== Task 4 (workload key: [“7e3f0cf5a6dd80d36dab1a3dad92674a”]) ==========
placeholder = PLACEHOLDER [1, 7, 7, 512]
PaddedInput(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 1) && (i1 < 8)) && (i2 >= 1)) && (i2 < 8)), placeholder[i0, (i1 - 1), (i2 - 1), i3], 0f)
placeholder = PLACEHOLDER [3, 3, 512, 512]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, (yy + ry), (xx + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 1, 1, 512]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)

========== Task 5 (workload key: [“e0a9eb3795b531085e0ebb772e7e800c”]) ==========
placeholder = PLACEHOLDER [1, 7, 7, 2048]
PaddedInput(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
placeholder = PLACEHOLDER [1, 1, 2048, 512]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, (yy + ry), (xx + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 1, 1, 512]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)

========== Task 6 (workload key: [“03614e726dc588d11887eb0953a77e53”]) ==========
placeholder = PLACEHOLDER [1, 7, 7, 512]
PaddedInput(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
placeholder = PLACEHOLDER [1, 1, 512, 2048]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, (yy + ry), (xx + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 7, 7, 2048]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, ax1, ax2, ax3])

========== Task 7 (workload key: [“7657f886f5e9d8b5f19a5fd2c5b90d8d”]) ==========
placeholder = PLACEHOLDER [1, 14, 14, 1024]
PaddedInput(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
placeholder = PLACEHOLDER [1, 1, 1024, 512]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, ((yy2) + ry), ((xx2) + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 1, 1, 512]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)

========== Task 8 (workload key: [“7e09b626cf077cd419190fee02091dd6”]) ==========
placeholder = PLACEHOLDER [1, 14, 14, 256]
PaddedInput(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
placeholder = PLACEHOLDER [1, 1, 256, 1024]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, (yy + ry), (xx + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 14, 14, 1024]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, ax1, ax2, ax3])
placeholder = PLACEHOLDER [1, 1, 1, 1024]
T_add(ax0, ax1, ax2, ax3) = (T_add[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)

========== Task 9 (workload key: [“95bf49cc8cf7a351e974b2359702aac0”]) ==========
placeholder = PLACEHOLDER [1, 14, 14, 256]
PaddedInput(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 1) && (i1 < 15)) && (i2 >= 1)) && (i2 < 15)), placeholder[i0, (i1 - 1), (i2 - 1), i3], 0f)
placeholder = PLACEHOLDER [3, 3, 256, 256]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, (yy + ry), (xx + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 1, 1, 256]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)

========== Task 10 (workload key: [“e043f834cc7f19597227e09dc7f59503”]) ==========
placeholder = PLACEHOLDER [1, 14, 14, 1024]
PaddedInput(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
placeholder = PLACEHOLDER [1, 1, 1024, 256]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, (yy + ry), (xx + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 1, 1, 256]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)

========== Task 11 (workload key: [“cd7c4a374fb2bbc0d075c8cae638ad14”]) ==========
placeholder = PLACEHOLDER [1, 14, 14, 256]
PaddedInput(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
placeholder = PLACEHOLDER [1, 1, 256, 1024]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, (yy + ry), (xx + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 14, 14, 1024]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, ax1, ax2, ax3])

========== Task 12 (workload key: [“1dce2c5e4269b8a12dfc50cd4dd23ff1”]) ==========
placeholder = PLACEHOLDER [1, 28, 28, 512]
PaddedInput(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
placeholder = PLACEHOLDER [1, 1, 512, 256]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, ((yy2) + ry), ((xx2) + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 1, 1, 256]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)

========== Task 13 (workload key: [“d3b36ce001dc24d693facfbdae1979b4”]) ==========
placeholder = PLACEHOLDER [1, 28, 28, 128]
PaddedInput(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
placeholder = PLACEHOLDER [1, 1, 128, 512]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, (yy + ry), (xx + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 28, 28, 512]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, ax1, ax2, ax3])
placeholder = PLACEHOLDER [1, 1, 1, 512]
T_add(ax0, ax1, ax2, ax3) = (T_add[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)

========== Task 14 (workload key: [“0fb1dfcdb5b755e2dab290ed0129dcf2”]) ==========
placeholder = PLACEHOLDER [1, 28, 28, 128]
PaddedInput(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 1) && (i1 < 29)) && (i2 >= 1)) && (i2 < 29)), placeholder[i0, (i1 - 1), (i2 - 1), i3], 0f)
placeholder = PLACEHOLDER [3, 3, 128, 128]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, (yy + ry), (xx + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 1, 1, 128]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)

========== Task 15 (workload key: [“45acfc473c772458684f36a34549d8aa”]) ==========
placeholder = PLACEHOLDER [1, 28, 28, 512]
PaddedInput(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
placeholder = PLACEHOLDER [1, 1, 512, 128]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, (yy + ry), (xx + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 1, 1, 128]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)

========== Task 16 (workload key: [“5e3ceb6e23ae8c351d5a1770d5fc6c7c”]) ==========
placeholder = PLACEHOLDER [1, 28, 28, 128]
PaddedInput(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
placeholder = PLACEHOLDER [1, 1, 128, 512]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, (yy + ry), (xx + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 28, 28, 512]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, ax1, ax2, ax3])

========== Task 17 (workload key: [“a085717fb3dcb046e5c4c2c04d3dc541”]) ==========
placeholder = PLACEHOLDER [1, 56, 56, 256]
PaddedInput(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
placeholder = PLACEHOLDER [1, 1, 256, 128]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, ((yy2) + ry), ((xx2) + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 1, 1, 128]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)

========== Task 18 (workload key: [“691feef049c8693bbe91bd5e7c9cdf34”]) ==========
placeholder = PLACEHOLDER [1, 56, 56, 64]
PaddedInput(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
placeholder = PLACEHOLDER [1, 1, 64, 256]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, (yy + ry), (xx + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 56, 56, 256]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, ax1, ax2, ax3])
placeholder = PLACEHOLDER [1, 1, 1, 256]
T_add(ax0, ax1, ax2, ax3) = (T_add[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)

========== Task 19 (workload key: [“a9e632e5167afb60fbe29e7aeef1d152”]) ==========
placeholder = PLACEHOLDER [1, 56, 56, 64]
PaddedInput(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 1) && (i1 < 57)) && (i2 >= 1)) && (i2 < 57)), placeholder[i0, (i1 - 1), (i2 - 1), i3], 0f)
placeholder = PLACEHOLDER [3, 3, 64, 64]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, (yy + ry), (xx + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 1, 1, 64]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)

========== Task 20 (workload key: [“b51e06c1131d4cded40d1b215f722a4e”]) ==========
placeholder = PLACEHOLDER [1, 56, 56, 256]
PaddedInput(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
placeholder = PLACEHOLDER [1, 1, 256, 64]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, (yy + ry), (xx + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 1, 1, 64]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)

========== Task 21 (workload key: [“8fcee68a4342c38248a827f1c6c69177”]) ==========
placeholder = PLACEHOLDER [1, 56, 56, 64]
PaddedInput(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
placeholder = PLACEHOLDER [1, 1, 64, 256]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, (yy + ry), (xx + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 56, 56, 256]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, ax1, ax2, ax3])

========== Task 22 (workload key: [“8dd7d81db440763f622f03fdc99e6d46”]) ==========
placeholder = PLACEHOLDER [1, 56, 56, 64]
PaddedInput(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
placeholder = PLACEHOLDER [1, 1, 64, 64]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, (yy + ry), (xx + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 1, 1, 64]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)

========== Task 23 (workload key: [“ba2026d923536b75e9b4faed89287d5f”]) ==========
placeholder = PLACEHOLDER [1, 112, 112, 64]
pad_temp(ax0, ax1, ax2, ax3) = tir.if_then_else(((((ax1 >= 1) && (ax1 < 113)) && (ax2 >= 1)) && (ax2 < 113)), placeholder[ax0, (ax1 - 1), (ax2 - 1), ax3], -3.40282e+38f)
tensor(ax0, ax1, ax2, ax3) max= pad_temp[ax0, ((ax12) + dh), ((ax22) + dw), ax3]
placeholder = PLACEHOLDER [1, 1, 1, 64]
T_add(ax0, ax1, ax2, ax3) = (tensor[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)

========== Task 24 (workload key: [“a0eb8d6048282a4a0986cc2ccf14eaa2”]) ==========
placeholder = PLACEHOLDER [1, 224, 224, 3]
PaddedInput(i0, i1, i2, i3) = tir.if_then_else(((((i1 >= 3) && (i1 < 227)) && (i2 >= 3)) && (i2 < 227)), placeholder[i0, (i1 - 3), (i2 - 3), i3], 0f)
placeholder = PLACEHOLDER [7, 7, 3, 64]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, ((yy2) + ry), ((xx2) + rx), rc]*placeholder[ry, rx, rc, ff])
placeholder = PLACEHOLDER [1, 1, 1, 64]
T_add(ax0, ax1, ax2, ax3) = (Conv2dOutput[ax0, ax1, ax2, ax3] + placeholder[ax0, 0, 0, ax3])
T_relu(ax0, ax1, ax2, ax3) = max(T_add[ax0, ax1, ax2, ax3], 0f)

========== Task 25 (workload key: [“45b4de07687dee43ee1cbde9f516b2bf”]) ==========
placeholder = PLACEHOLDER [1, 56, 56, 64]
PaddedInput(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
placeholder = PLACEHOLDER [1, 1, 64, 256]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, (yy + ry), (xx + rx), rc]*placeholder[ry, rx, rc, ff])

========== Task 26 (workload key: [“b2010aa63c95dedf1f58f3fe8bc78634”]) ==========
placeholder = PLACEHOLDER [1, 56, 56, 256]
PaddedInput(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
placeholder = PLACEHOLDER [1, 1, 256, 512]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, ((yy2) + ry), ((xx2) + rx), rc]*placeholder[ry, rx, rc, ff])

========== Task 27 (workload key: [“4d7e646d99bfa3cea8245bd7100369cb”]) ==========
placeholder = PLACEHOLDER [1, 28, 28, 512]
PaddedInput(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
placeholder = PLACEHOLDER [1, 1, 512, 1024]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, ((yy2) + ry), ((xx2) + rx), rc]*placeholder[ry, rx, rc, ff])

========== Task 28 (workload key: [“537c8642716948c33a6eaaabc86b159d”]) ==========
placeholder = PLACEHOLDER [1, 14, 14, 1024]
PaddedInput(i0, i1, i2, i3) = placeholder[i0, i1, i2, i3]
placeholder = PLACEHOLDER [1, 1, 1024, 2048]
Conv2dOutput(nn, yy, xx, ff) += (PaddedInput[nn, ((yy2) + ry), ((xx2) + rx), rc]*placeholder[ry, rx, rc, ff])
開始Tuning除錯
現在，設定一些選項來優化和啟動搜尋任務
• num_measure_trials是在除錯期間可以使用的測量試驗次數。可以將其設定為較小的數字（例如200）以進行快速演示。實際上，建議將其設定為800 * len(tasks)，通常足以使搜尋收斂。例如，resnet-50中有29個任務，可以將其設定為20000。可以根據時間預算除錯此引數。
• 此外，還用RecordToFile將測量記錄轉儲到日誌檔案中，這些測量記錄可用於最好地查詢歷史記錄，恢復搜尋以及以後進行更多分析。
• 有關更多引數，請參見auto_scheduler.TuningOptions, auto_scheduler.LocalRunner。
def run_tuning():
print(“Begin tuning…”)
tuner = auto_scheduler.TaskScheduler(tasks, task_weights)
tune_option = auto_scheduler.TuningOptions(
num_measure_trials=200, # change this to 20000 to achieve the best performance
runner=auto_scheduler.LocalRunner(repeat=10, enable_cpu_cache_flush=True),
measure_callbacks=[auto_scheduler.RecordToFile(log_file)],
)

tuner.tune(tune_option)

# We do not run the tuning in our webpage server since it takes too long.
# Uncomment the following line to run it by yourself.
# run_tuning()
注意
tuning除錯期間說明列印的資訊
在tuning除錯期間，控制檯上會列印很多資訊。它們用於除錯目的。最重要的資訊是任務排程程式的輸出。下表是示例輸出。

------------------------------ [ Task Scheduler ]

| ID | Latency (ms) | Speed (GFLOPS) | Trials |

| 0 | 0.010 | 0.40 | 64 |
| 1 | 0.087 | 47.19 | 64 |
| 2 | 0.008 | -0.00 | 64 |
| 3 | 0.177 | 582.07 | 64 |
| 4 | 0.268 | 862.37 | 256 |
| 5 | 0.166 | 621.13 | 128 |
| 6 | 0.170 | 605.10 | 128 |
| 7 | 0.128 | 403.20 | 64 |
| 8 | 0.189 | 545.71 | 64 |
| 9 | 0.231 | 1001.01 | 448 |
| 10 | 0.155 | 664.80 | 256 |
| 11 | 0.155 | 662.86 | 256 |
| 12 | 0.119 | 434.08 | 64 |
| 13 | 0.199 | 522.13 | 64 |
| 14 | 0.235 | 986.56 | 320 |
| 15 | 0.149 | 689.13 | 128 |
| 16 | 0.155 | 664.80 | 192 |
| 17 | 0.151 | 340.64 | 64 |
| 18 | 0.176 | 597.55 | 128 |
| 19 | 0.220 | 1054.37 | 192 |
| 20 | 0.150 | 686.01 | 128 |
| 21 | 0.159 | 650.88 | 128 |
| 22 | 0.073 | 358.19 | 64 |
| 23 | 0.031 | 70.63 | 64 |
| 24 | 0.251 | 947.73 | 128 |
| 25 | 0.157 | 652.47 | 128 |
| 26 | 0.215 | 954.84 | 128 |
| 27 | 0.237 | 868.92 | 128 |
| 28 | 0.266 | 774.06 | 128 |

Estimated total latency: 10.016 ms Trials: 3992 Used time : 1131 s Next ID: 15
下表列出了所有任務的延遲和（估計）速度。它還列出了所有任務的測量試驗分配。最後一行顯示這些任務的總加權延遲，這可以粗略估計網路的端到端執行時間。最後一行還顯示測量試驗的總數，自動除錯所花費的總時間以及要除錯的下一個任務的ID。
也將出現一些“ dmlc :: Error”錯誤，因為自動排程程式將嘗試某些無效的排程。如果可以繼續進行除錯，則可以放心地忽略它們，因為這些錯誤與主要過程是隔離的。
注意
提前終止除錯
可以通過強制終止此過程來提前終止除錯。只要為日誌檔案中的每個任務獲得至少一個有效的排程，就應該能夠進行編譯（下面的部分）。
編譯和評估
自動除錯後，可以使用發現的最佳時間表來編譯網路。在自動除錯過程中，所有測量記錄都將轉儲到日誌檔案中，因此可以讀取日誌檔案並載入最佳排程。

Compile with the history best

print(“Compile…”)
with auto_scheduler.ApplyHistoryBest(log_file):
with tvm.transform.PassContext(opt_level=3, config={“relay.backend.use_auto_scheduler”: True}):
lib = relay.build(mod, target=target, params=params)

Create graph runtime

ctx = tvm.context(str(target), 0)
module = graph_runtime.GraphModule(lib"default")
data_tvm = tvm.nd.array((np.random.uniform(size=input_shape)).astype(dtype))
module.set_input(“data”, data_tvm)

Evaluate

print(“Evaluate inference time cost…”)
ftimer = module.module.time_evaluator(“run”, ctx, repeat=3, min_repeat_ms=500)
prof_res = np.array(ftimer().results) * 1e3 # convert to millisecond
print(“Mean inference time (std dev): %.2f ms (%.2f ms)” % (np.mean(prof_res), np.std(prof_res)))
出：
Compile…
Evaluate inference time cost…
Mean inference time (std dev): 30.72 ms (0.09 ms)
其他技巧
• 在除錯期間，自動排程器需要編譯許多程式並從中提取功能。此部分佔用大量CPU，因此建議使用具有多個核心的高效能CPU以加快搜索速度。
• 可以用python3 -m tvm.auto_scheduler.measure_record --mode distill --i log.json來提取大型日誌檔案，而僅儲存最有用的記錄。
• 可以從上一個日誌檔案繼續搜尋。load_log_file在function中建立任務排程程式時，只需新增一個新引數run_tuning。也就是， tuner = auto_scheduler.TaskScheduler(tasks, task_weights, load_log_file=log_file)
• 如果有多個目標CPU，則可以將它們全部用於測量以並行化測量。檢查本節以瞭解如何使用RPC跟蹤器和RPC伺服器。要在自動排程使用RPC跟蹤，在TuningOptions中用auto_scheduler.RPCRunner更換runner 。

為x86 CPU自動排程神經網路

------------------------------ [ Task Scheduler ]

| ID | Latency (ms) | Speed (GFLOPS) | Trials |

Compile with the history best

Create graph runtime

Evaluate

為x86 CPU自動排程神經網路

Keras構建神經網路踩坑(解決model.predict預測值全為0.0的問題)

為AI注入一縷意識：構建更智慧、更強大的神經網路

使用keras為什麼accuracy一直為0_Keras乾貨|神經網路模型超引數調優

特斯拉 CEO 馬斯克：自動駕駛可通過視覺神經網路實現，安全性超人工駕駛十倍以上

Adobe MAX 2021 釋出 Photoshop 重磅更新，支援懸停自動蒙版物件選擇工具、全新神經網路濾鏡、更自然真實漸變...

自動機器學習：利用遺傳演算法優化遞迴神經網路

keras學習筆記-黑白照片自動著色的神經網路-Alpha版

keras學習筆記-黑白照片自動著色的神經網路-Beta版

Pytorch 神經網路—自定義資料集上實現教程

numpy實現神經網路反向傳播演算法的步驟

自適應線性神經網路Adaline的python實現詳解

基於python的BP神經網路及異或實現過程解析

Pytorch實現神經網路的分類方式

Pytorch 搭建分類迴歸神經網路並用GPU進行加速的例子

pytorch下使用LSTM神經網路寫詩例項

Python利用全連線神經網路求解MNIST問題詳解

關於pytorch中全連線神經網路搭建兩種模式詳解

pytorch-神經網路擬合曲線例項

Python使用迴圈神經網路解決文字分類問題的方法詳解

為x86 CPU自動排程神經網路

------------------------------ [ Task Scheduler ]

| ID | Latency (ms) | Speed (GFLOPS) | Trials |

Compile with the history best

Create graph runtime

Evaluate

相關推薦