python版openvino使用

「爱情、让人受尽委屈。」 2023-01-08 08:23 218阅读 0赞

openvino作为一个优秀的cpu推理引擎

这里使用一下python版api

使用之前需要编译安装openvino

[https://docs.openvinotoolkit.org/latest/openvino\_docs\_get\_started\_get\_started\_windows.html][https_docs.openvinotoolkit.org_latest_openvino_docs_get_started_get_started_windows.html]

安装好以后，直接pip install openvino安装python版的openvino

实际上，python版的api也是调用的c++编译好的openvino，这就是为啥使用python版，也需要编译安装openvino

接下来可以使用了，使用之前需要转一下openvino模型文件，当然也可以不转，openvino是支持读取onnx的

python /opt/intel/openvino_2021/deployment_tools/model_optimizer/mo.py --input_model ctdet_coco_dlav0_512.onnx --output_dir ./ctdet_coco_dlav0_512  --data_type FP16

onnx模型来自这里[https://download.01.org/opencv/public\_models/122019/ctdet\_coco\_dlav0/][https_download.01.org_opencv_public_models_122019_ctdet_coco_dlav0]

现在可以进行推理了 ，先直接使用onnx文件推理

from openvino.inference_engine import IECore
    import numpy as np
    import cv2
    import time
    
    ie = IECore()
    model="ctdet_coco_dlav0_512.onnx"
    #model="ctdet_coco_dlav0_512/ctdet_coco_dlav0_512.xml"
    net = ie.read_network(model=model)
    input_blob = next(iter(net.input_info))
    out_blob = next(iter(net.outputs))
    net.batch_size=16#batchsize
    
    n, c, h, w = net.input_info[input_blob].input_data.shape
    print(n, c, h, w)
    images = np.ndarray(shape=(n, c, h, w))
    for i in range(n):
            image = cv2.imread("123.jpg")
            if image.shape[:-1] != (h, w):
                image = cv2.resize(image, (w, h))
            image = image.transpose((2, 0, 1))
            images[i] = image
    exec_net = ie.load_network(network=net, device_name="CPU")
    start=time.time()
    res = exec_net.infer(inputs={input_blob: images})
    #print(res)
    print('infer total time is %.4f s'%(time.time()-start))

运行结果：

![20210119164826211.png][]

我们上面的代码设置的batchsize为16，现在试一下batchsize为1的结果，修改net.batch\_size=1即可

![20210119164922901.png][]

接下来使用onnx转的文件进行推理：

from openvino.inference_engine import IECore
    import numpy as np
    import cv2
    import time
    
    ie = IECore()
    #model="ctdet_coco_dlav0_512.onnx"
    model="ctdet_coco_dlav0_512/ctdet_coco_dlav0_512.xml"
    net = ie.read_network(model=model)
    input_blob = next(iter(net.input_info))
    out_blob = next(iter(net.outputs))
    net.batch_size=16#batchsize
    
    n, c, h, w = net.input_info[input_blob].input_data.shape
    print(n, c, h, w)
    images = np.ndarray(shape=(n, c, h, w))
    for i in range(n):
            image = cv2.imread("123.jpg")
            if image.shape[:-1] != (h, w):
                image = cv2.resize(image, (w, h))
            image = image.transpose((2, 0, 1))
            images[i] = image
    exec_net = ie.load_network(network=net, device_name="CPU")
    start=time.time()
    res = exec_net.infer(inputs={input_blob: images})
    #print(res)
    print('infer total time is %.4f s'%(time.time()-start))

运行结果

![20210119165031938.png][]

同理， net.batch\_size=1的运行结果：

![20210119165157280.png][]

可以发现，使用onnx的推理时间和转换过的推理时间基本一致

[https_docs.openvinotoolkit.org_latest_openvino_docs_get_started_get_started_windows.html]: https://docs.openvinotoolkit.org/latest/openvino_docs_get_started_get_started_windows.html
[https_download.01.org_opencv_public_models_122019_ctdet_coco_dlav0]: https://download.01.org/opencv/public_models/122019/ctdet_coco_dlav0/
[20210119164826211.png]: /images/20221119/6f2ac91e25144fbe8bbbc27eb2ae6935.png
[20210119164922901.png]: /images/20221119/9b27846e0f38486aa10125e805b3cf00.png
[20210119165031938.png]: /images/20221119/6bb30ac255a6454cb8d82387895bf673.png
[20210119165157280.png]: /images/20221119/7d5683a5dae1417cb570ba593b405b72.png