NPU YOLOV5S 目标检测¶

主板	主控	平台	NPU
K3B	RK3562	Rockchip	1T
K11C	RK3566	Rockchip	1T
K1/K1B/K1Mini	RK3568	Rockchip	1T
K7/K7C/K7S-K7F	RK3576	Rockchip	6T
K8/K8D	RK3588	Rockchip	6T
K9	T527	Allwinner	2T
K10B	A733	Allwinner	3T

Allwinner¶

Allwinner 平台通过 AW NPU Model Zoo 提供 YOLOv5s 示例，包含完整的 PC 端模型转换脚本和板端 C 代码 demo。

资源文件¶

linux_aw_npu 下载链接

资源	路径
Docker 镜像	linux_aw_npu/docker/ubuntu-npu_v2.0.10.2.tar.zip
Model Zoo (v0.8.0)	linux_aw_npu/model_zoo/v0.8.0/
YOLOv5s 示例	linux_aw_npu/model_zoo/v0.8.0/examples/yolov5
YOLOv5s ONNX 模型	linux_aw_npu/model_zoo/v0.8.0/examples/yolov5/convert_model/yolov5s_rt.onn
预转换 NBG (A733)	linux_aw_npu/nbg_models/yolov5s_rt_uint8_a733.n
测试图片	linux_aw_npu/model_zoo/v0.8.0/examples/yolov5/model/dog.jpg
gcc-arm-10.2-2020.11	linux_aw_npu/toolchain/

PC 端模型转换¶

Note

测试平台 Window Ubuntu22.04

# 1. 解压 Docker 镜像并加载
cd ~/linux_aw_npu/docker
unzip ubuntu-npu_v2.0.10.2.tar.zip
sudo docker load -i ubuntu-npu_v2.0.10.2.tar

# 2. 启动容器，挂载 model zoo
sudo docker run --ipc=host -itd \
  -v ~/linux_aw_npu/model_zoo/v0.8.0:/workspace \
  --name npu_env ubuntu-npu:v2.0.10.2 /bin/bash

# 3. 进入容器执行转换
sudo docker exec -it npu_env /bin/bash
export ACUITY_PATH=$HOME/acuity-toolkit-whl-6.30.22/bin
export VIV_SDK=$HOME/Vivante_IDE/VivanteIDE5.11.0/cmdtools
cd /workspace/examples/yolov5/convert_model
./convert_model_env.sh
./pegasus_import.sh yolov5s_rt         # 导入 ONNX
./pegasus_quantize.sh yolov5s_rt uint8 12  # uint8 量化
./pegasus_export_ovx_nbg.sh yolov5s_rt uint8 a733  # 导出 A733 NBG
# 产物：/workspace/examples/yolov5/model/yolov5s_rt_uint8_a733.nb

平台参数对照：

平台	`--optimize` 参数
A733 / T736	`VIP9000NANODI_PLUS_PID0X1000003B`
T527 / MR527	`VIP9000NANOSI_PLUS_PID0X10000016`
MR536 / T536	`VIP9000NANODI_PLUS_PID0X1000003B`
V85x / R853	`VIP9000PICO_PID0XEE`

PC 端交叉编译¶

Note

测试平台 Window Ubuntu22.04。

gcc-arm-10.2-2020.11 官方下载链接，在 PC 上使用 gcc-arm-10.2-2020.11（glibc 2.31）交叉编译，产物可直接在 K10B Debian 11 上运行。工具链已预置在 ~/。

# 配置工具链到 model zoo
cd ~/linux_aw_npu/model_zoo/v0.8.0
mkdir -p 0-toolchains/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu
ln -sf ~/linux_aw_npu/toolchain/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/* 0-toolchains/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/

# 编译（-s debian11 指定 debian 兼容工具链）
cd examples/yolov5
rm -rf build_linux_aarch64 install
../build_linux.sh -t a733 -s debian11

# 推送运行
adb push install/yolov5_demo_linux_a733/yolov5_demo_a733 /tmp/
adb push ~/linux_aw_npu/nbg_models/yolov5s_rt_uint8_a733.nb /tmp/
adb push ~/linux_aw_npu/model_zoo/v0.8.0/common/npuruntime/lib_linux_aarch64/A733/*.so /tmp/
adb shell "cd /tmp && LD_LIBRARY_PATH=. ./yolov5_demo_a733 -nb yolov5s_rt_uint8_a733.nb -i model/dog.jpg"

板端推理测试¶

# 推送 NBG 到板端
adb push ~/linux_aw_npu/nbg_models/yolov5s_rt_uint8_a733.nb /tmp/

# 创建测试配置
adb shell "echo '[network]' > /tmp/test.txt && echo '/tmp/yolov5s_rt_uint8_a733.nb' >> /tmp/test.txt && echo '[input]' >> /tmp/test.txt && echo '/aw-test/npu/vpm_run/input_0.dat' >> /tmp/test.txt"

# 运行推理
adb shell "vpm_run -s /tmp/test.txt -l 5 -d 0 -b 1"

预期输出：

input 0 dim 3 640 640 1
ouput 0 dim 80 80 85 3
ouput 1 dim 40 40 85 3
ouput 2 dim 20 20 85 3
create network 0: 7844 us.
run time for this network 0: 20375 us.
profile inference time=20096us
vpm run ret=0

板端原生编译 C Demo¶

K10B Debian 11 自带 gcc 和 libopencv-dev，也可直接在板端编译运行（无需交叉工具链）。

# 推源码到板端
adb shell "mkdir -p /tmp/yolov5_build/model"
adb push ~/linux_aw_npu/model_zoo/v0.8.0/examples/yolov5/*.cpp /tmp/yolov5_build/
adb push ~/linux_aw_npu/model_zoo/v0.8.0/examples/yolov5/*.h /tmp/yolov5_build/
adb push ~/linux_aw_npu/model_zoo/v0.8.0/common/npuruntime/npulib.* /tmp/yolov5_build/
adb push ~/linux_aw_npu/model_zoo/v0.8.0/common/npuruntime/npu_util.* /tmp/yolov5_build/
adb push ~/linux_aw_npu/model_zoo/v0.8.0/common/npuruntime/include/vip_lite*.h /tmp/yolov5_build/
adb push ~/linux_aw_npu/model_zoo/v0.8.0/common/npuruntime/lib_linux_aarch64/A733/*.so /tmp/yolov5_build/
adb push ~/linux_aw_npu/nbg_models/yolov5s_rt_uint8_a733.nb /tmp/yolov5_build/model/
adb push ~/linux_aw_npu/model_zoo/v0.8.0/examples/yolov5/model/dog.jpg /tmp/yolov5_build/model/

# 板端编译
adb shell "cd /tmp/yolov5_build && g++ -std=c++11 -O2 *.cpp -o yolov5_demo -I. -L. -lNBGlinker -lVIPhal \$(pkg-config --cflags --libs opencv4) -lm -ldl"

# 运行推理
adb shell "cd /tmp/yolov5_build && LD_LIBRARY_PATH=. ./yolov5_demo -nb model/yolov5s_rt_uint8_a733.nb -i model/dog.jpg"

预期输出（K10B A733）：

input  0 dim 3 640 640 1
output 0 dim 80 80 85 3
output 1 dim 40 40 85 3
output 2 dim 20 20 85 3
create network 0: 13257 us
run time: 21611 us
detection num: 3
 16: 92%, [134, 226, 307, 545], dog
  7: 69%, [471,  77, 689, 173], truck
  1: 52%, [162, 125, 559, 423], bicycle

aw_yolov5_output

板端视频推理 Demo¶

基于 OpenCV VideoCapture 逐帧读取视频送入 NPU 推理。源码位于 ~/linux_aw_npu/model_zoo/v0.8.0/examples/yolov5_video/。

板端编译运行：

# 推源码到板端
adb shell "mkdir -p /tmp/yolov5_video"
for f in main.cpp CMakeLists.txt model_config.h; do
  adb push ~/linux_aw_npu/model_zoo/v0.8.0/examples/yolov5_video/\$f /tmp/yolov5_video/
done
for f in yolov5_pre.cpp yolov5_post.cpp; do
  adb push ~/linux_aw_npu/model_zoo/v0.8.0/examples/yolov5/\$f /tmp/yolov5_video/
done
adb push ~/linux_aw_npu/model_zoo/v0.8.0/common/npuruntime/npulib.* /tmp/yolov5_video/
adb push ~/linux_aw_npu/model_zoo/v0.8.0/common/npuruntime/npu_util.* /tmp/yolov5_video/
adb push ~/linux_aw_npu/model_zoo/v0.8.0/common/npuruntime/include/vip_lite*.h /tmp/yolov5_video/
adb push ~/linux_aw_npu/model_zoo/v0.8.0/common/npuruntime/lib_linux_aarch64/A733/*.so /tmp/yolov5_video/
adb push ~/linux_aw_npu/nbg_models/yolov5s_rt_uint8_a733.nb /tmp/yolov5_video/

# 编译
adb shell "cd /tmp/yolov5_video && g++ -std=c++11 -O2 main.cpp yolov5_pre.cpp yolov5_post.cpp npulib.cpp npu_util.cpp -o yolov5_video_demo -I. -I/tmp/yolov5_video -L. -lNBGlinker -lVIPhal \$(pkg-config --cflags --libs opencv4) -lm -ldl"

# 推视频到板端（视频素材自己准备）
adb push video.mp4 /tmp/yolov5_video.mp4

# 运行推理（画面显示到桌面）
# 视频推理
adb shell "export DISPLAY=:0 && cd /tmp/yolov5_video && LD_LIBRARY_PATH=. ./yolov5_video_demo -nb yolov5s_rt_uint8_a733.nb -i /tmp/yolov5_video.mp4"

# 摄像头实时推理（-i 0 表示 /dev/video0）
adb shell "export DISPLAY=:0 && cd /tmp/yolov5_video && LD_LIBRARY_PATH=. ./yolov5_video_demo -nb yolov5s_rt_uint8_a733.nb -i 0"

推理窗口会实时显示检测结果（按 q 或 ESC 退出）。

aw_yolov5_video_detect

预期输出（300 帧视频，K10B A733）：

model=yolov5s_rt_uint8_a733.nb, video=/tmp/yolov5_video.mp4
...
frame 1/300: 20866 us
frame 2/300: 20459 us
...
frame 300/300: 21023 us
avg: 20866 us (47.9 FPS)

Rockchip¶

RK 平台支持通过 rknn_model_zoo 和 RKNPU2 SDK 部署 YOLOv5s 目标检测。

官方参考文档：rknn-toolkit2 · rknn_model_zoo

PC 端交叉编译¶

环境配置¶

Tip

export TOOL_CHAIN=SDK目录/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/
export GCC_COMPILER=SDK目录/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu

编译图像 demo¶

根据实际 IC 选择对应的脚本进行编译：

cd external/rknpu2/examples/rknn_yolov5_demo/
./build-linux_RK3562.sh

产物：install/rknn_yolov5_demo_Linux/

ls install/rknn_yolov5_demo_Linux/
lib  model  rknn_yolov5_demo  rknn_yolov5_video_demo

板端推理测试¶

图片检测¶

Usage: ./rknn_yolov5_demo <rknn model> <jpg>

Tip

person @ / bus @ 为对应识别信息

./rknn_yolov5_demo model/RK3588/yolov5s-640-640.rknn model/bus.jpg
post process config: box_conf_threshold = 0.25, nms_threshold = 0.45
Read model/bus.jpg ...
img width = 640, img height = 640
Loading mode...
sdk version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22) driver version: 0.9.2
model input num: 1, output num: 3
  index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, w_stride = 640, size_with_stride=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
  index=0, name=output, n_dims=4, dims=[1, 255, 80, 80], n_elems=1632000, size=1632000, w_stride = 0, size_with_stride=1638400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003860
  index=1, name=283, n_dims=4, dims=[1, 255, 40, 40], n_elems=408000, size=408000, w_stride = 0, size_with_stride=491520, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
  index=2, name=285, n_dims=4, dims=[1, 255, 20, 20], n_elems=102000, size=102000, w_stride = 0, size_with_stride=163840, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003915
model is NHWC input fmt
model input height=640, width=640, channel=3
once run use 31.119000 ms
loadLabelName ./model/coco_80_labels_list.txt
person @ (209 244 286 506) 0.884139
person @ (478 238 559 526) 0.867678
person @ (110 238 230 534) 0.824685
bus @ (94 129 553 468) 0.705055
person @ (79 354 122 516) 0.339254
loop count = 10 , average run  23.615900 ms

视频流检测¶

Tip

需要使用 h264/h265 码流视频。

Usage: ./rknn_yolov5_video_demo <rknn_model> <video_path> <video_type 264/265>

Tip

car @ / bus @ 为对应视频识别信息

./rknn_yolov5_video_demo model/RK3588/yolov5s-640-640.rknn model/yolov5_test.h264 264
Loading mode...
sdk version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22) driver version: 0.9.2
model input num: 1, output num: 3
  index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
  index=0, name=output, n_dims=4, dims=[1, 255, 80, 80], n_elems=1632000, size=1632000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003860
  index=1, name=283, n_dims=4, dims=[1, 255, 40, 40], n_elems=408000, size=408000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
  index=2, name=285, n_dims=4, dims=[1, 255, 20, 20], n_elems=102000, size=102000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003915
model is NHWC input fmt
model input height=640, width=640, channel=3
mpi_dec_test start mpi_dec_test decoder test start mpp_type 7 app_ctx=0x7fc1305278 decoder=0x23ad0100
read video size=720664
...
once run use 26.630000 ms
loadLabelName ./model/coco_80_labels_list.txt
car @ (380 412 438 461) 0.868004
car @ (532 362 574 399) 0.866127
car @ (496 328 530 354) 0.736625
bus @ (748 272 806 329) 0.527151
...

板端编译运行¶

Debian11¶

本地视频流解析¶

Note

目前 RK3568、RK3588 可使用此 demo。Ubuntu 系统库存在适配问题，暂时无法使用。

示例程序测试：

YOLOv5s 目标检测示例程序已内置在 Debian11 文件系统：

cd /rockchip-test/npu2/rknn_yolov5_demo_Linux/
./rknn_yolov5_demo model/RK356X/yolov5s-640-640.rknn model/test.mp4

获取示例程序源码：

ls external/rknpu2/examples/rknn_yolov5_video_demo/
    build  build-android_RK356X.sh  build-android_RK3588.sh  build-linux_RK356X.sh  build-linux_RK3588.sh  
    CMakeLists.txt  convert_rknn_demo  include  install  model  README.md  src

编译源码：

export TOOL_CHAIN=SDK目录/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/
export GCC_COMPILER=SDK目录/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu
cd external/rknpu2/examples/rknn_yolov5_video_demo/
./build-linux_RK356X.sh

产物：install/rknn_yolov5_demo_Linux/

Tip

目前 rknn_yolov5_demo_Linux 的 /lib 需要使用主板 /rockchip-test/npu2/rknn_yolov5_demo_Linux/lib
若无需执行编译操作，可直接从网盘中获取可执行文件

ls install/rknn_yolov5_demo_Linux/
    lib/              model/            rknn_yolov5_demo

操作示例：

cd rknn_yolov5_demo_Linux/
export LD_LIBRARY_PATH=/usr/local/opencv4/lib/
export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/opencv4/lib/
export PKG_CONFIG_LIBDIR=$PKG_CONFIG_LIBDIR:/usr/local/opencv4/lib/pkgconfig
cp /rockchip-test/npu2/rknn_yolov5_demo_Linux/lib . -rf
./rknn_yolov5_demo model/RK356X/yolov5s-640-640.rknn model/test.mp4

程序运行：

Tip

RK356X 平台的运行帧率最高仅能达到 7 帧 / 秒，可自行优化程序性能。

cd rknn_yolov5_demo_Linux/
./rknn_yolov5_demo model/RK356X/yolov5s-640-640.rknn model/test.mp4

f8944680e7bd81aeec4cbddf2eab4b0

解析摄像头视频流¶

Note

目前 RK3568、RK3588 可使用此 demo。Ubuntu 系统库存在适配问题，暂时无法使用。

示例程序测试：

cd /rockchip-test/npu2/rknn_yolov5_demo_Linux/

Tip

运行时间过长会因内存不足，终止进程

./rknn_yolov5_demo model/RK356X/yolov5s-640-640.rknn /dev/video10

获取示例程序源码：

ls external/rknpu2/examples/rknn_yolov5_video_demo/
    build  build-android_RK356X.sh  build-android_RK3588.sh  build-linux_RK356X.sh  build-linux_RK3588.sh  
    CMakeLists.txt  convert_rknn_demo  include  install  model  README.md  src

Debian12 / Ubuntu24.04¶

摄像头及视频流解析¶

Yolov5 获取：

yolov5_video_demo 下载链接

环境安装：

sudo apt install libopencv-dev git cmake make gcc g++ libsndfile1-dev -y

编译：

Note

yolov5_video 在主板端进行编译。
示例源码说明：
main_camera.cc: camera 模式抓图
main_gst.cc: GStreamer 模式抓图（Ubuntu 优先使用）
编译前请任选其一，覆盖替换为 main.cc 后再编译。

rknn_model_zoo/examples/yolov5_video/cpp
├── CMakeLists.txt
├── camera_preview.cpp
├── main.cc 
├── main_camera.cc
├── main_gst.cc
├── postprocess.cc
├── postprocess.h
├── rknpu1
│   └── yolov5.cc
├── rknpu2
│   ├── yolov5.cc
│   └── yolov5_rv1106_1103.cc
└── yolov5.h

在 RK Linux Ubuntu 24.04 平台上，GStreamer MPP 硬编解码与Chromium MPP 硬编解码存在冲突，二者无法同时启用，仅能选择其一使用。

若需正常使用 GStreamer MPP，可执行以下命令修复，修复后 Chromium 将不再使用硬件编解码：

sudo bash /rockchip-test/gstreamer/gstreamer_mpp_fix.sh

编译流程：

cd rknn_model_zoo
./build-linux.sh -t rk356x -a aarch64 -d yolov5_video

测试识别视频流：

rknn_model_zoo/install/rk356x_linux_aarch64/rknn_yolov5_video_demo$ ./rknn_yolov5_demo model/yolov5s_relu_rk3566.rknn model/yolo.mp4

42926c7f2d912d6d530067f00f73c987

测试识别摄像头：

rknn_model_zoo/install/rk356x_linux_aarch64/rknn_yolov5_video_demo$ ./rknn_yolov5_demo model/yolov5s_relu_rk3566.rknn 0

b2fb203b25ccf1f842e4b3120d60ff44