Last active
February 18, 2025 14:51
-
-
Save ZlodeiBaal/847569b24bb338566680ab2a8f22c6b2 to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| sudo apt-get update | |
| sudo apt-get install -y python3 python3-dev python3-pip gcc | |
| sudo apt-get install -y python3-opencv | |
| sudo apt-get install -y python3-numpy | |
| sudo apt-get install git | |
| sudo apt-get install wget | |
| sudo apt-get install python3-setuptools | |
| wget https://github.com/rockchip-linux/rknpu2/raw/master/runtime/RK356X/Linux/librknn_api/aarch64/librknnrt.so | |
| sudo mv librknnrt.so /usr/lib/librknnrt.so | |
| git clone https://github.com/rockchip-linux/rknn-toolkit2.git | |
| cd rknn-toolkit2/rknn_toolkit_lite2/packages/ | |
| pip3 install rknn_toolkit_lite2-1.5.2-cp39-cp39-linux_aarch64.whl |
Sorry, I never saw this.
Also, I tried to use a new lib + create link file for this, and it works:
wget https://github.com/airockchip/rknn-toolkit2/tree/master/rknpu2/runtime/Linux/librknn_api/aarch64/librknnrt.so
sudo mv librknnrt.so /usr/lib
cd /usr/lib
sudo ln -s librknnrt.so librknn_api.so
Hey following your guide, I successfully ran YoloV8 on RK3568 and RK3588. However recevied below error while doing the same for RK3399PRO Boar (even though it is supported by conversion). Can you please help me with it and advice.
:~/rock_chip/rknn_model_zoo/models/CV/object_detection/yolo/RKNN_model_convert$ sh ./convert_yolo.sh
WARNING - 'configs' in yaml file is deprecate, rename it as 'config'
W __init__: rknn-toolkit2 version: 1.5.2+b642f30c
========== parser_config ==========
TOOLKIT_MAIN_VERSION: 1
RK_device_platform: RK3399PRO
model_framework: pytorch
verbose: False
quantize: True
dataset: ../../../../../datasets/COCO/coco_subset_10.txt
graph:
in_0:
shape: 1,3,640,640
mean_values: 0
std_values: 255
img_type: RGB
config:
quantized_dtype: asymmetric_affine-u8
target_platform: RK3399PRO
quantized_algorithm: normal
optimization_level: 3
mean_values: [[0, 0, 0]]
std_values: [[255, 255, 255]]
reorder_channel: 0 1 2
pre_compile: online
core_mask: 1
export_rknn:
export_path: ./model_cvt/RK1808_3399pro/yolov8n_rknnopt_RK1808_3399pro_u8.rknn
model_file_path: yolov8n_rknnopt.torchscript
build:
do_quantization: True
dataset: ./../../../../../datasets/COCO/coco_subset_10.txt
qnnpack: False
load:
model: ./yolov8n_rknnopt.torchscript
input_size_list: [[3, 640, 640]]
inputs:
in_0:
shape: [1, 3, 640, 640]
mean_values: [0, 0, 0]
std_values: [255, 255, 255]
img_type: RGB
outputs:
input_example: [{'in_0': '../../../../../datasets/COCO/val_test/000000000285.jpg'}, {'in_0': '../../ ...
export_pre_compile_path: ./model_cvt/RK1808_3399pro/yolov8n_rknnopt_RK1808_3399pro_u8_precompile.rknn
===================================
---> Create RKNN object
W __init__: rknn-toolkit2 version: 1.5.2+b642f30c
---> Seting RKNN config
E config: Invalid target_platform 'rk3399pro'!
W config: ===================== WARN(2) =====================
E rknn-toolkit2 version: 1.5.2+b642f30c
Traceback (most recent call last):
File "../../../../../common/rknn_converter/rknn_convert.py", line 99, in <module>
convert(config_dict, args)
File "../../../../../common/rknn_converter/rknn_convert.py", line 19, in convert
rknn = cp.convert()
File "/home/fazliddin/rock_chip/rknn_model_zoo/common/rknn_converter/phase.py", line 118, in convert
rknn.config(**model_config_dict['config'])
File "/home/fazliddin/.conda/envs/rockchip_yolo/lib/python3.8/site-packages/rknn/api/rknn.py", line 120, in config
return self.rknn_base.config(args)
File "rknn/api/rknn_base.py", line 941, in rknn.api.rknn_base.RKNNBase.config
File "rknn/api/rknn_log.py", line 112, in rknn.api.rknn_log.RKNNLog.e
ValueError: Invalid target_platform 'rk3399pro'!
Thank You, I tried using the old lib, and it worked for my OPi 5B
Did you use the old lib and 1.6.0 version of the model zoo?
@Askovia did you use the old lib and 1.6.0 version of model zoo?
@ZlodeiBaal do you mean 1.6.0 model zoo version with old lib?
Hi
When I try to run code (or other npu related code) I get:
E RKNN: [22:57:09.806] failed to open rknpu module, need to insmod rknpu dirver!
E RKNN: [22:57:09.806] failed to open rknn device!
E Catch exception when init runtime!
On Orange PI 5, Ubuntu 22.04 downloaded from the OrangePi web site.
@ZlodeiBaal Can you updated the instruction for the new library Rokchip released a while back? It will be of great help.
@ZlodeiBaal, Hello, can you help me with this type of error:
(rknn-toolkit2-env) orangepi@orangepi5plus:~/Desktop/sky_anomaly_detection/rknn-toolkit2/rknn_toolkit_lite2/examples/resnet18$ python test.py
--> Load RKNN model
done
--> Init runtime environment
I RKNN: [22:50:07.209] RKNN Runtime Information: librknnrt version: 1.4.0 (a10f100eb@2022-09-09T09:07:14)
I RKNN: [22:50:07.209] RKNN Driver Information: version: 0.8.2
E RKNN: [22:50:07.209] 6, 1
E RKNN: [22:50:07.209] Invalid RKNN model version 6
E RKNN: [22:50:07.209] rknn_init, load model failed!
E Catch exception when init runtime!
E Traceback (most recent call last):
File "/home/orangepi/Desktop/sky_anomaly_detection/rknn-toolkit2-env/lib/python3.10/site-packages/rknnlite/api/rknn_lite.py", line 148, in init_runtime
self.rknn_runtime.build_graph(self.rknn_data, self.load_model_in_npu)
File "rknnlite/api/rknn_runtime.py", line 919, in rknnlite.api.rknn_runtime.RKNNRuntime.build_graph
Exception: RKNN init failed. error code: RKNN_ERR_FAIL
Init runtime environment failed
I try to run the examples from rknn toolkit lite and the above error comes out
I'm trying to run the following script: https://github.com/rockchip-linux/rknn-toolkit2/blob/master/rknn_toolkit_lite2/examples/dynamic_shape/test.py
and found an error in the if in the main function. Here it is:
if host_name == 'RK3588':
# For RK3588, specify which NPU core the model runs on through the core_mask parameter.
print('2')
ret = rknn_lite.init_runtime(core_mask=RKNNLite.NPU_CORE_0)
print('3')
else:
ret = rknn_lite.init_runtime()
if ret != 0:
print('Init runtime environment failed')
exit(ret)
print('done')
in line: ret = rknn_lite.init_runtime(core_mask=RKNNLite.NPU_CORE_0), and also in line exit(ret).
If I comment out the line with exit(ret), the following error appears:
(rknn-toolkit2-env) orangepi@orangepi5plus:~/Desktop/sky_anomaly_detection/rknn-toolkit2/rknn_toolkit_lite2/examples/dynamic_shape$ python test.py
--> Load RKNN model
done
--> Init runtime environment
1
2
I RKNN: [15:50:16.481] RKNN Runtime Information: librknnrt version: 1.4.0 (a10f100eb@2022-09-09-09T09:07:14)
I RKNN: [15:50:16.481] RKNN Driver Information: version: 0.8.2
E RKNN: [15:50:16.482] 6, 1
E RKNN: [15:50:16.482] Invalid RKNN model version 6
E RKNN: [15:50:16.482] rknn_init, load model failed!
E Catch exception when init runtime!
E Traceback (most recent call last):
File “/home/orangepi/Desktop/sky_anomaly_detection/rknn-toolkit2-env/lib/python3.10/site-packages/rknnlite/api/rknn_lite.py”, line 148, in init_runtime
self.rknn_runtime.build_graph(self.rknn_data, self.load_model_in_npu)
File “rknnlite/api/rknn_runtime.py”, line 919, in rknnlite.api.rknn_runtime.RKNNRuntime.build_graph
Exception: RKNN init failed. error code: RKNN_ERR_FAIL
3
4
Init runtime environment failed
5
6
done
--> Running model
model: mobilenet_v2
input shape: 1,3,224,224
E Catch exception when setting inputs.
E Traceback (most recent call last):
File “/home/orangepi/Desktop/sky_anomaly_detection/rknn-toolkit2-env/lib/python3.10/site-packages/rknnlite/api/rknn_lite.py”, line 200, in inference
self.rknn_runtime.set_inputs(inputs, data_type, data_format, inputs_pass_through=inputs_pass_through)
File “rknnlite/api/rknn_runtime.py”, line 983, in rknnlite.api.rknn_runtime.RKNNRuntime.set_inputs
File “rknnlite/api/rknn_runtime.py”, line 1207, in rknnlite.api.rknn_runtime.RKNNRuntime.RKNNRuntime.get_in_out_num
ctypes.ArgumentError: argument 1: TypeError: wrong type
Traceback (most recent call last):
File “/home/orangepi/Desktop/sky_anomaly_detection/rknn-toolkit2/rknn_toolkit_lite2/examples/dynamic_shape/test.py”, line 116, in <module>
show_top5(outputs)
File “/home/orangepi/Desktop/sky_anomaly_detection/rknn-toolkit2/rknn_toolkit_lite2/examples/dynamic_shape/test.py”, line 40, in show_top5
output = result[0].reshape(-1)
TypeError: 'NoneType' object is not subscriptable
(print() added as a debug)
I think I've found the solution to running the test scripts:
It may be necessary to install and replace the librknnrt.so library with an older one, for this purpose it is suggested to download the following libraries from the github repository:
- librknnrt.so
- librknn_api.so
from here: https://github.com/rockchip-linux/rknpu2/tree/master/runtime/RK3588/Linux/librknn_api/aarch64
The next thing to do is as follows:
# Download the files with the command (everything happens in the “/Desktop/sky_anomaly_detection/rknn-toolkit2/rknn_toolkit_lite2/examples/dynamic_shape” directory):
wget https://raw.github.com/rockchip-linux/rknpu2/blob/master/runtime/RK3588/Linux/librknn_api/aarch64/librknnrt.so
# Rename those files that were on the computer to _old.so:
sudo mv /usr/lib/librknn_api.so /usr/lib/librknnrt_api_old.so
sudo mv /usr/lib/librknnrt.so /usr/lib/librknn_old.so
# Move all files starting with lib from “/Desktop/sky_anomaly_detection/rknn-toolkit2/rknn_toolkit_lite2/examples/dynamic_shape” directory to /usr/lib/ directory:
sudo mv lib* /usr/lib/
Then run the test file from the directory “/Desktop/sky_anomaly_detection/rknn-toolkit2/rknn_toolkit_lite2/examples/dynamic_shape”
python test.py
If the file runs and outputs something like the following:
(rknn-toolkit2-env) orangepi@orangepi5plus:~/Desktop/sky_anomaly_detection/rknn-toolkit2/rknn_toolkit_lite2/examples/dynamic_shape$ python test.py
--> Load RKNN model
done
--> Init runtime environment
1
2
I RKNN: [22:52:44.033] RKNN Runtime Information: librknnrt version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22)
I RKNN: [22:52:44.033] RKNN Driver Information: version: 0.8.2
W RKNN: [22:52:44.033] Current driver version: 0.8.2, recommend to upgrade the driver to the new version: >= 0.8.8
I RKNN: [22:52:44.033] RKNN Model Information: version: 6, toolkit version: 1.6.0+81f21f4d(compiler version: 1.6.0 (585b3edcf@2023-12-11T07:42:56)), target: RKNPU v2, target platform: rk3588, framework name: Caffe, framework layout: NCHW, model inference type: dynamic_shape
W RKNN: [22:52:44.033] RKNN Model version: 1.6.0 not match with rknn runtime version: 1.5.2
3
done
--> Running model
model: mobilenet_v2
input shape: 1,3,224,224
7
8
9
10
W The input[0] need NHWC data format, but NCHW set, the data format and data buffer will be changed to NHWC.
W RKNN: [22:52:44.052] Output(prob): size_with_stride larger than model origin size, if need run OutputOperator in NPU, please call rknn_create_memory using size_with_stride.
11
-----TOP 5-----
[155] score:0.936035 class:"Shih-Tzu"
[204] score:0.002516 class:"Lhasa, Lhasa apso"
[154] score:0.002172 class:"Pekinese, Pekingese, Peke"
[283] score:0.001601 class:"Persian cat"
[284] score:0.000286 class:"Siamese cat, Siamese"
12
input shape: 1,3,160,160
13
14
15
16
W The input[0] need NHWC data format, but NCHW set, the data format and data buffer will be changed to NHWC.
W RKNN: [22:52:44.059] Output(prob): size_with_stride larger than model origin size, if need run OutputOperator in NPU, please call rknn_create_memory using size_with_stride.
17
-----TOP 5-----
[155] score:0.606934 class:"Shih-Tzu"
[154] score:0.329834 class:"Pekinese, Pekingese, Peke"
[204] score:0.025085 class:"Lhasa, Lhasa apso"
[194] score:0.001038 class:"Dandie Dinmont, Dandie Dinmont terrier"
[219] score:0.000241 class:"cocker spaniel, English cocker spaniel, cocker"
18
input shape: 1,3,256,256
19
20
21
22
W The input[0] need NHWC data format, but NCHW set, the data format and data buffer will be changed to NHWC.
W RKNN: [22:52:44.064] Output(prob): size_with_stride larger than model origin size, if need run OutputOperator in NPU, please call rknn_create_memory using size_with_stride.
22
-----TOP 5-----
[155] score:0.927246 class:"Shih-Tzu"
[154] score:0.007225 class:"Pekinese, Pekingese, Peke"
[204] score:0.004616 class:"Lhasa, Lhasa apso"
[193] score:0.000878 class:"Australian terrier"
[283] score:0.000482 class:"Persian cat"
done
23
24
Congratulations, you can rejoice, everything is set up correctly!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
@ZlodeiBaal Thanks to your response based on which i solved the issue using old library, however faced below issue:
Can you advice on this?