gistfile1.txt

sudo apt-get update
sudo apt-get install -y python3 python3-dev python3-pip gcc
sudo apt-get install -y python3-opencv
sudo apt-get install -y python3-numpy
sudo apt-get install git
sudo apt-get install wget
sudo apt-get install python3-setuptools
wget https://github.com/rockchip-linux/rknpu2/raw/master/runtime/RK356X/Linux/librknn_api/aarch64/librknnrt.so
sudo mv librknnrt.so /usr/lib/librknnrt.so
git clone https://github.com/rockchip-linux/rknn-toolkit2.git
cd rknn-toolkit2/rknn_toolkit_lite2/packages/
pip3 install rknn_toolkit_lite2-1.5.2-cp39-cp39-linux_aarch64.whl

Hi
When I try to run code (or other npu related code) I get:
E RKNN: [22:57:09.806] failed to open rknpu module, need to insmod rknpu dirver!
E RKNN: [22:57:09.806] failed to open rknn device!
E Catch exception when init runtime!

On Orange PI 5, Ubuntu 22.04 downloaded from the OrangePi web site.

@ZlodeiBaal Can you updated the instruction for the new library Rokchip released a while back? It will be of great help.

@ZlodeiBaal, Hello, can you help me with this type of error:

(rknn-toolkit2-env) orangepi@orangepi5plus:~/Desktop/sky_anomaly_detection/rknn-toolkit2/rknn_toolkit_lite2/examples/resnet18$ python test.py
--> Load RKNN model
done
--> Init runtime environment
I RKNN: [22:50:07.209] RKNN Runtime Information: librknnrt version: 1.4.0 (a10f100eb@2022-09-09T09:07:14)
I RKNN: [22:50:07.209] RKNN Driver Information: version: 0.8.2
E RKNN: [22:50:07.209] 6, 1
E RKNN: [22:50:07.209] Invalid RKNN model version 6
E RKNN: [22:50:07.209] rknn_init, load model failed!
E Catch exception when init runtime!
E Traceback (most recent call last):
  File "/home/orangepi/Desktop/sky_anomaly_detection/rknn-toolkit2-env/lib/python3.10/site-packages/rknnlite/api/rknn_lite.py", line 148, in init_runtime
    self.rknn_runtime.build_graph(self.rknn_data, self.load_model_in_npu)
  File "rknnlite/api/rknn_runtime.py", line 919, in rknnlite.api.rknn_runtime.RKNNRuntime.build_graph
Exception: RKNN init failed. error code: RKNN_ERR_FAIL

Init runtime environment failed

I try to run the examples from rknn toolkit lite and the above error comes out

@ZlodeiBaal

I'm trying to run the following script: https://github.com/rockchip-linux/rknn-toolkit2/blob/master/rknn_toolkit_lite2/examples/dynamic_shape/test.py

and found an error in the if in the main function. Here it is:

    if host_name == 'RK3588':
        # For RK3588, specify which NPU core the model runs on through the core_mask parameter.
        print('2')
        ret = rknn_lite.init_runtime(core_mask=RKNNLite.NPU_CORE_0)
        print('3')
    else:
        ret = rknn_lite.init_runtime()
    if ret != 0:
        print('Init runtime environment failed')
        exit(ret)
    print('done')

in line: ret = rknn_lite.init_runtime(core_mask=RKNNLite.NPU_CORE_0), and also in line exit(ret).

If I comment out the line with exit(ret), the following error appears:

(rknn-toolkit2-env) orangepi@orangepi5plus:~/Desktop/sky_anomaly_detection/rknn-toolkit2/rknn_toolkit_lite2/examples/dynamic_shape$ python test.py
--> Load RKNN model
done
--> Init runtime environment
1
2
I RKNN: [15:50:16.481] RKNN Runtime Information: librknnrt version: 1.4.0 (a10f100eb@2022-09-09-09T09:07:14)
I RKNN: [15:50:16.481] RKNN Driver Information: version: 0.8.2
E RKNN: [15:50:16.482] 6, 1
E RKNN: [15:50:16.482] Invalid RKNN model version 6
E RKNN: [15:50:16.482] rknn_init, load model failed!
E Catch exception when init runtime!
E Traceback (most recent call last):
  File “/home/orangepi/Desktop/sky_anomaly_detection/rknn-toolkit2-env/lib/python3.10/site-packages/rknnlite/api/rknn_lite.py”, line 148, in init_runtime
    self.rknn_runtime.build_graph(self.rknn_data, self.load_model_in_npu)
  File “rknnlite/api/rknn_runtime.py”, line 919, in rknnlite.api.rknn_runtime.RKNNRuntime.build_graph
Exception: RKNN init failed. error code: RKNN_ERR_FAIL

3
4
Init runtime environment failed
5
6
done
--> Running model
model: mobilenet_v2

input shape: 1,3,224,224
E Catch exception when setting inputs.
E Traceback (most recent call last):
  File “/home/orangepi/Desktop/sky_anomaly_detection/rknn-toolkit2-env/lib/python3.10/site-packages/rknnlite/api/rknn_lite.py”, line 200, in inference
    self.rknn_runtime.set_inputs(inputs, data_type, data_format, inputs_pass_through=inputs_pass_through)
  File “rknnlite/api/rknn_runtime.py”, line 983, in rknnlite.api.rknn_runtime.RKNNRuntime.set_inputs
  File “rknnlite/api/rknn_runtime.py”, line 1207, in rknnlite.api.rknn_runtime.RKNNRuntime.RKNNRuntime.get_in_out_num
ctypes.ArgumentError: argument 1: TypeError: wrong type

Traceback (most recent call last):
  File “/home/orangepi/Desktop/sky_anomaly_detection/rknn-toolkit2/rknn_toolkit_lite2/examples/dynamic_shape/test.py”, line 116, in <module>
    show_top5(outputs)
  File “/home/orangepi/Desktop/sky_anomaly_detection/rknn-toolkit2/rknn_toolkit_lite2/examples/dynamic_shape/test.py”, line 40, in show_top5
    output = result[0].reshape(-1)
TypeError: 'NoneType' object is not subscriptable

(print() added as a debug)

I think I've found the solution to running the test scripts:

It may be necessary to install and replace the librknnrt.so library with an older one, for this purpose it is suggested to download the following libraries from the github repository:

• librknnrt.so
• librknn_api.so

from here: https://github.com/rockchip-linux/rknpu2/tree/master/runtime/RK3588/Linux/librknn_api/aarch64

The next thing to do is as follows:

# Download the files with the command (everything happens in the “/Desktop/sky_anomaly_detection/rknn-toolkit2/rknn_toolkit_lite2/examples/dynamic_shape” directory):
wget https://raw.github.com/rockchip-linux/rknpu2/blob/master/runtime/RK3588/Linux/librknn_api/aarch64/librknnrt.so

# Rename those files that were on the computer to _old.so:
sudo mv /usr/lib/librknn_api.so /usr/lib/librknnrt_api_old.so
sudo mv /usr/lib/librknnrt.so /usr/lib/librknn_old.so

# Move all files starting with lib from “/Desktop/sky_anomaly_detection/rknn-toolkit2/rknn_toolkit_lite2/examples/dynamic_shape” directory to /usr/lib/ directory:
sudo mv lib* /usr/lib/

Then run the test file from the directory “/Desktop/sky_anomaly_detection/rknn-toolkit2/rknn_toolkit_lite2/examples/dynamic_shape”

python test.py

If the file runs and outputs something like the following:

(rknn-toolkit2-env) orangepi@orangepi5plus:~/Desktop/sky_anomaly_detection/rknn-toolkit2/rknn_toolkit_lite2/examples/dynamic_shape$ python test.py
--> Load RKNN model
done
--> Init runtime environment
1
2
I RKNN: [22:52:44.033] RKNN Runtime Information: librknnrt version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22)
I RKNN: [22:52:44.033] RKNN Driver Information: version: 0.8.2
W RKNN: [22:52:44.033] Current driver version: 0.8.2, recommend to upgrade the driver to the new version: >= 0.8.8
I RKNN: [22:52:44.033] RKNN Model Information: version: 6, toolkit version: 1.6.0+81f21f4d(compiler version: 1.6.0 (585b3edcf@2023-12-11T07:42:56)), target: RKNPU v2, target platform: rk3588, framework name: Caffe, framework layout: NCHW, model inference type: dynamic_shape
W RKNN: [22:52:44.033] RKNN Model version: 1.6.0 not match with rknn runtime version: 1.5.2
3
done
--> Running model
model: mobilenet_v2

input shape: 1,3,224,224
7
8
9
10
W The input[0] need NHWC data format, but NCHW set, the data format and data buffer will be changed to NHWC.
W RKNN: [22:52:44.052] Output(prob): size_with_stride larger than model origin size, if need run OutputOperator in NPU, please call rknn_create_memory using size_with_stride.
11
-----TOP 5-----
[155] score:0.936035 class:"Shih-Tzu"
[204] score:0.002516 class:"Lhasa, Lhasa apso"
[154] score:0.002172 class:"Pekinese, Pekingese, Peke"
[283] score:0.001601 class:"Persian cat"
[284] score:0.000286 class:"Siamese cat, Siamese"

12
input shape: 1,3,160,160
13
14
15
16
W The input[0] need NHWC data format, but NCHW set, the data format and data buffer will be changed to NHWC.
W RKNN: [22:52:44.059] Output(prob): size_with_stride larger than model origin size, if need run OutputOperator in NPU, please call rknn_create_memory using size_with_stride.
17
-----TOP 5-----
[155] score:0.606934 class:"Shih-Tzu"
[154] score:0.329834 class:"Pekinese, Pekingese, Peke"
[204] score:0.025085 class:"Lhasa, Lhasa apso"
[194] score:0.001038 class:"Dandie Dinmont, Dandie Dinmont terrier"
[219] score:0.000241 class:"cocker spaniel, English cocker spaniel, cocker"

18
input shape: 1,3,256,256
19
20
21
22
W The input[0] need NHWC data format, but NCHW set, the data format and data buffer will be changed to NHWC.
W RKNN: [22:52:44.064] Output(prob): size_with_stride larger than model origin size, if need run OutputOperator in NPU, please call rknn_create_memory using size_with_stride.
22
-----TOP 5-----
[155] score:0.927246 class:"Shih-Tzu"
[154] score:0.007225 class:"Pekinese, Pekingese, Peke"
[204] score:0.004616 class:"Lhasa, Lhasa apso"
[193] score:0.000878 class:"Australian terrier"
[283] score:0.000482 class:"Persian cat"

done
23
24

Congratulations, you can rejoice, everything is set up correctly!