zipformer语音识别的部署

1、简介

Chinese-English ASR model using k2-zipformer-streaming.

The model used in this example comes from the following open source projects:  

https://huggingface.co/csukuangfj/k2fsa-zipformer-bilingual-zh-en-t

经测试在电脑端可以正常跑

开发板3588上跑缺失一个安装包报错

Traceback (most recent call last):
  File "/home/orangepi/work_11.15/rknn_model_zoo/examples/zipformer/python/zipformer.py", line 4, in <module>
    import kaldifeat

解决方法：安装kaldifeat模块包（这里没有解决）

官网没有找到在开发板3588上可以运行的安装包；有操作成功的可以在评论区给一个链接，感谢！

2.、支持平台

RK3566, RK3568, RK3588, RK3562, RK3576

3.、下载模型（在3588上）

cd model

 

./download_model.sh

4、模型转换

cd python

python convert.py ../model/encoder-epoch-99-avg-1.onnx rk3588

# output model will be saved as ../model/encoder-epoch-99-avg-1.rknn3

python convert.py ../model/decoder-epoch-99-avg-1.onnx rk3588

# output model will be saved as ../model/decoder-epoch-99-avg-1.rknn

python convert.py ../model/joiner-epoch-99-avg-1.onnx rk3588

# output model will be saved as ../model/joiner-epoch-99-avg-1.rknn

5、运行

From pre-compiled wheels (Recommended) — kaldifeat 1.25.5 documentation

https://csukuangfj.github.io/kaldifeat/cpu.html

# Install kaldifeat
# Refer to https://csukuangfj.github.io/kaldifeat/installation/from_wheels.html for installation.
# This python demo is tested under version: kaldifeat-1.25.4.dev20240223

开发板3588上跑

cd python

python zipformer.py --encoder_model_path encoder-epoch-99-avg-1.rknn --decoder_model_path decoder-epoch-99-avg-1.rknn --joiner_model_path joiner-epoch-99-avg-1.rknn --target 3588

电脑端跑

python zipformer.py --encoder_model_path encoder-epoch-99-avg-1.onnx --decoder_model_path decoder-epoch-99-avg-1.onnx --joiner_model_path joiner-epoch-99-avg-1.onnx 

成功运行输出：

ONNX转换为RKNN模型需要使用官方rknn_model_zoo工具：rknn_model_zoo-2.2.0