Windows 11部署FunASR离线语音识别系统

windows 11部署funasr离线语音识别系统

官网连接

https://github.com/alibaba-damo-academy/funasr/blob/main/runtime/docs/sdk_advanced_guide_online_zh.md

1-安装docker

运行docker desktop installer.exe安装docker

2-windows添加删除程序增加虚拟机和linux子系统功能

hyper-v（windows 11可能不显示，通过命令systeminfo显示：hyper-v 要求: 已检测到虚拟机监控程序。将不显示 hyper-v 所需的功能。说明系统支持hyper-v的）
适用于linux的windows子系统
虚拟机平台
windows虚拟机监控程序平台

3-升级linux子系统（可选）

wsl --update

4-部署docker

docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.9
mkdir d://funasr//model

5-启动docker

docker run -p 10095:10095 -it --privileged=true -v d:/funasr/model:/workspace/models registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.9

6-启动funasr

cd funasr/runtime
nohup bash run_server_2pass.sh --certfile 0  --download-model-dir /workspace/models --vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx --model-dir damo/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-onnx  --online-model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online-onnx  --punc-dir damo/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727-onnx --itn-dir thuduj12/fst_itn_zh > log.txt 2>&1 &

7-关闭funasr

ps -x | grep funasr-wss-server-2pass
kill -9 pid

8-打开新的控制台查看执行日志

docker exec -it pid /bin/sh
tail -f nohup.out

9-批处理自动执行

@echo off
rem 设置container_name变量为容器的pid
set container_name=3991fdb6c269

chcp 65001

rem 等待 30 秒让容器完全启动
timeout /t 30

docker start %container_name%

rem 进入 docker 容器
docker exec -it %container_name% bash -c "cd funasr/runtime && nohup bash run_server_2pass.sh --certfile 0 --download-model-dir /workspace/models --vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx --model-dir damo/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-onnx --online-model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online-onnx --punc-dir damo/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727-onnx --itn-dir thuduj12/fst_itn_zh > log.txt 2>&1 &"

rem 实时查看日志
docker exec -it %container_name% bash -c "tail -f funasr/runtime/nohup.out"

:: 暂停，等待用户按下任意键
pause

10-c#客户端

流程：
客户端 ----------发起websocket连接---------->服务器
客户端----------发送json握手协议---------->服务器
客户端----------发送pcm二进制数据帧---------->服务器
客户端<----------返回json数据----------服务器
…
握手协议：

{"chunk_size":[5,10,5],"wav_name":"h5","is_speaking":true,"chunk_interval":10,"itn":false,"mode":"2pass","hotwords":"{\"阿里巴巴\":20,\"hello world\":40}"}

返回json数据有两种
实时返回数据

{"is_final":false,"mode":"2pass-online","text":"小明","wav_name":"h5"}

返回断句数据

{"is_final":false,"mode":"2pass-offline","stamp_sents":[{"end":39950,"punc":"。","start":39670,"text_seg":"小 明","ts_list":[[39670,39850],[39850,39950]]},{"end":40465,"punc":"","start":39950,"text_seg":"小 明","ts_list":[[39950,40050],[40050,40465]]}],"text":"小明。小明","timestamp":"[[39670,39850],[39850,39950],[39950,40050],[40050,40465]]","wav_name":"h5"}

不同点在于mode是online还是offline。

主要代码

  /*
   * 与funasr建立websocket连接
   * */
  public async task startwebsocketsync()
  {
    // 全局忽略证书验证错误
    servicepointmanager.servercertificatevalidationcallback += (sender, cert, chain, sslpolicyerrors) => true;

    string wssurl = "ws://127.0.0.1:10095";
    websocket = new clientwebsocket();
   
    try
    {
      await websocket.connectasync(new uri(wssurl), cancellationtoken.none);
      await sendinitialmessage(websocket);
      console.writeline("websocket已连接...");
      bwebsocketready = true;
      await task.whenall(receivemessagessync(websocket), sendaudiosync(websocket));
    }
    catch (exception ex)
    {
      console.writeline($"websocket连接错误: {ex.message}");
    }
  }
  /*
   * 发送握手数据
   * */
  private async task sendinitialmessage(clientwebsocket websocket)
  {
    var request = new
    {
      chunk_size = new int[] { 5, 10, 5 },
      wav_name = "h5",
      is_speaking = true,
      chunk_interval = 10,
      itn = false,
      mode = "2pass"
    };
    string jsonrequest = jsonconvert.serializeobject(request);
    byte[] bytes = encoding.utf8.getbytes(jsonrequest);
    await websocket.sendasync(new arraysegment<byte>(bytes), websocketmessagetype.text, true, cancellationtoken.none);
  }
  /*
   * 接收数据并处理
   * */
  private async task receivemessagessync(clientwebsocket ws)
  {
    var buffer = new byte[1024 * 100];
    while (ws.state == websocketstate.open)
    {
      var result = await ws.receiveasync(new arraysegment<byte>(buffer), cancellationtoken.none);
      if (result.messagetype == websocketmessagetype.close)
      {
        await ws.closeasync(websocketclosestatus.normalclosure, string.empty, cancellationtoken.none);
        console.writeline("websocket已关闭。");
      }
      else
      {
        string message = encoding.utf8.getstring(buffer, 0, result.count);
        if (!string.isnullorempty(message))
        {
          // handlemessage(message); // 收到为包含识别文本的json数据
        }
      }
    }
  }
  /*
   * 开始监听
   * */
    private async task startlisteningasync(cancellationtoken cancellationtoken)
  {
    var wavein = new waveinevent
    {
      waveformat = new waveformat(16000, 1) // 使用16khz单声道
    };

    wavein.dataavailable += (s, e) =>
    {
      lock (this)
      {
        // 读取数据
        byte[] bytes = new byte[e.bytesrecorded];
        for (var i = 0; i < e.bytesrecorded; i++)
        {
          bytes[i] = e.buffer[i];
        }

        // 缓存数据
        _wavebuffer.add(bytes);
      }
    };

    wavein.startrecording();
    console.writeline("开始监听麦克风...");

    while (!cancellationtoken.iscancellationrequested)
    {
      await startwebsocketsync();
      await task.delay(100, cancellationtoken); // 适当的延迟
    }

    wavein.stoprecording();
    console.writeline("停止监听麦克风。");
  }