kaldi的编译安装与报错解决方法

曾经终败给现在 2022-01-23 12:59 1656阅读 0赞

# 引入 #

kaldi是语音识别领域，最常用的一个工具。

它自带了很多特征提取模块，能提取MFCC/ivector/xvector等语音特征；也自带了很多语音模型代码，可以直接使用或重新训练GMM-HMM等模型；它还支持GPU进行训练。可以说是功能很强大了。

更厉害的是，你只需要简单的SHELL编程，就能使用kaldi。kaldi作为一个工具，不需要像库一样进行大量编程，所以使用门槛其实不高。

但是，它并没有做到像tensorflow/keras这样非常容易的安装配置。kaldi是需要自己在不同平台上重新编译，才能使用的。这就需要我们配置编译环境，以及各种编译依赖。

虽然有一个python版本的[pykaldi][]，但它只是一个wrapper，底层还是得自己重新编译安装kaldi才能使用。并且笔者和pykaldi的研发人员交流过，它目前并不支持kaldi的所有功能。感兴趣的读者可以查看这里（[https://github.com/pykaldi/pykaldi/issues/101）。][https_github.com_pykaldi_pykaldi_issues_101]

相信很多新手在编译kaldi时，都遇到过很多问题。网上有一些资料（虽然有的比较老，与目前的编译方式不一样了），也能搜到一些解决方法，但笔者还是遇到了不少需要自己解决的问题。

所以把自己编译安装kaldi的过程，以及遇到问题的解决方法记录在此，供参考。

# 编译安装kaldi #

我的系统是Linux Ubuntu 16.04。具体过程如下

1.  安装一些依赖工具与第三方库

sudo apt-get update
    sudo apt-get upgrade
    sudo apt-get install git
    sudo apt-get install bc
    sudo apt-get install g++
    sudo apt-get install zlib1g-dev make automake autoconf bzip2 libtool subversion
    sudo apt-get install libatlas3-base

1.  下载kaldi，目前kaldi最新版本的代码都是在github（早期的老版本是在svn）

git clone https://github.com/kaldi-asr/kaldi.git
    cd kaldi/tools

1.  用kaldi自带的脚本`check_dependencies.sh`来检测是否安装完所有必须的依赖工具

extras/check_dependencies.sh

根据提示安装必须的依赖。我这里提示缺少MKL，也提示了需要用tools目录下自带的脚本`extras/install_mkl.sh`来安装MKL。

1.  安装MKL

运行脚本`extras/install_mkl.sh`。

extras/install_mkl.sh

安装过程中报错如下：

AppStream cache update completed, but some metadata was ignored due to errors.
    Reading package lists... Done
    E: Failed to fetch http://security.ubuntu.com/ubuntu/dists/xenial-security/multiverse/dep11/Components-amd64.yml  Could not open file /var/lib/apt/lists/parti
    al/security.ubuntu.com_ubuntu_dists_xenial-security_multiverse_dep11_Components-amd64.yml.xz - open (13: Permission denied) [IP: 91.xxx.88.xxx 80]
    E: Some index files failed to download. They have been ignored, or old ones used instead.
    extras/install_mkl.sh: MKL package intel-mkl-64bit-2019.2-057 installation FAILED.
    
    Please open an issue with us at https://github.com/kaldi-asr/kaldi/ if you
    believe this is a bug.

用ll命令，检查报错的文件，发现是个链接，如下所示：

kaldi/tools# ll /var/lib/apt/lists/partial/security.ubuntu.com_ubuntu_dists_xenial-security_multiverse_dep11_Components-amd64.yml.xz
    
    lrwxrwxrwx 1 root root 108 Nov 22 01:31 /var/lib/apt/lists/partial/security.ubuntu.com_ubuntu_dists_xenial-security_multiverse_dep11_Components-amd64.yml.xz -> /var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_xenial-security_multiverse_dep11_Components-amd64.yml.gz

查看链接到的目标文件权限，发现它没有写权限。这就是问题所在了，上面报错是`Permission denied`，正好是权限不够。

kaldi/tools# ll /var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_xenial-security_multiverse_dep11_Components-amd64.yml.gz
    
    -rw-r--r-- 1 root root 158 Feb  2  2017 /var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_xenial-security_multiverse_dep11_Components-amd64.yml.gz

改动该文件的权限，让其可写，如下所示

kaldi/tools# chmod 666  /var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_xenial-security_multiverse_dep11_Components-amd64.yml.gz

然后，重新安装mkl

extras/install_mkl.sh

得到如下提示，说明MKL安装成功

extras/install_mkl.sh: Configuring ld runtime bindings
    + echo '/opt/intel/lib/intel64
    /opt/intel/mkl/lib/intel64'
    + ldconfig
    extras/install_mkl.sh: MKL package intel-mkl-64bit-2019.2-057 was successfully installed

在运行`check_dependencies.sh`，检查依赖，得到如下提示，则说明所有依赖都安装成功。

# extras/check_dependencies.sh
    
    extras/check_dependencies.sh: all OK.

1.  编译tools

依赖安装成功，说明编译所需的工具和环境都配置好了，接下来就可以编译tools。

kaldi/tools#  make -j 8

1.  编译src

tools编译成功后，就可以到src目录下，编译src

kaldi/src#  ./configure --shared
    
    kaldi/src#  make depend -j
    
    kaldi/src#  make -j

这个步骤比较花时间，编译成功后，提示如下

Running matrix-lib-test ... 3s... SUCCESS matrix-lib-test
    Running sparse-matrix-test ... 0s... SUCCESS sparse-matrix-test
    make[1]: Leaving directory '/home/rootuser/speech_2019/kaldi/src/matrix'
    echo Done
    Done

接下来，可以跑一下最简单的例子，来验证kaldi是否安装成功。

# 跑yesno例子来验证安装成功 #

运行自带的yesno例子，跑`./run.sh`即可运行：

kaldi/egs/yesno/s5# ls
    
    conf  input  local  path.sh  run.sh  steps  utils
    
    kaldi/egs/yesno/s5# ./run.sh

如果没有报错，那恭喜您，到这里，kaldi编译安装成功！

# 附录：运行yesno例子的完整输出 #

kaldi/egs/yesno/s5# ./run.sh
    
    steps/train_mono.sh: Pass 25
    steps/train_mono.sh: Pass 26
    steps/train_mono.sh: Aligning data
    steps/train_mono.sh: Pass 27
    steps/train_mono.sh: Pass 28
    steps/train_mono.sh: Pass 29
    steps/train_mono.sh: Aligning data
    steps/train_mono.sh: Pass 30
    steps/train_mono.sh: Pass 31
    steps/train_mono.sh: Pass 32
    steps/train_mono.sh: Aligning data
    steps/train_mono.sh: Pass 33
    steps/train_mono.sh: Pass 34
    steps/train_mono.sh: Pass 35
    steps/train_mono.sh: Aligning data
    steps/train_mono.sh: Pass 36
    steps/train_mono.sh: Pass 37
    steps/train_mono.sh: Pass 38
    steps/train_mono.sh: Aligning data
    steps/train_mono.sh: Pass 39
    steps/diagnostic/analyze_alignments.sh --cmd utils/run.pl data/lang exp/mono0a
    steps/diagnostic/analyze_alignments.sh: see stats in exp/mono0a/log/analyze_alignments.log
    1 warnings in exp/mono0a/log/update.*.log
    6 warnings in exp/mono0a/log/init.log
    exp/mono0a: nj=1 align prob=-81.88 over 0.05h [retry=0.0%, fail=0.0%] states=11 gauss=371
    steps/train_mono.sh: Done training monophone system in exp/mono0a
    tree-info exp/mono0a/tree
    tree-info exp/mono0a/tree
    fstpushspecial
    fstminimizeencoded
    fsttablecompose data/lang_test_tg/L_disambig.fst data/lang_test_tg/G.fst
    fstdeterminizestar --use-log=true
    fstisstochastic data/lang_test_tg/tmp/LG.fst
    0.534295 0.533859
    [info]: LG not stochastic.
    fstcomposecontext --context-size=1 --central-position=0 --read-disambig-syms=data/lang_test_tg/phones/disambig.int --write-disambig-syms=data/lang_test_tg/tmp
    /disambig_ilabels_1_0.int data/lang_test_tg/tmp/ilabels_1_0.21306 data/lang_test_tg/tmp/LG.fst
    fstisstochastic data/lang_test_tg/tmp/CLG_1_0.fst
    0.534295 0.533859
    [info]: CLG not stochastic.
    make-h-transducer --disambig-syms-out=exp/mono0a/graph_tgpr/disambig_tid.int --transition-scale=1.0 data/lang_test_tg/tmp/ilabels_1_0 exp/mono0a/tree exp/mono
    0a/final.mdl
    fsttablecompose exp/mono0a/graph_tgpr/Ha.fst data/lang_test_tg/tmp/CLG_1_0.fst
    fstdeterminizestar --use-log=true
    fstminimizeencoded
    fstrmsymbols exp/mono0a/graph_tgpr/disambig_tid.int
    fstrmepslocal
    fstisstochastic exp/mono0a/graph_tgpr/HCLGa.fst
    0.5342 -0.000299216
    HCLGa is not stochastic
    add-self-loops --self-loop-scale=0.1 --reorder=true exp/mono0a/final.mdl exp/mono0a/graph_tgpr/HCLGa.fst
    steps/decode.sh --nj 1 --cmd utils/run.pl exp/mono0a/graph_tgpr data/test_yesno exp/mono0a/decode_test_yesno
    decode.sh: feature type is delta
    steps/diagnostic/analyze_lats.sh --cmd utils/run.pl exp/mono0a/graph_tgpr exp/mono0a/decode_test_yesno
    steps/diagnostic/analyze_lats.sh: see stats in exp/mono0a/decode_test_yesno/log/analyze_alignments.log
    Overall, lattice depth (10,50,90-percentile)=(1,1,2) and mean=1.2
    steps/diagnostic/analyze_lats.sh: see stats in exp/mono0a/decode_test_yesno/log/analyze_lattice_depth_stats.log
    local/score.sh --cmd utils/run.pl data/test_yesno exp/mono0a/graph_tgpr exp/mono0a/decode_test_yesno
    local/score.sh: scoring with word insertion penalty=0.0,0.5,1.0
    %WER 0.00 [ 0 / 232, 0 ins, 0 del, 0 sub ] exp/mono0a/decode_test_yesno/wer_10_0.0

# 参考 #

*  [https://techsupportallbugs.wordpress.com/2018/02/28/install-kaldi-asr-on-ubuntu/][https_techsupportallbugs.wordpress.com_2018_02_28_install-kaldi-asr-on-ubuntu]
 *  [https://blog.csdn.net/zl87758539/article/details/73647963][https_blog.csdn.net_zl87758539_article_details_73647963]

[pykaldi]: https://github.com/pykaldi/pykaldi
[https_github.com_pykaldi_pykaldi_issues_101]: https://github.com/pykaldi/pykaldi/issues/101%EF%BC%89%E3%80%82
[https_techsupportallbugs.wordpress.com_2018_02_28_install-kaldi-asr-on-ubuntu]: https://techsupportallbugs.wordpress.com/2018/02/28/install-kaldi-asr-on-ubuntu/
[https_blog.csdn.net_zl87758539_article_details_73647963]: https://blog.csdn.net/zl87758539/article/details/73647963