Nvidia 显卡 Failed to initialize NVML Driver/library version mismatch 错误解决方案（不重启方案）【简单方案：重启】

清疚 2023-09-28 13:06 25阅读 0赞

本文记录错误 Failed to initialize NVML: Driver/library version mismatch 错误解决方案。（使用GPU运行程序会出现：RuntimeError: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW）

问题复现

$ nvidia-smi 
    
    -->
    Failed to initialize NVML: Driver/library version mismatch

问题分析

*  NVIDIA 内核驱动版本与系统驱动不一致

查看显卡驱动所使用的内核版本

cat /proc/driver/nvidia/version
    
    -->
    NVRM version: NVIDIA UNIX x86_64 Kernel Module  430.34  Wed Jun 26 12:19:48 CDT 2019
    GCC version:  gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.12)

*  内核版本 Kernel Module 为 430.34， 系统内核 16.04.12

查看系统驱动日志

cat /var/log/dpkg.log | grep nvidia
    
    -->
    2021-03-30 14:04:55 install libnvidia-compute-460-server:amd64 <none> 460.32.03-0ubuntu0.18.04.2
    2021-03-30 14:04:55 status half-installed libnvidia-compute-460-server:amd64 460.32.03-0ubuntu0.18.04.2
    2021-03-30 14:04:57 status unpacked libnvidia-compute-460-server:amd64 460.32.03-0ubuntu0.18.04.2
    2021-03-30 14:04:57 status unpacked libnvidia-compute-460-server:amd64 460.32.03-0ubuntu0.18.04.2
    2021-03-30 14:05:15 install nvidia-cuda-dev:amd64 <none> 9.1.85-3ubuntu1
    2021-03-30 14:05:15 status half-installed nvidia-cuda-dev:amd64 9.1.85-3ubuntu1
    2021-03-30 14:05:34 status unpacked nvidia-cuda-dev:amd64 9.1.85-3ubuntu1
    2021-03-30 14:05:34 status unpacked nvidia-cuda-dev:amd64 9.1.85-3ubuntu1
    2021-03-30 14:05:34 install nvidia-cuda-doc:all <none> 9.1.85-3ubuntu1
    2021-03-30 14:05:34 status half-installed nvidia-cuda-doc:all 9.1.85-3ubuntu1
    2021-03-30 14:05:38 status unpacked nvidia-cuda-doc:all 9.1.85-3ubuntu1
    2021-03-30 14:05:38 status unpacked nvidia-cuda-doc:all 9.1.85-3ubuntu1
    2021-03-30 14:05:38 install nvidia-cuda-gdb:amd64 <none> 9.1.85-3ubuntu1
    2021-03-30 14:05:38 status half-installed nvidia-cuda-gdb:amd64 9.1.85-3ubuntu1
    2021-03-30 14:05:39 status unpacked nvidia-cuda-gdb:amd64 9.1.85-3ubuntu1
    2021-03-30 14:05:39 status unpacked nvidia-cuda-gdb:amd64 9.1.85-3ubuntu1
    2021-03-30 14:05:39 install nvidia-profiler:amd64 <none> 9.1.85-3ubuntu1
    2021-03-30 14:05:39 status half-installed nvidia-profiler:amd64 9.1.85-3ubuntu1

可以看到曾经安装过系统内核 18.04 的 460.32 的驱动  
查看驱动程序

sudo dpkg --list | grep nvidia-*
    
    -->
    ii  libnvidia-compute-460-server:amd64   460.32.03-0ubuntu0.18.04.2                 amd64        NVIDIA libcompute package
    ii  libnvidia-container-tools            1.0.5-1                                    amd64        NVIDIA container runtime library (command-line tools)
    ii  libnvidia-container1:amd64           1.0.5-1                                    amd64        NVIDIA container runtime library
    ii  nvidia-container-runtime             3.1.4-1                                    amd64        NVIDIA container runtime
    ii  nvidia-container-toolkit             1.0.5-1                                    amd64        NVIDIA container runtime hook
    ii  nvidia-cuda-dev                      9.1.85-3ubuntu1                            amd64        NVIDIA CUDA development files
    ii  nvidia-cuda-doc                      9.1.85-3ubuntu1                            all          NVIDIA CUDA and OpenCL documentation

*  可以看到系统安装了ubuntu 内核18.04 下的 nvidia 460 驱动
 *  实际系统内核版本与驱动需求的版本不一致是问题产生的根源

解决方案

*  卸载现有驱动，重新安装

卸载驱动

sudo /usr/bin/nvidia-uninstall
    sudo apt-get --purge remove nvidia-*
    sudo apt-get purge nvidia*
    sudo apt-get purge libnvidia*

直到命令不输出任何内容

sudo dpkg --list | grep nvidia-*

重新安装

sudo chmod a+x NVIDIA-Linux-x86_64-450.80.02.run
    sudo ./NVIDIA-Linux-x86_64-450.80.02.run -no-x-check -no-nouveau-check -no-opengl-files

–no-opengl-files 只安装驱动文件,不安装OpenGL文件
    
    –no-x-check 安装驱动时不检查X服务
    
    –no-nouveau-check 安装驱动时不检查nouveau

查看驱动更新结果

$ nvidia-smi

![6fc2f8c3d9d30aad680ff9ccfbf34e96.png][]

参考资料  
https://blog.csdn.net/qq\_40200387/article/details/90341107

https://www.zywvvd.com/2020/12/03/linux/driver/nvidia-driver-install-linux/

[【nvidia-smi】Failed to initialize NVML: Driver/library version mismatch解决方法(不用重启)\_Wumbuk的博客-CSDN博客][nvidia-smi_Failed to initialize NVML_ Driver_library version mismatch_Wumbuk_-CSDN]

[6fc2f8c3d9d30aad680ff9ccfbf34e96.png]: https://img-blog.csdnimg.cn/img_convert/6fc2f8c3d9d30aad680ff9ccfbf34e96.png
[nvidia-smi_Failed to initialize NVML_ Driver_library version mismatch_Wumbuk_-CSDN]: https://blog.csdn.net/qq_48081868/article/details/122349627