k8s glusterfs failed to get the port number for remote subvolume. Please run ‘gluster volume status‘

柔情只为你懂 2022-09-02 00:50 6阅读 0赞

错误信息

warning
    (combined from similar events): MountVolume.SetUp failed for volume "pvc-448a0c86-fc65-447d-aad7-5db11144b056" : 
    mount failed: mount failed: exit status 1 Mounting command: systemd-run Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/e9e44e7d-2972-448e-a97e-3151151980c5/volumes/kubernetes.io~glusterfs/pvc-448a0c86-fc65-447d-aad7-5db11144b056 --scope -- mount -t glusterfs -o auto_unmount,log-file=/var/lib/kubelet/plugins/kubernetes.io/glusterfs/pvc-448a0c86-fc65-447d-aad7-5db11144b056/sonatype-nexus-7475c48fd-tmht7-glusterfs.log,log-level=ERROR 10.2.1.247:vol_7fed0ddb79a8644e46ca4efc1e7de1b0 /var/lib/kubelet/pods/e9e44e7d-2972-448e-a97e-3151151980c5/volumes/kubernetes.io~glusterfs/pvc-448a0c86-fc65-447d-aad7-5db11144b056 Output: Running scope as unit run-17649.scope. Mount failed. Check the log file /var/lib/kubelet/plugins/kubernetes.io/glusterfs/pvc-448a0c86-fc65-447d-aad7-5db11144b056/sonatype-nexus-7475c48fd-tmht7-glusterfs.log for more details. , the following error information was pulled from the glusterfs log to help diagnose this issue: [2021-07-29 02:31:22.557002] E [MSGID: 114058] [client-handshake.c:1448:client_query_portmap_cbk] 0-vol_7fed0ddb79a8644e46ca4efc1e7de1b0-client-0: failed to get the port number for remote subvolume. Please run 'gluster volume status' on server to see if brick process is running. [2021-07-29 02:31:22.558947] E [fuse-bridge.c:5338:fuse_first_lookup] 0-fuse: first lookup on root failed (Transport endpoint is not connected)

登录glusterfs daemonset对应pod

执行

gluster volume status | grep 7fed -A5

[root@10-2-1-247 /]# gluster volume status | grep 7fed -A5
    Status of volume: vol_7fed0ddb79a8644e46ca4efc1e7de1b0
    Gluster process                             TCP Port  RDMA Port  Online  Pid
    ------------------------------------------------------------------------------
    Brick 10.2.1.247:/var/lib/heketi/mounts/vg_
    da3c5c7b97f48e8cbe1144fb723137a6/brick_0196
    fb3f06ced9c2045946a8258f5349/brick          N/A       N/A        N       N/A  
    --
    Task Status of Volume vol_7fed0ddb79a8644e46ca4efc1e7de1b0
    ------------------------------------------------------------------------------
    There are no active volume tasks
     
    Status of volume: vol_897e7ef257eae053f30af7f9e20c6c98
    Gluster process                             TCP Port  RDMA Port  Online  Pid

vol\_7fed0ddb79a8644e46ca4efc1e7de1b0已经是开启状态

[root@10-2-1-247 /]# gluster volume info | grep 7fed -A14
    Volume Name: vol_7fed0ddb79a8644e46ca4efc1e7de1b0
    Type: Distribute
    Volume ID: c5ba8345-4e6e-4074-b450-0698b360b0ab
    Status: Started
    Snapshot Count: 0
    Number of Bricks: 1
    Transport-type: tcp
    Bricks:
    Brick1: 10.2.1.247:/var/lib/heketi/mounts/vg_da3c5c7b97f48e8cbe1144fb723137a6/brick_0196fb3f06ced9c2045946a8258f5349/brick
    Options Reconfigured:
    nfs.disable: on
    storage.fips-mode-rchecksum: on
    transport.address-family: inet
    user.heketi.id: 7fed0ddb79a8644e46ca4efc1e7de1b0

重启这个volume

[root@10-2-1-247 /]# gluster volume stop vol_7fed0ddb79a8644e46ca4efc1e7de1b0
    Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y
    volume stop: vol_7fed0ddb79a8644e46ca4efc1e7de1b0: success
    [root@10-2-1-247 /]# gluster volume start vol_7fed0ddb79a8644e46ca4efc1e7de1b0
    volume start: vol_7fed0ddb79a8644e46ca4efc1e7de1b0: success
    [root@10-2-1-247 /]# 
    [root@10-2-1-247 /]# 
    [root@10-2-1-247 /]# mount -t glusterfs -o auto_unmount,log-file=/tmp/gluster.log,log-level=DEBUG  10.2.1.247:vol_7fed0ddb79a8644e46ca4efc1e7de1b0 /tmp/test/

可以看到测试挂载成功