Suse 11下多路径及udev配置

野性酷女 2022-08-11 16:59 315阅读 0赞

最近给客户基于SuSe 11 SP3下多路径部署Oracle 10g RAC。SuSe 11下用10g，也算一朵奇葩，连篇文档都比较难找，谁叫Oracle太贵呢。下面主要是描述了在该环境下如何去配置多路径。由于10g下的ocr与votingdisk不能直接存放到asm磁盘，所以依旧要使用raw设备方式来保存。下文供大家参考。

一、查看当前scsi设备及获取设备wwid  
\#使用 cat /proc/partitions或者/sbin/sfdisk命令查看当前系统中的scsi设备  
suse11a:~ \# more /proc/partitions  
major minor  \#blocks  name

8        0  877264896 sda  
   8        1     200812 sda1  
   8        2  104856255 sda2  
   8        3  104856255 sda3  
   8        4  667340100 sda4  
   8       16 2097152000 sdb  
   8       17 2097150976 sdb1  
               .....  
   8      240    1048576 sdp  
  65        0    1048576 sdq  
               .....  
  65       48 2097152000 sdt  
  65       49 2097150976 sdt1  
 253        0 2097152000 dm-0  
 253        1 2097152000 dm-1  
 253        2 1751121920 dm-2  
 253        3    1048576 dm-3  
 253        4    1048576 dm-4  
 253        5 1751120896 dm-5  
 253        6 2097150976 dm-6  
 253        7 2097150976 dm-7  
   
\#获取多路径设备的wwid，使用下面的命令来获取当前系统中多路径设备的wwid  
for i in \`cat /proc/partitions | awk \{'print $4'\} |grep sd\`; do echo "\#\#\# $i: \`/lib/udev/scsi\_id --whitelist  --replace-whitespace /dev/$i\`"; done

\#\#\# sda: 3600605b007030d301a192f2d13c017a0  
\#\#\# sda1: 3600605b007030d301a192f2d13c017a0  
     .....................  
\#\#\# sdt: 360022a11000cb6b10a902db200000000  
\#\#\# sdt1: 360022a11000cb6b10a902db200000000  
\#获取后，过滤本地磁盘及重复的wwid

二、配置mulitpatch.conf文件  
\#配置mulitpatch.conf文件的主要目的是用于使用有意义，用户自定义的设备名称，持久化多路径设备  
\#这个步骤是与配置单路径设备最大的区别  
\#下面是已经配置过后的multipath.conf文件  
suse11a:~ \# cp /etc/multipath.conf /etc/multipath.conf.bak  
suse11a:~ \# vi /etc/multipath.conf  
suse11a:~ \# grep -v ^\# /etc/multipath.conf  
defaults \{  
        user\_friendly\_names yes  
\}  
blacklist \{  
        devnode ^hd\[a-z\]  
\}  
multipaths \{  
      multipath \{  
              wwid                    36200bc710099b9190ba6fc350000000a  
              alias                   mpatha  
       \}

multipath \{  
              wwid                    36200bc710099b9190ba7109b0000000b  
              alias                   mpathb  
       \}  
      multipath \{  
              wwid                    36200bc710099b9190ab1155300000009  
              alias                   mpathc  
       \}  
      multipath \{  
              wwid                    360022a11000cb6b10a902db200000000  
              alias                   mpathd  
       \}  
       multipath \{  
              wwid                    360022a11000cb6b10a9116d000000002  
              alias                   mpathe  
       \}  
\}

三、校验多路径设备  
\#使用下面的命令用于多路径配置生效  
suse11a:~ \# service multipathd stop  
suse11a:~ \# service multipathd start

\#下面校验多路径设备  
suse11a:~ \# multipath -ll | grep mpath  
mpathe (360022a11000cb6b10a9116d000000002) dm-1 HUAWEI  ,S5500T           
mpathd (360022a11000cb6b10a902db200000000) dm-0 HUAWEI  ,S5500T           
mpathc (36200bc710099b9190ab1155300000009) dm-2 HUAWEI  ,S2600T           
mpathb (36200bc710099b9190ba7109b0000000b) dm-4 HUAWEI  ,S2600T           
mpatha (36200bc710099b9190ba6fc350000000a) dm-3 HUAWEI  ,S2600T

\#下面是设备的详细信息  
suse11a:~ \# multipath -ll  
mpathe (360022a11000cb6b10a9116d000000002) dm-1 HUAWEI  ,S5500T           
size=2.0T features='0' hwhandler='0' wp=rw  
|-+- policy='service-time 0' prio=1 status=active  
| \`- 5:0:0:1 sdc 8:32  active ready running  
|-+- policy='service-time 0' prio=1 status=enabled  
| \`- 5:0:3:1 sdk 8:160 active ready running  
|-+- policy='service-time 0' prio=1 status=enabled  
| \`- 6:0:2:1 sds 65:32 active ready running  
\`-+- policy='service-time 0' prio=1 status=enabled  
  \`- 6:0:3:1 sdu 65:64 active ready running  
mpathd (360022a11000cb6b10a902db200000000) dm-0 HUAWEI  ,S5500T           
size=2.0T features='0' hwhandler='0' wp=rw  
|-+- policy='service-time 0' prio=1 status=active  
| \`- 5:0:0:0 sdb 8:16  active ready running  
|-+- policy='service-time 0' prio=1 status=enabled  
| \`- 5:0:3:0 sdj 8:144 active ready running  
|-+- policy='service-time 0' prio=1 status=enabled  
| \`- 6:0:2:0 sdr 65:16 active ready running  
\`-+- policy='service-time 0' prio=1 status=enabled  
  \`- 6:0:3:0 sdt 65:48 active ready running  
            .........

suse11a:~ \# dmsetup ls |sort  
mpatha  (253:3)  
mpathb  (253:4)  
mpathc  (253:2)  
mpathc\_part1    (253:5)  
mpathd  (253:0)  
mpathd\_part1    (253:6)  
mpathe  (253:1)  
mpathe\_part1    (253:7)  
     
\#以下命令均可以用于校验多路径设备  
\# ll /dev/disk/by-id/  
\# ls -l /dev/dm-\*  
\# ll /dev/mapper/

\#/dev/dm-\*下面的为动态名，即系统重启后会发生变化，此部分用于os 内部使用。  
\#/dev/mapper/下面查看到的内容为持久化名称，也即是我们在mulitpatch.conf文件中定义的。

四、配置RAW  
\#在Oracle 10g R2 rac安装的过程中，OUI不能够校验共享的快涉笔，因此需要使用raw。Oracle 11g R1后可以直接使用多路径设备。  
\#使用udev来管理raw，编辑raw文件，如下  
suse11a:~ \# cat /etc/raw | grep -v \\\#  
raw1:mapper/mpatha  
raw2:mapper/mpathb  
raw3:mapper/mpathc  
raw4:mapper/mpathd  
raw5:mapper/mpathe

\#重启raw以及配置开机启动raw  
suse11a:~ \# rcraw start  
suse11a:~ \# chkconfig raw on  
suse11a:~ \# raw -qa  
/dev/raw/raw1:  bound to major 253, minor 3  
/dev/raw/raw2:  bound to major 253, minor 4  
/dev/raw/raw3:  bound to major 253, minor 2  
/dev/raw/raw4:  bound to major 253, minor 0  
/dev/raw/raw5:  bound to major 253, minor 1

\#测试raw的可访问性  
\# dd if=/dev/zero of=/dev/raw/raw1 bs=1024 count=100  
100+0 records in  
100+0 records out  
102400 bytes (102 kB) copied, 0.762352 seconds, 134 kB/s

\#配置权限，该权限用于oracle用户访问裸设备  
\# Author : Leshami  
\# Blog   : [http://blog.csdn.net/leshami][http_blog.csdn.net_leshami]  
suse11a:~ \# more /etc/udev/rules.d/99-raw.rules  
KERNEL=="raw\[1-2\]\*", GROUP="oinstall", MODE="664"  
KERNEL=="raw\[3-5\]\*", OWNER="oracle", GROUP="oinstall", MODE="664"

五、安装clusterware  
\#安装clusterware时，关于ocr和votingdisk，选择/dev/raw/raw1，/dev/raw/raw2，如果有配置冗余，此处应该为多个裸设备。  
\#校验安转好clusterware后裸设备的使用情况  
suse11a:oracle:ora10g1 > ocrcheck  
Status of Oracle Cluster Registry is as follows :  
         Version                  :          2  
         Total space (kbytes)     :    1048296  
         Used space (kbytes)      :       4592  
         Available space (kbytes) :    1043704  
         ID                       :  892902908  
         Device/File Name         : /dev/raw/raw1  
                                    Device/File integrity check succeeded

Device/File not configured

Cluster registry integrity check succeeded

suse11a:oracle:ora10g1 > crsctl query  css votedisk  
 0.     0    /dev/raw/raw2

located 1 votedisk(s).   
\#对于ASM磁盘可以通过dbca来创建，直接使用/dev/raw/raw\*以及/dev/mapper/\*。当然suse 11支持asmlib，也可以通过asmlib方式来访问asm磁盘。

[![鹏城DBA总群][DBA]][DBA_DBA]

[http_blog.csdn.net_leshami]: http://blog.csdn.net/leshami
[DBA]: http://pub.idqqimg.com/wpa/images/group.png
[DBA_DBA]: /images/20220810/1606a162ea6544eb9847bbddf5fe0fc4.png