基于ZK自动切换模式的实现（六）

Myth丶恋晨 2022-01-26 09:05 190阅读 0赞

# 配置自动切换模式 #

（1）配置zookeeper集群

（2）开启自动切换模式

在hdfs-site.xml中配置dfs.ha.automatic-failover.enabled参数

（3）配置zookeeper实例

在core-site.xml中配置ha.zookeeper.quorum

（4）初始化zookeeper

（5）启动journalnode，namenode和datanode

（6）启动zkfc

根据主机名可以判断每一条命令是在那个节点执行，是用哪个用户执行的

例子：

\[hadoop@master opt\]$

mater主机

hadoop用户

**1） 上传zookeeper-3.4.6.tar.gz**

**2） 解压**

\[hadoop@master opt\]$ mv zookeeper-3.4.6.tar.gz /opt

\[hadoop@master opt\]$ tar -zxvf zookeeper-3.4.6.tar.gz

**3） 修改配置文件**

\[hadoop@master conf\]$ cd /opt/zookeeper-3.4.6/conf

\[hadoop@master conf\]$ cp zoo\_sample.cfg zoo.cfg

\[hadoop@master conf\]$ vi zoo.cfg

dataDir=/opt/zookeeper/data

server.1=hadoop04:8001:8002

server.2=slave1:8001:8002

server.3=slave2:8001:8002

**4）将zookeeper软件copy到规划好的节点**

\[hadoop@master opt\]$ scp -r zookeeper-3.4.6 hadoop04:/opt

\[hadoop@master opt\]$ scp -r zookeeper-3.4.6 slave1:/opt

\[hadoop@master opt\]$ scp -r zookeeper-3.4.6 slave2:/opt

**5）在hadoop04、slave1、slave2上创建链接**

\[hadoop@hadoop04 opt\]$ ln -s zookeeper-3.4.6 zookeeper

\[hadoop@slave1 opt\]$ ln -s zookeeper-3.4.6 zookeeper

\[hadoop@slave2 opt\]$ ln -s zookeeper-3.4.6 zookeeper

**6）修改环境变量**

On master

\[hadoop@master ~\]$ vi .bash\_profile

. ~/.bashrc

\# User specific environment and startup programs

PATH=$PATH:$HOME/bin

export PATH  
export JAVA\_HOME=/usr/java/default  
export PATH=$JAVA\_HOME/bin:$PATH  
export CLASSPATH=$CLASSPATH:$JAVA\_HOME/lib  
export LD\_LIBRARY\_PATH=$LD\_LIBRARY\_PATH:/usr/local/lib

\#Hadoop1.0  
\#export HADOOP1\_HOME=/home/hadoop/hadoop  
\#export PATH=$HADOOP1\_HOME/bin:$PATH  
\#export HADOOP\_CONF\_DIR=$\{HADOOP1\_HOME\}/conf

\#Hadoop2.0  
export HADOOP2\_HOME=/opt/hadoop  
export HADOOP\_CONF\_DIR=$\{HADOOP2\_HOME\}/etc/hadoop  
export HADOOP\_MAPRED\_HOME=$\{HADOOP2\_HOME\}  
export YARN\_CONF\_DIR=$\{HADOOP2\_HOME\}/etc/hadoop  
export HADOOP\_YARN\_HOME=$\{HADOOP2\_HOME\}  
export HADOOP\_COMMON\_HOME=$\{HADOOP2\_HOME\}  
export HADOOP\_HDFS\_HOME=$\{HADOOP2\_HOME\}  
export HDFS\_CONF\_DIR=$\{HADOOP2\_HOME\}/etc/hadoop  
export PATH=$HADOOP2\_HOME/bin:$HADOOP2\_HOME/sbin:$PATH

\#Ant  
export ANT\_HOME=/home/hadoop/apache-ant-1.8.0  
export PATH=$ANT\_HOME/bin:$PATH

\#zookeeper  
export ZOOKEEPER\_HOME=/opt/zookeeper  
export PATH=$ZOOKEEPER\_HOME/bin:$PATH

**7）同步环境变量**

\[hadoop@master ~\]$ scp -r .bash\_profile hadoop04:~

\[hadoop@master ~\]$ scp -r .bash\_profile slave1:~

\[hadoop@master ~\]$ scp -r .bash\_profile slave2:~

**8）在hadoop04、slave1、slave2创建目录**

\[hadoop@hadoop04 ~\]$ mkdir -p /opt/zookeeper/data

\[hadoop@master ~\]$ slaves.sh mkdir -p /opt/ zookeeper /data    《===相当于在slave1和slave2节点执行了mkdir命令

小技巧：使用slaves.sh 可以简单的在slave节点同意执行命令

**9）创建myid文件**

\[hadoop@hadoop04 ~\]$ cd /opt/ zookeeper /data/  
\[hadoop@hadoop04 data\]$ echo 1 > myid

\[hadoop@slave1 ~\]$ cd /opt/ zookeeper /data/

\[hadoop@slave1 data\]$ echo 2 > myid

\[hadoop@slave2 hadoop\]$ cd /opt/ zookeeper /data/

\[hadoop@slave2 data\]$ echo 3 > myid

**10） 启动zookeeper**  
\[hadoop@hadoop04 ~\]$ zkServer.sh start

\[hadoop@slave1 ~\]$ zkServer.sh start

\[hadoop@slave2 ~\]$ zkServer.sh start

**11）修改zoo.cfg，添加一条日志记录**   《===只是为了报错有日志生成，可以不加

\[hadoop@master conf\]$ vi zoo.cfg

dataLogDir=/opt/zookeeper/log

**12）停掉所有其他服务**

On master

\[hadoop@master hadoop\]$ hadoop-daemons.sh --hostnames 'slave1 slave2 hadoop04' stop journalnode

\[hadoop@master hadoop\]$ hadoop-daemons.sh stop datanode

\[hadoop@master hadoop\]$ hadoop-daemons.sh --hostnames 'hadoop04 master'stop namenode

**13）打开自动切换模式**

\[hadoop@master ~\]$ cd /opt/hadoop/etc/hadoop/

\[hadoop@master hadoop\]$ vi hdfs-site.xml

<name>dfs.ha.automatic-failover.enabled</name>  
  <value>true</value>  
 </property>

\[hadoop@master hadoop\]$ vi core-site.xml

<property>  
  <name>ha.zookeeper.quorum</name>  
  <value>hadoop04:2181,slave1:2181,slave2:2181</value>  
 </property>

将这两个文件拷贝到hadoop04服务器  
\[hadoop@master hadoop\]$ scp -r hdfs-site.xml hadoop04:/opt/hadoop/etc/hadoop/

\[hadoop@master hadoop\]$ scp -r core-site.xml hadoop04:/opt/hadoop/etc/hadoop/

**14）Zookeeper的初始化**

\[hadoop@master hadoop\]$ hdfs zkfc –formatZK

\[hadoop@hadoop04 data\]$ zkCli.sh

WatchedEvent state:SyncConnected type:None path:null

\[zk: localhost:2181(CONNECTED) 0\] ls /

\[hadoop-ha, zookeeper\]

\[zk: localhost:2181(CONNECTED) 1\] ls /hadoop-ha

\[ns1\]  
\[zk: localhost:2181(CONNECTED) 2\] get /hadoop-ha/ns1

cZxid = 0x100000003  
ctime = Tue Mar 21 02:40:13 EDT 2017  
mZxid = 0x100000003  
mtime = Tue Mar 21 02:40:13 EDT 2017  
pZxid = 0x100000003  
cversion = 0  
dataVersion = 0  
aclVersion = 0  
ephemeralOwner = 0x0  
dataLength = 0  
numChildren = 0  
\[zk: localhost:2181(CONNECTED) 3\]

**15）启动所有服务**

\[hadoop@master hadoop\]$ hadoop-daemons.sh --hostnames 'slave1 slave2 hadoop04' start journalnode

\[hadoop@master hadoop\]$ hadoop-daemons.sh --hostnames 'hadoop04 master' start namenode

\[hadoop@master hadoop\]$ hadoop-daemons.sh start datanode

**16）在两个namenode节点启动zkfc**

\[hadoop@master hadoop\]$ hadoop-daemons.sh --hostnames 'hadoop04 master' start zkfc

**17）**

on hadoop04

![20170321162430550][]

**on master**

![20170321162503426][]

**18）模拟测试将当前active状态的namenode节点kill掉**

\[hadoop@master hadoop\]$ jps  
3436 Jps  
2760 NameNode  
2952 DFSZKFailoverController  
\[hadoop@master hadoop\]$ kill -9 2760  
\[hadoop@master hadoop\]$

**19）**

**on hadoop04**

![20170321162643990][]

**on master**

![20170321162718418][]

切换成功

**20）创建目录**

\[hadoop@master hadoop\]$ hdfs dfs -mkdir hdfs://ns1/test  
\[hadoop@master ~\]$ hdfs dfs -put a.txt hdfs://ns1/test  
\[hadoop@master ~\]$ hdfs dfs -ls hdfs://ns1/test  
Found 1 items  
\-rw-r--r--   2 hadoop supergroup          6 2017-03-21 03:32 hdfs://ns1/test/a.txt  
\[hadoop@master ~\]$

由于我们在core-site.xml配置了  
<configuration>  
 <property>  
  <name>fs.defaultFS</name>  
  <value>hdfs://ns1</value>  
 </property>

因此，我们可以直接指定目录，不需要加hdfs://ns1这个前缀了

\[hadoop@master ~\]$ hdfs dfs -ls /test  
Found 1 items  
\-rw-r--r--   2 hadoop supergroup          6 2017-03-21 03:32 /test/a.txt

基于zk自动切换，环境搭建完成  
\---------------------   
作者：forever19870418   
来源：CSDN   
原文：https://blog.csdn.net/forever19870418/article/details/64442920   
版权声明：本文为博主原创文章，转载请附上博文链接！

[20170321162430550]: /images/20220126/ede99cc31eef46c0a88466a69aafbaf2.png
[20170321162503426]: /images/20220126/61f4f441e11843419a62cb2f261b8c35.png
[20170321162643990]: /images/20220126/a02a2b37766d4f6abf2d9f640876e3b7.png
[20170321162718418]: /images/20220126/844e6b27b4774d51abde2d8985dbd66d.png