【Hadoop】Hadoop生态系列之搭建HadoopHA集群

野性酷女 2023-10-05 11:24 63阅读 0赞

上一篇：[Hadoop生态系列之MapReduce Shuffle概述及常见问题][Hadoop_MapReduce Shuffle]

#### 指路牌 ####

*  Hadoop HA构建
 *   *  概述
     *  准备工作
     *   *  主机和服务启动映射表
         *  主机信息
         *  JDK安装和配置
         *  IP主机名映射
         *  关闭防火墙
         *  SSH免密码认证
     *  Zookeeper服务的安装与配置
     *  搭建Hadoop 集群（HDFS）
     *   *  解压并配置HADOOP环境变量
         *  配置\`core-site.xml\`
         *  配置机架脚本
         *  配置\`hdfs-site.xml\`
         *  配置\`slaves\`
         *  启动HDFS(集群初始化启动)
         *  查看机架信息
     *  Resource Manager搭建
     *   *  配置\`yarn-site.xml\`
         *  配置\`mapred-site.xml\`
         *  启动 | 关闭\`Yarn\`服务

## Hadoop HA构建 ##

### 概述 ###

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ0NjY0MjMx_size_16_color_FFFFFF_t_70_pic_center]

### 准备工作 ###

*  至少需要CentOS7 x64三台，完成jdk的安装、SSH免密码认证、ip主机映射及关闭防火墙操作

> **详见：**[https://blog.csdn.net/qq\_44664231/article/details/108804561][https_blog.csdn.net_qq_44664231_article_details_108804561]

#### 主机和服务启动映射表 ####

<table> 
 <thead> 
  <tr> 
   <th>主机</th> 
   <th>服务</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>CentOSA</td> 
   <td>NameNode、zkfc、DataNode、JournalNode、Zookeeper、NodeManager</td> 
  </tr> 
  <tr> 
   <td>CentOSB</td> 
   <td>NameNode、zkfc、DataNode、JournalNode、Zookeeper、NodeManager、ResourceManager</td> 
  </tr> 
  <tr> 
   <td>CentOSC</td> 
   <td>DataNode、JournalNode、Zookeeper、NodeManager、ResourceManager</td> 
  </tr> 
 </tbody> 
</table>

#### 主机信息 ####

<table> 
 <thead> 
  <tr> 
   <th>主机名</th> 
   <th>IP信息</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>CentOSA</td> 
   <td>192.168.73.131</td> 
  </tr> 
  <tr> 
   <td>CentOSB</td> 
   <td>192.168.73.132</td> 
  </tr> 
  <tr> 
   <td>CentOSC</td> 
   <td>192.168.73.133</td> 
  </tr> 
 </tbody> 
</table>

#### JDK安装和配置 ####

[root@CentOSX ~]# rpm -ivh jdk-8u171-linux-x64.rpm
    [root@CentOSX ~]# vi .bashrc
    JAVA_HOME=/usr/java/latest
    PATH=$PATH:$JAVA_HOME/bin
    CLASSPATH=.
    export JAVA_HOME
    export CLASSPATH
    export PATH
    [root@CentOSX ~]# source .bashrc

#### IP主机名映射 ####

[root@CentOSX ~]# vi /etc/hosts
    
    192.168.253.130 CentOSA
    192.168.253.131 CentOSB
    192.168..132 CentOSC

#### 关闭防火墙 ####

[root@CentOSX ~]# systemctl stop firewalld
    [root@CentOSX ~]# systemctl disable firewalld
    Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
    Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
    [root@CentOSX ~]# firewall-cmd --state
    not running

#### SSH免密码认证 ####

[root@CentOSX ~]# ssh-keygen -t rsa
    [root@CentOSX ~]# ssh-copy-id CentOSA
    [root@CentOSX ~]# ssh-copy-id CentOSB
    [root@CentOSX ~]# ssh-copy-id CentOSC

### Zookeeper服务的安装与配置 ###

[root@CentOSX ~]# tar -zxf zookeeper-3.4.6.tar.gz -C /usr/
    [root@CentOSX ~]# mkdir /root/zkdata
    
    [root@CentOSA ~]# echo 1 >> /root/zkdata/myid
    [root@CentOSB ~]# echo 2 >> /root/zkdata/myid
    [root@CentOSC ~]# echo 3 >> /root/zkdata/myid
    
    [root@CentOSX ~]# touch /usr/zookeeper-3.4.6/conf/zoo.cfg
    [root@CentOSX ~]# vi /usr/zookeeper-3.4.6/conf/zoo.cfg
    tickTime=2000
    dataDir=/root/zkdata
    clientPort=2181
    initLimit=5
    syncLimit=2
    server.1=CentOSA:2887:3887
    server.2=CentOSB:2887:3887
    server.3=CentOSC:2887:3887
    
    [root@CentOSX ~]# /usr/zookeeper-3.4.6/bin/zkServer.sh start zoo.cfg
    [root@CentOSX ~]# /usr/zookeeper-3.4.6/bin/zkServer.sh status zoo.cfg
    JMX enabled by default
    Using config: /usr/zookeeper-3.4.6/bin/../conf/zoo.cfg
    Mode: `follower|leader`
    [root@CentOSX ~]# jps
    5879 `QuorumPeerMain`
    7423 Jps

### 搭建Hadoop 集群（HDFS） ###

#### 解压并配置HADOOP环境变量 ####

[root@CentOSX ~]# tar -zxf hadoop-2.9.2.tar.gz -C /usr/
    [root@CentOSX ~]# vi .bashrc
    HADOOP_HOME=/usr/hadoop-2.9.2
    JAVA_HOME=/usr/java/latest
    PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
    CLASSPATH=.
    export JAVA_HOME
    export CLASSPATH
    export PATH
    export HADOOP_HOME
    [root@CentOSX ~]# source .bashrc

#### 配置`core-site.xml` ####

<property>		
          <name>fs.defaultFS</name>		
          <value>hdfs://mycluster</value>	
    </property>
    <property>		
         <name>hadoop.tmp.dir</name>		
         <value>/usr/hadoop-2.9.2/hadoop-${user.name}</value>    
    </property>
    <property>		
         <name>fs.trash.interval</name>		
         <value>30</value>    
    </property>
    
    <property>		
         <name>net.topology.script.file.name</name>		
         <value>/usr/hadoop-2.9.2/etc/hadoop/rack.sh</value>    
    </property>
    
    <property>   
    	<name>ha.zookeeper.quorum</name>
    	<value>CentOSA:2181,CentOSB:2181,CentOSC:2181</value> 
    </property>
    
    <property>
         <name>dfs.ha.fencing.methods</name>
         <value>sshfence</value>
    </property>
    <property>
         <name>dfs.ha.fencing.ssh.private-key-files</name>
         <value>/root/.ssh/id_rsa</value>
    </property>

#### 配置机架脚本 ####

[root@CentOSX ~]# touch /usr/hadoop-2.9.2/etc/hadoop/rack.sh
    [root@CentOSX ~]# chmod u+x /usr/hadoop-2.9.2/etc/hadoop/rack.sh
    [root@CentOSX ~]# vi /usr/hadoop-2.9.2/etc/hadoop/rack.sh
    while [ $# -gt 0 ] ; do
    	  nodeArg=$1
    	  exec</usr/hadoop-2.9.2/etc/hadoop/topology.data
    	  result="" 
    	  while read line ; do
    		ar=( $line ) 
    		if [ "${ar[0]}" = "$nodeArg" ] ; then
    		  result="${ar[1]}"
    		fi
    	  done 
    	  shift 
    	  if [ -z "$result" ] ; then
    		echo -n "/default-rack"
    	  else
    		echo -n "$result "
    	  fi
    done
    [root@CentOSX ~]# touch /usr/hadoop-2.9.2/etc/hadoop/topology.data
    [root@CentOSX ~]# vi /usr/hadoop-2.9.2/etc/hadoop/topology.data
    192.168.253.130 /rack01
    192.168.253.131 /rack01
    192.168.253.132 /rack03

#### 配置`hdfs-site.xml` ####

<property>
    	<name>dfs.replication</name>
    	<value>3</value>
    </property> 
    
    <property>
    	<name>dfs.ha.automatic-failover.enabled</name>
    	<value>true</value>
    </property>
    
    <property>
    	<name>dfs.nameservices</name>
    	<value>mycluster</value>
    </property>
    <property>
    	<name>dfs.ha.namenodes.mycluster</name>
    	<value>nn1,nn2</value>
    </property>
    <property>
    	<name>dfs.namenode.rpc-address.mycluster.nn1</name>
    	<value>CentOSA:9000</value>
    </property>
    <property>
    	 <name>dfs.namenode.rpc-address.mycluster.nn2</name>
    	 <value>CentOSB:9000</value>
    </property>
    
    <property>
      <name>dfs.namenode.shared.edits.dir</name>
      <value>qjournal://CentOSA:8485;CentOSB:8485;CentOSC:8485/mycluster</value>
    </property>
    
    <property>
        <name>dfs.client.failover.proxy.provider.mycluster</name>	<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
    </property>

#### 配置`slaves` ####

CentOSA
    CentOSB
    CentOSC

> 这里因为是CentOS7,需要我们额外安装一个小插件`psmic`，否则NameNode无法实现故障自动切换。

[root@CentOSX ~]# yum install -y psmisc

#### 启动HDFS(集群初始化启动) ####

[root@CentOSX ~]# hadoop-daemon.sh start journalnode (等待10s钟)
    [root@CentOSA ~]# hdfs namenode -format
    [root@CentOSA ~]# hadoop-daemon.sh start namenode
    [root@CentOSB ~]# hdfs namenode -bootstrapStandby
    [root@CentOSB ~]# hadoop-daemon.sh start namenode
    #注册Namenode信息到zookeeper中,只需要在CentOSA或者B上任意一台执行一下指令
    [root@CentOSA|B ~]# hdfs zkfc -formatZK
    [root@CentOSA ~]# hadoop-daemon.sh start zkfc
    [root@CentOSB ~]# hadoop-daemon.sh start zkfc
    [root@CentOSX ~]# hadoop-daemon.sh start datanode

#### 查看机架信息 ####

[root@CentOSC ~]# hdfs dfsadmin -printTopology
    Rack: /rack01
       192.168.253.130:50010 (CentOSA)
       192.168.253.131:50010 (CentOSB)
    
    Rack: /rack03
       192.168.253.132:50010 (CentOSC)

### Resource Manager搭建 ###

#### 配置`yarn-site.xml` ####

<property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.resourcemanager.ha.enabled</name>
        <value>true</value>
    </property>
    <property>
        <name>yarn.resourcemanager.cluster-id</name>
        <value>cluster</value>
    </property>
    <property>
        <name>yarn.resourcemanager.ha.rm-ids</name>
        <value>rm1,rm2</value>
    </property>
    <property>
        <name>yarn.resourcemanager.hostname.rm1</name>
        <value>CentOSB</value>
    </property>
    <property>
        <name>yarn.resourcemanager.hostname.rm2</name>
        <value>CentOSC</value>
    </property>
    <property>
        <name>yarn.resourcemanager.zk-address</name>
        <value>CentOSA:2181,CentOSB:2181,CentOSC:2181</value>
    </property>
    
    <property>
        <name>yarn.nodemanager.pmem-check-enabled</name>
        <value>false</value>
    </property>
    
    <property>
        <name>yarn.nodemanager.vmem-check-enabled</name>
        <value>false</value>
    </property>

#### 配置`mapred-site.xml` ####

<property>
            <name>mapreduce.framework.name</name>
            <value>yarn</value>
    </property>

#### 启动 | 关闭`Yarn`服务 ####

[root@CentOSB ~]# yarn-daemon.sh start|stop resourcemanager
    [root@CentOSC ~]# yarn-daemon.sh start|stop resourcemanager
    [root@CentOSX ~]# yarn-daemon.sh start|stop nodemanager

--------------------

至此，Hadoop集群就已经搭建完毕，可以使用jps命令查看当前的java进程，如果进程如下图所示，即为环境搭建成功！  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ0NjY0MjMx_size_16_color_FFFFFF_t_70_pic_center 1]

[Hadoop_MapReduce Shuffle]: https://blog.csdn.net/qq_44664231/article/details/108890798
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ0NjY0MjMx_size_16_color_FFFFFF_t_70_pic_center]: https://img-blog.csdnimg.cn/20201009175445120.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ0NjY0MjMx,size_16,color_FFFFFF,t_70#pic_center
[https_blog.csdn.net_qq_44664231_article_details_108804561]: https://blog.csdn.net/qq_44664231/article/details/108804561
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ0NjY0MjMx_size_16_color_FFFFFF_t_70_pic_center 1]: https://img-blog.csdnimg.cn/20201009190548848.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ0NjY0MjMx,size_16,color_FFFFFF,t_70#pic_center