Cloudera Manager平台部署说明书

水深无声 2022-05-14 10:18 266阅读 0赞

# 一、基础环境 #

### 1. 修改主机名 ###

\# vim /etc/sysconfig/network

NETWORKING=yes
    HOSTNAME=master1

\# vim /etc/sysconfig/network

NETWORKING=yes
    HOSTNAME=master2

\# vim /etc/sysconfig/network

NETWORKING=yes
    HOSTNAME=node1

\# vim /etc/sysconfig/network

NETWORKING=yes
    HOSTNAME=node2

\# vim /etc/sysconfig/network

NETWORKING=yes
    HOSTNAME=node3

### 2. 配置网络映射 ###

\# vim /etc/hosts

192.168.233.101     master1
    192.168.233.102     master2
    192.168.233.103     node1
    192.168.233.104     node2
    192.168.233.105     node3

### 3. 关闭防火墙 ###

\# service iptables stop  
    \# chkconfig iptables off  
    \# chkconfig --list | grep iptables

### 4. 禁用selinux ###

\# vim /etc/selinux/config

SELINUX=disabled

### 5. 禁用IPv6 ###

\# echo " " >> /etc/modprobe.d/dist.conf  
    \# echo "alias net-pf-10 off" >> /etc/modprobe.d/dist.conf  
    \# echo "alias ipv6 off" >> /etc/modprobe.d/dist.conf

### 6. 配置SSH无秘钥登录 ###

**6.1 生产公钥和私钥**

\# ssh-keygen

**6.2 拷贝公钥给目标服务器**

\# ssh-copy-id -i master1  
    \# ssh-copy-id -i master2  
    \# ssh-copy-id -i node1  
    \# ssh-copy-id -i node2  
    \# ssh-copy-id -i node3

### 7. 集群时间同步 ###

\# ntpdate -u time.nuri.net

### 8. 设置用户的可打开最大文件数及进程数 ###

\# vim /etc/security/limits.conf

*     soft     nofile     32728
    *     hard     nofile     1024567
    *     soft     nproc      65535
    *     hard     nproc      unlimited
    *     soft     memlock    unlimited
    *     hard     memlock    unlimited

--------------------

# 二、基础软件 #

### 1. JDK安装 ###

**1.1 下载JDK安装包**

*  本平台采用 JDK-8u172 的版本：jdk-8u172-linux-x64.tar.gz
 *  [http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html][http_www.oracle.com_technetwork_java_javase_downloads_jdk8-downloads-2133151.html]

**1.2 解压JDK安装程序**

\# mkdir -p /opt/java/  
    \# tar zxf /opt/tools/jdk-8u172-linux-x64.tar.gz -C /opt/java/

**1.3 拷贝JDK至其它节点**

\# scp -r java/ master2:/opt/  
    \# scp -r java/ node1:/opt/  
    \# scp -r java/ node2:/opt/  
    \# scp -r java/ node3:/opt/

**1.4 配置JDK环境变量**

vim /etc/profile

# JAVA_HOME
    export JAVA_HOME=/opt/java/jdk1.8.0_172
    export CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar
    export PATH=$PATH:$JAVA_HOME/bin

**1.5 验证JDK环境**

\# source /etc/profile  
    \# java -version

![70][]

### 2. MySQL安装 ###

*  [CentOS-6.9 二进制安装Mysql-5.7.12][CentOS-6.9 _Mysql-5.7.12]

--------------------

# 三、Cloudera Manager安装 #

### 1. 安装介质 ###

**1.1 下载Cloudera Manager**

*  本平台采用 Cloudera Manager-5.12.0的版本：cloudera-manager-el6-cm5.12.0\_x86\_64.tar.gz
 *  [http://archive.cloudera.com/cm5/cm/5/][http_archive.cloudera.com_cm5_cm_5]

**1.2 下载MySQL JDBC驱动**

*  本平台采用 MySQL-5.1.46的JDBC驱动：mysql-connector-java-5.1.46.jar
 *  [http://dev.mysql.com/downloads/connector/j/][http_dev.mysql.com_downloads_connector_j]

### 2. 部署准备 ###

**2.1 MySQL创建元数据库**

\# mysql -uroot -pmysqlpasswd

create database activity default charset utf8 collate utf8_general_ci;
    grant all on activity.* to 'cm'@'%' identified by 'cm!@#edc';
    flush privileges;

**2.2 创建cloudera-scm用户**

<table> 
 <thead> 
  <tr> 
   <th style="width:498px;"> <p>参数</p> </th> 
   <th style="width:351px;"> <p>参数说明</p> </th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td style="width:498px;"> <p>--system</p> </td> 
   <td style="width:351px;"> <p>表示系统用户</p> </td> 
  </tr> 
  <tr> 
   <td style="width:498px;"> <p>--home=/opt/cm-5.12.0/run/cloudera-scm-server&nbsp;--no-create-home</p> </td> 
   <td style="width:351px;"> <p>指定用户的主目录</p> </td> 
  </tr> 
  <tr> 
   <td style="width:498px;"> <p>--shell=/bin/false&nbsp;&nbsp;</p> </td> 
   <td colspan="1" style="width:351px;"> <p>不作为登陆用户</p> </td> 
  </tr> 
  <tr> 
   <td style="width:498px;"> <p>--comment&nbsp;"Cloudera&nbsp;SCM&nbsp;User"&nbsp;&nbsp;</p> </td> 
  </tr> 
  <tr> 
   <td style="width:498px;"> <p>cloudera-scm</p> </td> 
   <td style="width:351px;"> <p>用户名</p> </td> 
  </tr> 
 </tbody> 
</table>

useradd --system \
    --home=/opt/cm-5.12.0/run/cloudera-scm-server --no-create-home \
    --shell=/bin/false \
    --comment "Cloudera SCM User" cloudera-scm

### 3. 部署Cloudera Manager ###

**3.1 解压Cloudera Manager安装程序**

\# tar zxf /opt/tools/cloudera-manager-el6-cm5.12.0\_x86\_64.tar.gz -C /opt/

**3.2 初始化Cloudera Manager数据库**

*  拷贝MySQL驱动

\# cp /opt/tools/mysql-connector-java-5.1.46.jar /opt/cm-5.12.0/share/cmf/lib/

*  初始化Cloudera Manager数据库，创建的数据库名为cm，对应的用户名为cm，密码为cm!@\#edc

\# /opt/cm-5.12.0/share/cmf/schema/scm\_prepare\_database.sh mysql cm -hlocalhost -uroot -p'mysqlpasswd' cm 'cm!@\#edc'

**3.3 修改配置**

\# vim /opt/cm-5.12.0/etc/cloudera-scm-agent/config.ini

server_host=master1

**3.4 拷贝CM至其它节点**

\# scp -r /opt/cm-5.12.0 master2:/opt/  
    \# scp -r /opt/cm-5.12.0 node1:/opt/  
    \# scp -r /opt/cm-5.12.0 node2:/opt/  
    \# scp -r /opt/cm-5.12.0 node3:/opt/

### 4. 启动Cloudera Manager ###

**4.1 master1节点启动CM服务和代理**

\# /opt/cm-5.12.0/etc/init.d/cloudera-scm-server start  
    \# /opt/cm-5.12.0/etc/init.d/cloudera-scm-agent start

**4.2 其它节点启动CM代理**

\# /opt/cm-5.12.0/etc/init.d/cloudera-scm-agent start

### 5. Cloudera Manager控制台 ###

*  访问WEB UI：[http://192.168.233.101:7180][http_192.168.233.101_7180]
 *  用户名：admin
 *  密码：admin

![70 1][]

[http_www.oracle.com_technetwork_java_javase_downloads_jdk8-downloads-2133151.html]: http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
[70]: /images/20220514/63b031316cfb491db81732180ded731b.png
[CentOS-6.9 _Mysql-5.7.12]: https://blog.csdn.net/volitationLong/article/details/78362695
[http_archive.cloudera.com_cm5_cm_5]: http://archive.cloudera.com/cm5/cm/5/
[http_dev.mysql.com_downloads_connector_j]: http://dev.mysql.com/downloads/connector/j/
[http_192.168.233.101_7180]: http://192.168.233.101:7180
[70 1]: /images/20220514/3010dc1cc39d4682b48ab8b56eda69a9.png