Linux部署oozie任务调度框架

墨蓝 2022-12-27 02:13 233阅读 0赞

修改版资料

链接：https://pan.baidu.com/s/1KWvkBG-QNYNBdyZkV42X9w

提取码：06x6

复制这段内容后打开百度网盘手机App，操作更方便哦

一:Oozie的部署

1:上传的目录

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz_size_16_color_FFFFFF_t_70][]

2:解压安装

1:解压安装包

tar -zxvf oozie-4.0.0-cdh5.3.6.tar.gz -C /training/oozie/

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz_size_16_color_FFFFFF_t_70 1][]

2:在oozie根目录下解压oozie-hadooplibs-4.0.0-cdh5.3.6.tar.gz

tar -zxvf oozie-hadooplibs-4.0.0-cdh5.3.6.tar.gz -C ../

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz_size_16_color_FFFFFF_t_70 2][]

3:在oozie的目录下创建libext目录

mdkir libext/

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz_size_16_color_FFFFFF_t_70 3][]

创建目录后,这个时候,就需要拷贝我们所需要的jar包了

1: 将hadooplibs里面的jar包，拷贝到libext目录下

cp -ra hadooplibs/hadooplib-2.5.0-cdh5.3.6.oozie-4.0.0-cdh5.3.6/\* libext/

![20201207161210684.png][]

2:拷贝mysql驱动包到libext目录下

cp -a /tools/oozie/mysql-connector-java-5.1.27-bin.jar ./libext/

![20201207161222527.png][]

3:将ext-2.2.zip拷贝到libext/目录下

ext是一个js框架，用于展示oozie前端页面：

cp -a /tools/oozie/ext-2.2.zip libext/

3:修改oozie配置文件

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz_size_16_color_FFFFFF_t_70 4][]

/training/oozie/oozie-4.0.0-cdh5.3.6/conf/oozie-site.xml

1:修改jdbc驱动(142行)

<name>oozie.service.JPAService.jdbc.driver</name>

<value>com.mysql.jdbc.Driver</value>

JDBC driver class.

</description>

</property>

2:配置数据库地址(150行)

<name>oozie.service.JPAService.jdbc.url</name>

<value>jdbc:mysql://server2:3306/oozie</value>

JDBC URL.

</description>

</property>

3:配置数据库用户名(158)

<name>oozie.service.JPAService.jdbc.username</name>

DB user name.

</description>

</property>

4:配置数据库密码(166)

<name>oozie.service.JPAService.jdbc.password</name>

DB user password.

IMPORTANT: if password is emtpy leave a 1 space string, the service trims the value,

if empty Configuration assumes it is NULL.

</description>

</property>

5:让oozie引用hadoop的配置文件

<name>oozie.service.HadoopAccessorService.hadoop.configurations</name>

<value>\*=/training/oozie/cdh/hadoop-2.5.0-cdh5.3.6/etc/hadoop</value>

Comma separated AUTHORITY=HADOOP\_CONF\_DIR, where AUTHORITY is the HOST:PORT of

the Hadoop service (JobTracker, HDFS). The wildcard '\*' configuration is

used when there is no exact match for an authority. The HADOOP\_CONF\_DIR contains

the relevant Hadoop \*-site.xml files. If the path is relative is looked within

the Oozie configuration directory; though the path can be absolute (i.e. to point

to Hadoop client conf/ directories in the local filesystem.

</description>

</property>

4:在mysql中创建oozie的数据库

mysql -uroot -p123456

create database oozie;

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz_size_16_color_FFFFFF_t_70 5][]

5:初始化oozie

1:上传Oozie目录下的yarn.tar.gz文件到HDFS

yarn.tar.gz文件会自行解压

注意: 需要启动cdh版本的hadoop

进入目录: cd /training/oozie/oozie-4.0.0-cdh5.3.6/ 执行下面命令

bin/oozie-setup.sh sharelib create -fs hdfs://server2:8020 -locallib oozie-sharelib-4.0.0-cdh5.3.6-yarn.tar.gz

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz_size_16_color_FFFFFF_t_70 6][]

注意: 执行成功后,去web(地址是server2:50070)上检查对应的目录是否有文件生成.

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz_size_16_color_FFFFFF_t_70 7][]

2:创建oozie.sql文件

bin/ooziedb.sh create -sqlfile oozie.sql -run

3:打包项目,生成war包

\[root@server2 oozie-4.0.0-cdh5.3.6\]\# bin/oozie-setup.sh prepare-war

6:oozie的启动和关闭

1:oozie的启动

bin/oozied.sh start

2:oozie的关闭

bin/oozied.sh stop

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz_size_16_color_FFFFFF_t_70 8][]

7:访问oozie的web页面

[http://server2:11000/oozie][http_server2_11000_oozie]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz_size_16_color_FFFFFF_t_70 9][]

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz_size_16_color_FFFFFF_t_70]: /images/20221120/935ab24f010f44ec857335c263ba4f02.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz_size_16_color_FFFFFF_t_70 1]: /images/20221120/53ca8811f22c4fd38c4ca492ed880920.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz_size_16_color_FFFFFF_t_70 2]: /images/20221120/45d025e3a63d4ea1b96655c25cd0bfc5.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz_size_16_color_FFFFFF_t_70 3]: /images/20221120/737a6496e7c34cb5b88c5bb3159c0b09.png
[20201207161210684.png]: /images/20221120/c18f0b2f5eb04f7580750abef6ce6940.png
[20201207161222527.png]: /images/20221120/db8127cd43924c5790f958ad9f00b0a0.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz_size_16_color_FFFFFF_t_70 4]: /images/20221120/43b5c937380344c9aea943da39cb6251.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz_size_16_color_FFFFFF_t_70 5]: https://img-blog.csdnimg.cn/20201207161247847.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz_size_16_color_FFFFFF_t_70 6]: https://img-blog.csdnimg.cn/20201207161259244.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz_size_16_color_FFFFFF_t_70 7]: https://img-blog.csdnimg.cn/20201207161309913.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz_size_16_color_FFFFFF_t_70 8]: https://img-blog.csdnimg.cn/20201207161408735.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz,size_16,color_FFFFFF,t_70
[http_server2_11000_oozie]: http://server2:11000/oozie
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz_size_16_color_FFFFFF_t_70 9]: https://img-blog.csdnimg.cn/20201207161422148.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI2OTYzNDMz,size_16,color_FFFFFF,t_70