hadoop2.9.2源码编译

迈不过友情╰ 2023-02-16 12:00 13阅读 0赞

1、环境准备

centos7 、maven-3.6.1、jdk1.8.0、apache-ant-1.9.9 、 protobuf-2.5.0、 hadoop-2.9.2-src

将maven-3.6.1、jdk1.8.0、apache-ant-1.9.9 、 protobuf-2.5.0的压缩包下载好，解压到指定的目录其中protobuf-2.5.0需要重新 编译

安装gcc编译器和make构建，cmake使用3以上的版本：

yum install glibc-headers
    yum install gcc-c++
    yum install make
    yum install cmake

编译protobuf

./configure 
    make 
    make check 
    make install 
    ldconfig

安装openssl库

yum install openssl-devel
    yum install ncurses-devel

将上述编译工具添加到环境变量中

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlbmdjaGVuZ3d1MjAxMg_size_16_color_FFFFFF_t_70][]

解压hadoop源码，进入源码目录，执行编译命令

mvn package -Pdist,native -DskipTests -Dtar

编译可能遇到的问题![20200609174247902.png][]

解决：下载亚马逊jar包，安装到本地maven仓库

wget https://s3-us-west-2.amazonaws.com/dynamodb-local/release/com/amazonaws/DynamoDBLocal/1.11.86/DynamoDBLocal-1.11.86.jar
    
    
    mvn install:install-file -Dfile=DynamoDBLocal-1.11.86.jar -DgroupId=com.amazonaws -DartifactId=DynamoDBLocal -Dversion=1.11.86 -Dpackaging=jar

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlbmdjaGVuZ3d1MjAxMg_size_16_color_FFFFFF_t_70 1][]

编译后的hadoop压缩包位置

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlbmdjaGVuZ3d1MjAxMg_size_16_color_FFFFFF_t_70 2][]

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlbmdjaGVuZ3d1MjAxMg_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20200609173739504.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlbmdjaGVuZ3d1MjAxMg==,size_16,color_FFFFFF,t_70
[20200609174247902.png]: https://img-blog.csdnimg.cn/20200609174247902.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlbmdjaGVuZ3d1MjAxMg_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20200610105311525.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlbmdjaGVuZ3d1MjAxMg==,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlbmdjaGVuZ3d1MjAxMg_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/20200610110429101.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlbmdjaGVuZ3d1MjAxMg==,size_16,color_FFFFFF,t_70