SparkBWA编译失败报错解决

落日映苍穹つ 2022-12-05 15:11 248阅读 0赞

## sparkbwa编译失败 ##

### 内网情况下失败遇到的问题 ###

1.maven镜像源获取不到，配置外网代理进行解决  
2.需要修改文件  
（1）在/src/main/native/Makefile.common文件中定义JAVA\_HOME环境变量。  
（2）BWA版本的Makefile中包含标志-fPIC。需要将此选项添加到BWA Makefile中CFLAGS变量的末尾即可。考虑到bwa-0.7.15，原始Makefile包含：  
CFLAGS= -g -Wall -Wno-unused-function -O2  
更改后应该是：  
CFLAGS= -g -Wall -Wno-unused-function -O2 -fPIC

### 外网情况下失败遇到的问题 ###

（1）报错一：jre rather than jdk。 解决方法：删除原来的jdk进行了重新配置后，source环境变量后解决了。  
（2）报错二：maven访问不到镜像地址，部分包无法下载。解决方法：更换了maven阿里源镜像，如下：

<mirror>
              <id>nexus-aliyun</id>
              <mirrorOf>central</mirrorOf>
              <name>Nexus aliyun</name>
              <url>http://maven.aliyun.com/nexus/content/groups/public</url> 
          </mirror>

（3）报错三：编译失败，错误如下：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RhbmRlbGlu_size_16_color_FFFFFF_t_70_pic_center]通过查找定位，发现是maven在打包过程中出现了包的编译失败造成，简言之是bwa包编译失败造成，回顾bwa包编译过程需要安装，gnu，gcc等系列软件，而此处定位报错的内容是：**error： zlib.h:no such file or directory**，而apt-get install zlib1g-dev命令在centos上无法使用，只能使用yum,直接进行yum -y install gcc将报错，因此采取，先升级gcc,再用yum下载zlib.h。

**注:环境使用的是centos7**,具体操作如下：

296  gcc -v
      297   yum -y install gcc
      298  yum install zlib-devel

最终重新进行编译mvn的sparkbwa，成功通过如下：

(base) [root@localhost SparkBWA]# mvn package
    [INFO] Scanning for projects...
    [INFO] 
    [INFO] --------------------< com.github.sparkbwa:SparkBWA >--------------------
    [INFO] Building SparkBWA 0.2
    [INFO] --------------------------------[ jar ]---------------------------------
    [WARNING] The POM for commons-codec:commons-codec:jar:1.15-SNAPSHOT is missing, no dependency information available
    [INFO] 
    [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ SparkBWA ---
    [INFO] Using 'UTF-8' encoding to copy filtered resources.
    [INFO] Copying 1 resource
    [INFO] 
    [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ SparkBWA ---
    [INFO] Nothing to compile - all classes are up to date
    [INFO] 
    [INFO] --- exec-maven-plugin:1.5.0:exec (makeBwa) @ SparkBWA ---
    
    Downloaded from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/apache/maven/maven-core/3.0/maven-core-3.0.jar (527 kB at 305 kB/s)
    Downloading from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/sonatype/aether/aether-api/1.7/aether-api-1.7.jar
    Downloaded from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/sonatype/aether/aether-spi/1.7/aether-spi-1.7.jar (14 kB at 7.3 kB/s)
    Downloading from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/sonatype/aether/aether-util/1.7/aether-util-1.7.jar
    Downloaded from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/apache/maven/maven-model-builder/3.0/maven-model-builder-3.0.jar (148 kB at 77 kB/s)
    Downloading from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/codehaus/plexus/plexus-interpolation/1.14/plexus-interpolation-1.14.jar
    Downloaded from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/apache/maven/maven-aether-provider/3.0/maven-aether-provider-3.0.jar (51 kB at 26 kB/s)
    Downloading from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/codehaus/plexus/plexus-classworlds/2.2.3/plexus-classworlds-2.2.3.jar
    Downloaded from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/sonatype/aether/aether-impl/1.7/aether-impl-1.7.jar (106 kB at 54 kB/s)
    Downloading from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/apache/maven/maven-compat/3.0/maven-compat-3.0.jar
    Downloaded from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/sonatype/aether/aether-api/1.7/aether-api-1.7.jar (74 kB at 35 kB/s)
    Downloading from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/apache/maven/wagon/wagon-provider-api/1.0-beta-6/wagon-provider-api-1.0-beta-6.jar
    Downloaded from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/sonatype/aether/aether-util/1.7/aether-util-1.7.jar (108 kB at 44 kB/s)
    Downloading from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/apache/maven/maven-artifact/3.0/maven-artifact-3.0.jar
    Downloaded from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/codehaus/plexus/plexus-classworlds/2.2.3/plexus-classworlds-2.2.3.jar (46 kB at 19 kB/s)
    Downloading from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/codehaus/plexus/plexus-utils/3.0.22/plexus-utils-3.0.22.jar
    Downloaded from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/codehaus/plexus/plexus-interpolation/1.14/plexus-interpolation-1.14.jar (61 kB at 25 kB/s)
    Downloading from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/ow2/asm/asm/5.0.2/asm-5.0.2.jar
    Downloaded from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/apache/maven/wagon/wagon-provider-api/1.0-beta-6/wagon-provider-api-1.0-beta-6.jar (53 kB at 21 kB/s)
    Downloading from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/ow2/asm/asm-commons/5.0.2/asm-commons-5.0.2.jar
    Downloaded from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/apache/maven/maven-compat/3.0/maven-compat-3.0.jar (285 kB at 104 kB/s)
    Downloading from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/ow2/asm/asm-tree/5.0.2/asm-tree-5.0.2.jar
    Downloaded from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/apache/maven/maven-artifact/3.0/maven-artifact-3.0.jar (52 kB at 18 kB/s)
    Downloading from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/jdom/jdom/1.1/jdom-1.1.jar
    Downloaded from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/ow2/asm/asm/5.0.2/asm-5.0.2.jar (53 kB at 17 kB/s)
    Downloading from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/apache/maven/shared/maven-dependency-tree/2.2/maven-dependency-tree-2.2.jar
    Downloaded from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/ow2/asm/asm-tree/5.0.2/asm-tree-5.0.2.jar (29 kB at 9.2 kB/s)
    Downloaded from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/org/ow2/asm/asm-commons/5.0.2/asm-commons-5.0.2.jar (42 kB at 13 kB/s)
    Downloading from alimaven: http://maven.aliyun.com/nexus/content/repositories/central/commons-io/commons-io/2.2/commons-io-2.2.jar
    
    [INFO] Excluding org.glassfish.jersey.containers:jersey-container-servlet:jar:2.22.2 from the shaded jar.
    [INFO] Excluding org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.22.2 from the shaded jar.
    [INFO] Excluding io.netty:netty-all:jar:4.1.47.Final from the shaded jar.
    [INFO] Excluding io.netty:netty:jar:3.9.9.Final from the shaded jar.
    [INFO] Excluding com.clearspring.analytics:stream:jar:2.7.0 from the shaded jar.
    [INFO] Excluding io.dropwizard.metrics:metrics-core:jar:3.1.5 from the shaded jar.
    [INFO] Excluding io.dropwizard.metrics:metrics-jvm:jar:3.1.5 from the shaded jar.
    [INFO] Excluding io.dropwizard.metrics:metrics-json:jar:3.1.5 from the shaded jar.
    [INFO] Excluding io.dropwizard.metrics:metrics-graphite:jar:3.1.5 from the shaded jar.
    [INFO] Excluding com.fasterxml.jackson.core:jackson-databind:jar:2.6.7.3 from the shaded jar.
    [INFO] Excluding com.fasterxml.jackson.module:jackson-module-scala_2.11:jar:2.6.7.1 from the shaded jar.
    [INFO] Excluding org.scala-lang:scala-reflect:jar:2.11.8 from the shaded jar.
    [INFO] Excluding com.fasterxml.jackson.module:jackson-module-paranamer:jar:2.7.9 from the shaded jar.
    [INFO] Excluding org.apache.ivy:ivy:jar:2.4.0 from the shaded jar.
    [INFO] Excluding oro:oro:jar:2.0.8 from the shaded jar.
    [INFO] Excluding net.razorvine:pyrolite:jar:4.13 from the shaded jar.
    [INFO] Excluding net.sf.py4j:py4j:jar:0.10.7 from the shaded jar.
    [INFO] Excluding org.apache.spark:spark-tags_2.11:jar:2.4.6 from the shaded jar.
    [INFO] Excluding org.apache.commons:commons-crypto:jar:1.0.0 from the shaded jar.
    [INFO] Excluding org.spark-project.spark:unused:jar:1.0.0 from the shaded jar.
    [INFO] Including cz.adamh.utils:native-utils:jar:1.0-SNAPSHOT in the shaded jar.
    [INFO] Replacing original artifact with shaded artifact.
    [INFO] Replacing /home/SparkBWA/target/SparkBWA-0.2.jar with /home/SparkBWA/target/SparkBWA-0.2-shaded.jar
    [INFO] Dependency-reduced POM written at: /home/SparkBWA/dependency-reduced-pom.xml
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time:  02:02 min
    [INFO] Finished at: 2020-09-10T10:32:46-04:00
    [INFO] ------------------------------------------------------------------------
    (base) [root@localhost SparkBWA]# cd target/
    (base) [root@localhost target]# ls
    classes  maven-archiver  maven-status  original-SparkBWA-0.2.jar  SparkBWA-0.2.jar

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RhbmRlbGlu_size_16_color_FFFFFF_t_70_pic_center 1]

最终完美解决！

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RhbmRlbGlu_size_16_color_FFFFFF_t_70_pic_center]: /images/20221123/34579c4658fb42f68d596b04a611dc2c.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3RhbmRlbGlu_size_16_color_FFFFFF_t_70_pic_center 1]: /images/20221123/46ec00cc34a74abe9ac29aa294425de5.png