DataX源码编译遇到的坑

阳光穿透心脏的1/2处 2022-10-04 13:58 797阅读 0赞

### 问题1 对Record的引用不明确 ###

> \[294,12\] 错误: 对Record的引用不明确  
> \[ERROR\] com.alibaba.datax.common.element 中的接口 com.alibaba.datax.common.element.Record 和 java.lang 中的类 java.lang.Record 都匹配  
> \[ERROR\] G:\\lantai\\DataX- master/core/src/main/java/com/alibaba/datax/core/transport/transformer/FilterTransformer.java:\[303,29\] 错误: 对Record的引用不明确 \[ERROR\] com.alibaba.datax.common.element 中的接口 com.alibaba.datax.common.element.Record 和 java.lang 中的类 java.lang.Record 都匹配  
> \[ERROR\] G:\\lantai\\DataX- master\\DataX/core/src/main/java/com/alibaba/datax/core/transport/transformer/FilterTransformer.java:\[303,12\] 错误: 对Record的引用不明确 \[ERROR\] -> \[Help 1\]

`检查下jdk版本是否是1.8的`

### 问题2 Failed to execute goal on project hdfsreader: Could not resolve dependencies for project com.alibaba.datax:hdfsreader:jar:0.0.1-SNAPSHOT: Failed to collect dependencies at org.apache.hive:hive-exec:jar:1.1.1 ###

> \[ERROR\] Failed to execute goal on project hdfsreader: Could not resolve dependencies for project com.alibaba.datax:hdfsreader:jar:0.0.1-SNAPSHOT: Failed to collect dependencies at org.apache.hive:hive-exec:jar:1.1.1 -> org.apache.ca lcite:calcite-core:jar:1.0.0-incubating -> org.apache.calcite:calcite-avatica:jar:1.0.0-incubating -> eigenbase:eigenbase-properties:jar:1.1.4: Failed to read artifact descriptor for eigenbase:eigenbase-properties:jar:1.1.4: Could n ot transfer artifact eigenbase:eigenbase-properties:pom:1.1.4 from/to maven-default-http-blocker (http://0.0.0.0/): Blocked mirror for repositories: \[datanucleus (http://www.datanucleus.org/downloads/maven2, default, releases), glas sfish-repository (http://maven.glassfish.org/content/groups/glassfish, default, disabled), glassfish-repo-archive (http://maven.glassfish.org/content/groups/glassfish, default, disabled), apache.snapshots (http://repository.apache.o rg/snapshots, default, snapshots), conjars (http://conjars.org/repo, default, releases+snapshots)\] -> \[Help 1\]

打开 maven 的配置文件(maven 安装目录的 `conf/settings.xml`)，在标签中添加 mirror 子节点:

<mirror>
      <id>aliyunmaven</id>
      <mirrorOf>*</mirrorOf>
      <name>阿里云公共仓库</name>
      <url>https://maven.aliyun.com/repository/public</url>
    </mirror>

### 问题3 Could not resolve dependencies for project com.alibaba.datax:hdfsreader:jar:0.0.1-SNAPSHOT: The following artifacts could not be resolved: org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde ###

> \[INFO\]------------------------------------------------------------------------  
> \[INFO\] BUILD FAILURE  
> \[INFO\]------------------------------------------------------------------------  
> \[INFO\] Total time: 01:47 min  
> \[INFO\] Finished at: 2021-07-15T15:18:46+08:00  
> \[INFO\] ------------------------------------------------------------------------  
> \[ERROR\] Failed to execute goal on project hdfsreader: Could not resolve dependencies for project com.alibaba.datax:hdfsreader:jar:0.0.1-SNAPSHOT: The following artifacts could not be resolved: org.pentaho:pentaho-aggdesigner-algorit hm:jar:5.1.5-jhyde: Could not find artifact org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde in aliyunmaven (https://maven.aliyun.com/repository/public) -> \[Help 1\]  
> \[ERROR\]

1.  手动下载pentaho-aggdesigner-algorithm相关pom及jar包  
    下载地址：[https://javalibs.com/artifact/org.pentaho/pentaho-aggdesigner-algorithm][https_javalibs.com_artifact_org.pentaho_pentaho-aggdesigner-algorithm]  
    将下载好的jar包放到本地仓库jar包对应目录下（比如我的本地仓库是）

// 本地仓库路径 
    C:\Users\EDZ\.m2\repository
    // 放jar包的路径 
    C:\Users\EDZ\.m2\repository\org\pentaho\pentaho-aggdesigner-algorithm\5.1.5-jhyde

1.  点击Download下面的pom  
    ![在这里插入图片描述][20210715175738854.png]
    
    新建一个名为pentaho-aggdesigner-algorithm-5.1.5-jhyde.pom的文件，将内容复制进去，保存  
    现在jar包所在目录如图  
    ![在这里插入图片描述][20210715173958152.png]

### 问题4 Could not find artifact eigenbase:eigenbase-properties:jar:1.1.4 ###

*  查看jar包所在仓库目录，删除图中选中的文件即可  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RhMTMxMjIzMTg4NjE_size_16_color_FFFFFF_t_70]
 *  清空文件目录，参照pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar包步骤手动下载也可以  
    `以上两种方式任意一种都可以`

### 问题5 Failed to execute goal on project oscarwriter: Could not resolve dependencies for project com.alibaba.datax:oscarwriter:jar:0.0.1-SNAPSHOT ###

> \[ERROR\] Failed to execute goal on project oscarwriter: Could not resolve dependencies for project com.alibaba.datax:oscarwriter:jar:0.0.1-SNAPSHOT: Could not find artifact com.oscar:oscar:jar:7.0.8 at specified path G:\\lantai\\DataX- master\\oscarwriter\\src\\main\\lib\\oscarJDBC.jar -> \[Help 1\] \[ERROR\]  
> \[ERROR\] To see the full stack trace of the errors, re-run Maven with the -e switch. \[ERROR\] Re-run Maven using the -X switch to enable full debug logging. \[ERROR\] \[ERROR\] For more information about the errors and possible solutions, please read the following articles:  
> \[ERROR\] \[Help 1\] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException \[ERROR\]  
> \[ERROR\] After correcting the problems, you can resume the build with the command  
> \[ERROR\] mvn < args > -rf :oscarwriter

`没用到这个可以注释掉，就不会打这个包了（后续更新）`

1.  修改项目根目录下的pom.xml  
    找到pom.xml，用文本编辑器打开文件，搜索oscar，将oscar有关的注释掉
2.  修改package.xml  
    搜索oscar，将oscar有关的注释掉

### 重新执行命令、成功 ###

mvn -U clean package assembly:assembly -Dmaven.test.skip=true

![在这里插入图片描述][20210715174541197.png]

[https_javalibs.com_artifact_org.pentaho_pentaho-aggdesigner-algorithm]: https://javalibs.com/artifact/org.pentaho/pentaho-aggdesigner-algorithm
[20210715175738854.png]: /images/20221004/f5c77f17a2ea47cfa79c56801ada1228.png
[20210715173958152.png]: /images/20221004/6032b4e7142f4938bc41180156dc56a5.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RhMTMxMjIzMTg4NjE_size_16_color_FFFFFF_t_70]: /images/20221004/3ca9b90f5dff4caea3a8f2975ae5883b.png
[20210715174541197.png]: /images/20221004/33d4a41fabca4f4ead302953c0a9d184.png