hive中的UDAF的使用流程记载

分手后的思念是犯贱 2023-01-01 12:51 130阅读 0赞

### 概述 ###

本文是对\[1\]的整理和复现。

### 环境 ###

<table style="width:297px;"> 
 <tbody> 
  <tr> 
   <td style="width:102px;"><strong>组件</strong></td> 
   <td style="width:192px;"><strong>版本</strong></td> 
  </tr> 
  <tr> 
   <td style="width:102px;"><strong>Hadoop</strong></td> 
   <td style="width:192px;">3.1.2</td> 
  </tr> 
  <tr> 
   <td style="width:102px;"><strong>Zookeeper</strong></td> 
   <td style="width:192px;">3.6.0</td> 
  </tr> 
  <tr> 
   <td style="width:102px;"><strong>Mysql</strong></td> 
   <td style="width:192px;">8.0.22-0ubuntu0.20.04.2</td> 
  </tr> 
  <tr> 
   <td style="width:102px;"><strong>Hive</strong></td> 
   <td style="width:192px;">2.3.7</td> 
  </tr> 
 </tbody> 
</table>

### Hive准备工作 ###

以\[2\]中的例子三为准

zhong

### 注册UDAF流程 ###

<table> 
 <tbody> 
  <tr> 
   <td><strong>注册命令(hive/beeline中进行)</strong></td> 
   <td><strong>备注</strong></td> 
  </tr> 
  <tr> 
   <td>add jar /home/appleyuchi/桌面/Flink_Code/FLINK读写各种数据源/Java/target/table_api-1.0-SNAPSHOT.jar;</td> 
   <td>指明自定义依赖包</td> 
  </tr> 
  <tr> 
   <td>create temporary function sum_test as 'UDAFSum_Sample';</td> 
   <td>依赖包中的类UDAFSum_Sample注册为sum_test</td> 
  </tr> 
 </tbody> 
</table>

### 使用GenericUDF流程 ###

<table style="width:683px;"> 
 <tbody> 
  <tr> 
   <td style="width:451px;"><strong>hive命令</strong></td> 
   <td style="width:145px;"><strong>实验结果</strong></td> 
   <td style="width:84px;"><strong>备注</strong></td> 
  </tr> 
  <tr> 
   <td style="width:451px;"> <p>use db1;</p> <p>select sum_test(sales_info_new.sku_id) from sales_info_new;<br> &nbsp;</p> </td> 
   <td style="width:145px;">+---------+<br> | &nbsp; _c0 &nbsp; |<br> +---------+<br> | 7165.0 &nbsp;|<br> +---------+<br> &nbsp;</td> 
   <td style="width:84px;">无</td> 
  </tr> 
 </tbody> 
</table>

**完整代码:**

[https://gitee.com/appleyuchi/Flink\_Code/blob/master/FLINK读写各种数据源/Java/src/main/java/UDAFSum\_Sample.java][https_gitee.com_appleyuchi_Flink_Code_blob_master_FLINK_Java_src_main_java_UDAFSum_Sample.java]

**Reference:**

\[1\][UDAF实现一个自定义的求和函数][UDAF]

\[2\][hive中操作struct与map三例][hive_struct_map]

[https_gitee.com_appleyuchi_Flink_Code_blob_master_FLINK_Java_src_main_java_UDAFSum_Sample.java]: https://gitee.com/appleyuchi/Flink_Code/blob/master/FLINK%E8%AF%BB%E5%86%99%E5%90%84%E7%A7%8D%E6%95%B0%E6%8D%AE%E6%BA%90/Java/src/main/java/UDAFSum_Sample.java
[UDAF]: https://www.iteye.com/blog/goaheadtw-1745359
[hive_struct_map]: https://yuchi.blog.csdn.net/article/details/112104594