Java8 stream 中利用 groupingBy 进行多字段分组求和

绝地灬酷狼 2023-06-30 09:07 25阅读 0赞

> Java8的groupingBy实现集合的分组，类似Mysql的group by分组功能，**注意得到的是一个map**

## 对集合按照单个属性分组、分组计数、排序 ##

List<String> items =
            Arrays.asList("apple", "apple", "banana",
                    "apple", "orange", "banana", "papaya");
    
    // 分组
    Map<String, List<String>> result1 = items.stream().collect(
            Collectors.groupingBy(
                    Function.identity()
            )
    );
    //{papaya=[papaya], orange=[orange], banana=[banana, banana], apple=[apple, apple, apple]}
    System.out.println(result1);
    
    
    // 分组计数
    Map<String, Long> result2 = items.stream().collect(
            Collectors.groupingBy(
                    Function.identity(), Collectors.counting()
            )
    );
    // {papaya=1, orange=1, banana=2, apple=3}
    System.out.println(result2);
    Map<String, Long> finalMap = new LinkedHashMap<>();
    
    
    //分组, 计数和排序
    result2.entrySet().stream()
            .sorted(Map.Entry.<String, Long>comparingByValue().reversed())
            .forEachOrdered(e -> finalMap.put(e.getKey(), e.getValue()));
    // {apple=3, banana=2, papaya=1, orange=1}
    System.out.println(finalMap);

# 集合按照多个属性分组 #

## 1.多个属性拼接出一个组合属性 ##

public static void main(String[] args) {
        User user1 = new User("zhangsan", "beijing", 10);
        User user2 = new User("zhangsan", "beijing", 20);
        User user3 = new User("lisi", "shanghai", 30);
        List<User> list = new ArrayList<User>();
        list.add(user1);
        list.add(user2);
        list.add(user3);
        Map<String, List<User>> collect = list.stream().collect(Collectors.groupingBy(e -> fetchGroupKey(e)));
        //{zhangsan#beijing=[User{age=10, name='zhangsan', address='beijing'}, User{age=20, name='zhangsan', address='beijing'}], 
        // lisi#shanghai=[User{age=30, name='lisi', address='shanghai'}]}
        System.out.println(collect);
    }
    
    
    private static String fetchGroupKey(User user){
        return user.getName() +"#"+ user.getAddress();
    }

## 2.嵌套调用groupBy ##

User user1 = new User("zhangsan", "beijing", 10);
    User user2 = new User("zhangsan", "beijing", 20);
    User user3 = new User("lisi", "shanghai", 30);
    List<User> list = new ArrayList<User>();
    list.add(user1);
    list.add(user2);
    list.add(user3);
    Map<String, Map<String, List<User>>> collect
            = list.stream().collect(
                    Collectors.groupingBy(
                            User::getAddress, Collectors.groupingBy(User::getName)
                    )
    );
    System.out.println(collect);

## 3. 使用`Arrays.asList` ##

我有一个与Web访问记录相关的域对象列表。这些域对象可以扩展到数千个。  
我没有资源或需求将它们以原始格式存储在数据库中，因此我希望预先计算聚合并将聚合的数据放在数据库中。  
我需要聚合在5分钟窗口中传输的总字节数，如下面的sql查询

select 
      round(request_timestamp, '5') as window, --round timestamp to the nearest 5 minute
      cdn, 
      isp, 
      http_result_code, 
      transaction_time, 
      sum(bytes_transferred)
    from web_records
    group by 
        round(request_timestamp, '5'), 
        cdn, 
        isp, 
        http_result_code, 
        transaction_time

在java 8中，我当前的第一次尝试是这样的，我知道这个解决方案类似于[Group by multiple field names in java 8][]

Map<Date, Map<String, Map<String, Map<String, Map<String, Integer>>>>>>> aggregatedData =
    webRecords
        .stream()
        .collect(Collectors.groupingBy(WebRecord::getFiveMinuteWindow,
                   Collectors.groupingBy(WebRecord::getCdn,
                     Collectors.groupingBy(WebRecord::getIsp,
                       Collectors.groupingBy(WebRecord::getResultCode,
                           Collectors.groupingBy(WebRecord::getTxnTime,
                             Collectors.reducing(0,
                                                 WebRecord::getReqBytes(),
                                                 Integer::sum)))))));

这是可行的，但它是丑陋的，所有这些嵌套的地图是一个噩梦！要将地图“展平”或“展开”成行，我必须这样做

for (Date window : aggregatedData.keySet()) {
      for (String cdn : aggregatedData.get(window).keySet()) {
        for (String isp : aggregatedData.get(window).get(cdn).keySet()) {
          for (String resultCode : aggregatedData.get(window).get(cdn).get(isp).keySet()) {
            for (String txnTime : aggregatedData.get(window).get(cdn).get(isp).get(resultCode).keySet()) {
    
               Integer bytesTransferred = aggregatedData.get(window).get(cdn).get(distId).get(isp).get(resultCode).get(txnTime);
               AggregatedRow row = new AggregatedRow(window, cdn, distId...

如你所见，这是相当混乱和难以维持。  
有谁知道更好的方法吗？任何帮助都将不胜感激。  
我想知道是否有更好的方法来展开嵌套的映射，或者是否有一个库允许您对集合进行分组。

**最佳答案**

您应该为地图创建自定义密钥。最简单的方法是使用`Arrays.asList`：

Function<WebRecord, List<Object>> keyExtractor = wr ->
        Arrays.<Object>asList(wr.getFiveMinuteWindow(), wr.getCdn(), wr.getIsp(),
                 wr.getResultCode(), wr.getTxnTime());
    
    Map<List<Object>, Integer> aggregatedData = webRecords.stream().collect(
          Collectors.groupingBy(keyExtractor, Collectors.summingInt(WebRecord::getReqBytes)));

在这种情况下，键是按固定顺序列出的5个元素。不是很面向对象，但很简单。或者，您可以定义自己的表示自定义键的类型，并创建适当的`hashCode`/`equals`实现。

参考链接：

*  [https://stackoverflow.com/questions/32531517/][https_stackoverflow.com_questions_32531517]
 *  [https://www.cnblogs.com/winner-0715/p/10246037.html][https_www.cnblogs.com_winner-0715_p_10246037.html]

[Group by multiple field names in java 8]: https://stackoverflow.com/questions/28342814/group-by-multiple-field-names-in-java-8
[https_stackoverflow.com_questions_32531517]: https://stackoverflow.com/questions/32531517/
[https_www.cnblogs.com_winner-0715_p_10246037.html]: https://www.cnblogs.com/winner-0715/p/10246037.html