Java动态解决This limit can be set by changing the [index.max_result_window] index level setting.]]深度分页查询

港控/mmm° 2024-02-20 08:51 35阅读 0赞

### 问题描述 ###

[ElasticSearch][]官方默认限制索引查询最多只能查询10000条数据，查询第10001条数据开始就会报错：

> reason=Result window is too large, from + size must be less than or equal to: \[10000\] but was \[29740\]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the \[index.max\_result\_window\] index level setting.\]\];

![cee6d67b636149c88f0e12d519b9c3f3.png][]

### 原因分析 ###

> elasticsearch中`max_result_window`有上限限制：默认10000。

### 解决方案 ###

##### 第一种解决方案 #####

1.在请求时解除限制

设置查询最大上限20000

PUT \_all/\_settings

> \{  
> "index.max\_result\_window": 20000  
> \}

![a0aae8cf85ee46ab929518dc1e68e6ec.png][]

2.代码中解除限制并且设置最大返回值

在查询时候把 track\_total\_hits 设置为 true。

track\_total\_hits 设置为false禁用跟踪匹配查询的总点击次数

设置为true就会返回真实的命中条数。

> SearchSourceBuilder query = new SearchSourceBuilder();  
> query.trackTotalHits(true);  
> query.trackTotalHitsUpTo(20000);  
> query.query(integratedQueryBuilder(searchConditionJson));

3.在使用工具访问时添加track\_total\_hits设置

GET 索引名/_search
    {
      "query": {
        "match_all": {}
      },
      "track_total_hits": true
    }

##### 第二种解决方案 #####

在创建索引时添加最大上限设置

> \{  
> "settings": \{  
> "index": \{  
> "max\_result\_window": 20000  
> \}  
> \}  
> \}

使用这种方式代码撰写时也要和第一种一样在代码中限制保持一致 。

**Java动态处理动态限制：**

AI的回答，

![0ac91dbdb3c44cfabb8e22539c5de3d0.png][]

官网资料：[链接][Link 1]

![b3b9d76f5d0547bb9ca6dd6b12721d22.png][]

![9636c348e88f4674a76bf1e2d2c99390.png][]

引入依赖：

> <dependency>  
> <groupId>org.springframework.boot</groupId>  
> <artifactId>spring-boot-starter-data-elasticsearch</artifactId>  
> </dependency>

通过RestHighLevelClient，实现动态修改setting的值：

/**
         * 修改索引的settings
         * @param indices
         * @param from
         * @param size
         */
        private boolean updateIndex(String indices, int from, int size) {
            int records = from * size + size;
            if (records <= 10000) {
                return true;
            }
            UpdateSettingsRequest request = new UpdateSettingsRequest(indices);
            request.settings(Settings.builder().put("index.max_result_window", records));
            try {
                AcknowledgedResponse updateSettingsResponse = restHighLevelClient.indices().putSettings(request, RequestOptions.DEFAULT);
                return updateSettingsResponse.isAcknowledged();
            } catch (IOException e) {
                log.error("error ",e);
                throw new RuntimeException("索引修改setting异常",e);
            }
        }

![daa4af7d302f4fe7aa20ff9f0aac7a16.png][]

参考：[链接1][1]，[链接2][2]，[链接3][3]

[ElasticSearch]: https://so.csdn.net/so/search?q=ElasticSearch&spm=1001.2101.3001.7020
[cee6d67b636149c88f0e12d519b9c3f3.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/20/8f57223df0a846f194d96af8ac8585d0.png
[a0aae8cf85ee46ab929518dc1e68e6ec.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/20/5a01eabb149744c1a2d5b5279c2e1e91.png
[0ac91dbdb3c44cfabb8e22539c5de3d0.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/20/e72dcf869b1d4b90af0e2e6f11190fb3.png
[Link 1]: https://www.elastic.co/cn/search?q=UpdateSettingsRequest&size=n_20_n&filters%5B0%5D%5Bfield%5D=website_area&filters%5B0%5D%5Bvalues%5D%5B0%5D=documentation&filters%5B0%5D%5Btype%5D=all
[b3b9d76f5d0547bb9ca6dd6b12721d22.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/20/0aab4343f6a54fa78ec1dc5e8e5218dc.png
[9636c348e88f4674a76bf1e2d2c99390.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/20/7cdd843102a84ba1b94df7ce271bc2eb.png
[daa4af7d302f4fe7aa20ff9f0aac7a16.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/02/20/36a4dd74aad340d987687ab63580acb7.png
[1]: https://blog.csdn.net/chemphone/article/details/103508417?ops_request_misc=%257B%2522request%255Fid%2522%253A%2522169829174416800226598086%2522%252C%2522scm%2522%253A%252220140713.130102334.pc%255Fall.%2522%257D&request_id=169829174416800226598086&biz_id=0&utm_medium=distribute.pc_search_result.none-task-blog-2~all~first_rank_ecpm_v1~rank_v31_ecpm-1-103508417-null-null.142%5Ev96%5Epc_search_result_base4&utm_term=%E4%B8%ADUpdateSettingsRequest%E7%A4%BA%E4%BE%8B&spm=1018.2226.3001.4187
[2]: https://blog.csdn.net/miaomiao19971215/article/details/107662689?ops_request_misc=&request_id=&biz_id=102&utm_term=ElasticSearch%20%E8%AE%BE%E7%BD%AE10000%E9%99%90%E5%88%B6&utm_medium=distribute.pc_search_result.none-task-blog-2~all~sobaiduweb~default-0-107662689.142%5Ev96%5Epc_search_result_base4&spm=1018.2226.3001.4187
[3]: https://blog.csdn.net/jiapeng976548/article/details/119079762?ops_request_misc=%257B%2522request%255Fid%2522%253A%2522169828922116800182753331%2522%252C%2522scm%2522%253A%252220140713.130102334.pc%255Fall.%2522%257D&request_id=169828922116800182753331&biz_id=0&utm_medium=distribute.pc_search_result.none-task-blog-2~all~first_rank_ecpm_v1~rank_v31_ecpm-1-119079762-null-null.142%5Ev96%5Epc_search_result_base4&utm_term=java%E4%B8%ADElasticsearchRestTemplate%E5%8A%A8%E6%80%81%E7%B4%A2%E5%BC%95%E8%AE%BE%E7%BD%AEmax_result_window&spm=1018.2226.3001.4187