elasticsearch的javaAPI简单使用

柔光的暖阳◎ 2022-10-01 05:47 239阅读 0赞

最近引入es库，存放一些日志类的数据，对数据进行一些简单的查询操作。下面是简单的使用讲解。（随笔记录）

es官方api地址：[https://www.elastic.co/guide/en/elasticsearch/client/java-api/6.0/client.html][https_www.elastic.co_guide_en_elasticsearch_client_java-api_6.0_client.html]

众所周知es的java调用方式有三种，一种官方都要放弃的TransportClient，一种LowLevelJavaRestClient（API比较完整的）以下简称LL，一种HighLevel的一下简称HL，我这就简单的使用的了HL的方式。

1、导入依赖

<dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>elasticsearch-rest-high-level-client</artifactId>
        <version>6.0.1</version>
    </dependency>

2、认证（如果es库引入认证的组件，如果没有认证组件直接newRestHightLevelClient传RestClientBuilder即可）

在HL的API中没有关于认证的部分。参考了LL。

RestClientBuilder rcb = RestClient.builder(new HttpHost(ES_HOSTNAME, ES_PORT, ES_PROTOCOL));
            final CredentialsProvider provider = new BasicCredentialsProvider();
            provider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(ES_NAME, ES_PASSWORD));
            rcb.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
                @Override
                public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                    httpClientBuilder.disableAuthCaching();
                    return httpClientBuilder.setDefaultCredentialsProvider(provider);
                }
            });
            final RestHighLevelClient client = new RestHighLevelClient(rcb);

3、封装查询searchRequest类，先指定要查询的index与type(相当于mysql中的库和表属性)

SearchRequest search = new SearchRequest("flow_log");
    search.types("flow_log");

4、构建SearchSourceBuilder（可以配置检索的大部分基本属性from，size、sort或超时等属性），之后就可以执行查询操作了

SearchSourceBuilder sourceBuilder = new SearchSourceBuilder（）; 
    //sourceBuilder.query（QueryBuilders.termQuery（"xx"，"xx"））; 
    //searchSourceBuilder.aggregation(aggregation);//聚合查询
    sourceBuilder.from（0）; 
    sourceBuilder.size（5）; 
    sourceBuilder.sort("xxx", SortOrder.DESC);
    sourceBuilder.timeout（new TimeValue（60，TimeUnit.SECONDS））; 
    searchRequest.source（sourceBuilder）;
    SearchResponse searchResponse = client.search(searchRequest);//执行查询操作

5、针对SearchSourceBuilder中的query可以是QueryBuilder的任意类型。以下有几种常用的QueryBuilder。

QueryBuilder：常用的查询条件，类似mysql中（select \* from xxx where xx=xx，select \* from xxx where xx in \[xx\]）

QueryBuilders.termQuery（"xx"，"xx"）
    QueryBuilders.termsQuery("xxx", array)

BoolQueryBuilder：组合查询（filter,must,should等）

BoolQueryBuilder query = QueryBuilders.boolQuery();//相当于两条语句同时满足
    query.must(QueryBuilders.termsQuery("xx", xxx));
    query.must(QueryBuilders.termsQuery("xx", xxx));
    
    BoolQueryBuilder query = QueryBuilders.boolQuery();//相当于满足两种
    query.filter(QueryBuilders.termsQuery("xx", xxx));
    query.filter(QueryBuilders.termsQuery("xx", xxx));
    
    BoolQueryBuilder query = QueryBuilders.boolQuery();//相当于满足一种即可
    query.should(QueryBuilders.termsQuery("xx", xxx));
    query.should(QueryBuilders.termsQuery("xx", xxx));

MatchQueryBuilder：构造查询（模糊查询等）

MatchQueryBuilder matchQueryBuilder = new MatchQueryBuilder("xx", "xx");
    matchQueryBuilder.fuzziness(Fuzziness.AUTO); //启用模糊查询
    matchQueryBuilder.prefixLength(3); //前缀长度
    matchQueryBuilder.maxExpansions(10); //最大扩展

6、AggregationBuilders：聚合查询（group by ，sum、max、min、avg等操作）

AggregationBuilders.terms("groupName").field("xxx");//已xxx字段groupby后组名groupName
    AggregationBuilders.sum("sumName").field("xxx");//已xxx字段求和后名称sumName
    AggregationBuilders.subAggregation（AggregationBuilders.xxx）//可以继续封装AggregationBuilders

7、对于查询结果的解析。（普通查询内容在hits中，聚合结果在buckets中）

SearchHits hits = searchResponse.getHits();
    for (SearchHit hit : hits) {
        Map<String, Object> sourceAsMap = hit.getSourceAsMap();
        String xx=sourceAsMap.get("xx").toString()
    }
    System.out.println("+++++++++++查询结果数：" + searchResponse.getHits().getTotalHits());

8、对于聚合查询的解析。

单层的聚合：

Terms groupByProtocol = searchResponse.getAggregations().get("groupName");
            List<? extends Terms.Bucket> buckets = groupByProtocol.getBuckets();
            ArrayList<Map<String, Object>> data = new ArrayList<>();
            for (Terms.Bucket bucket : buckets) {
                Map<String, Object> dataMap = new HashMap<>();
                String key = bucket.getKey().toString();
                long docCount = bucket.getDocCount();
                dataMap.put("type", key);
                dataMap.put("value", docCount);
                data.add(dataMap);
            }

多层的聚合：

Aggregations aggregations = searchResponse.getAggregations();
            Terms groupBySrcIp = aggregations.get("groupName1");
            List<? extends Terms.Bucket> buckets = groupBySrcIp.getBuckets();
            for (Terms.Bucket bucket : buckets) {
                String xx=bucket.getKey();
                int count=(int) bucket.getDocCount()
    
                Terms groupByDstIp = bucket.getAggregations().get("groupName2");
                List<? extends Terms.Bucket> buckets1 = groupByDstIp.getBuckets();
                for (Terms.Bucket bucket1 : buckets1) {
                    String xx=bucket1.getKey();
                    int count=(int) bucket1.getDocCount()
                }
            }

以上就是本人使用es的简单记录，还有一个小窍门就是在kibana的界面中，基本上你能实现的dsl语句基本上在javaAPI中都是可以补齐方法找到的，也是刚接触es，了解的只是皮毛，有问题欢迎随时指出。祝大家学习愉快！

[https_www.elastic.co_guide_en_elasticsearch_client_java-api_6.0_client.html]: https://www.elastic.co/guide/en/elasticsearch/client/java-api/6.0/client.html