elasticsearch学习（2）- elasticsearch的高级操作

r囧r小猫 2022-09-14 11:19 251阅读 0赞

## 1. 批量（Bulk）操作 ##

Bulk批量操作是将文档的增删改查一系列操作，通过一次请求全部做完，减少网络传输次数。  
语法：

POST /_bulk
    { 
    	"action":{ "metadata"}
    	{ "data"}
    }

举例：

GET person/_search
    
    # 批量操作
    # 1 删除4号记录
    # 2 修改xMqiWnwBLL2snwrIBtEv号记录，name改成张三二号
    # 3 添加3号记录
    POST person/_bulk
    { "delete":{  "_index":"person","_id":4 } } 
    {  "create":{  "_index":"person","_id":3 } } 
    {  "name":"三号", "age":23, "addr":"北京昌平区"  } 
    {  "update":{  "_index":"person", "_id":"xMqiWnwBLL2snwrIBtEv" } } 
    {  "doc":{  "name":"张三二号" } }

注意：中间不能有换行，所有的操作之间相互独立，相互不受影响。

## 2. 各种查询 ##

### 2.1 matchAll查询 ###

GET person/_search
    { 
      "query": { 
        "match_all": { }
      }
    }

注意：ES 存在默认分页，默认一次性查询10条数据  
如果想指定分页，如下：

GET person/_search
    { 
      "query": { 
        "match_all": { }
      },
      "from": 1,
      "size": 2
    }

from表示从哪一条数据开始，size表示显示条数

### 2.2 term查询 ###

# term查询中，本质其实是对词条的查询，keyword字段直接可以进行全量查询，将其视为一个词条，text字段是根据词条进行查询，不会全量匹配
    GET person3/_search
    { 
      "query": { 
        "term": { 
          "addr": { 
            "value": "北京海淀区"
          }
        }
      }
    }

批量查询

GET person3/_search
    { 
      "query": { 
        "terms": { 
          "name": [
            "张三",
            "李四"
          ]
        }
      }
    }

### 2.3 match查询 ###

会对查询条件进行分词，然后将分词后的查询条件和词条进行等值匹配，默认取并集。

GET person/_search
    { 
      "query": { 
        "match": { 
          "addr": "华为鸿蒙系统"
        }
      }
    }

### 2.4 模糊查询 ###

wildcard查询：会对查询条件进行分词，还可以使用通配符?（任意单个字符）和\*（0个或多个字符）。  
注意：尽量不要在前面写通配符，否则就会全部索引扫描。

# 模糊查询 *多个匹配 ?匹配一个
    GET person/_search
    { 
      "query": { 
        "wildcard": { 
          "addr": { 
            "value": "*华"
          }
        }
      }
    }

regexp查询 ：正则查询。

GET person/_search
    { 
      "query": { 
        "regexp": { 
          "addr": "\\w+(.)*"
        }
      }
    }

prefix查询：前缀查询

# 前缀查询
    GET person/_search
    { 
      "query": { 
        "prefix": { 
          "addr": { 
            "value": "三"
          }
        }
      }
    }

### 2.5 范围查询 ###

range 范围查询：查找指定字段在指定范围内包含值。

# 范围查询 gte>= lte<= gt> lt<
    GET person/_search
    { 
      "query": { 
        
        "range": { 
          "age": { 
            "gte": 10,
            "lte": 20
          }
        }
      }
    }

排序

GET person/_search
    { 
      "query": { 
        
        "range": { 
          "age": { 
            "gte": 10,
            "lte": 20
          }
        }
      },
      "sort": [
        { 
          "age": { 
            "order": "desc"
          }
        }
      ]
    }

### 2.6 queryString查询 ###

有时候需要根据其他字段进行多字段查询，可以用到它。

*  会对查询条件进行分词。
 *  然后将分词后的查询条件和词条进行等值匹配。
 *  默认取并集。
 *  可以指定多个查询字段。

# queryString查询 首先对你的条件进行分词 然后一个字段一个字段取匹配
    # or 表示的是在一个字段中 只要有一个条件满足就ok AND表示在一个字段中必须两个查询条件都满足才可以
    GET person/_search
    { 
      "query": { 
        
        "query_string": { 
          "fields": ["name","addr"],
          "query": "张三 OR 三星"
        }
      }
    }

### 2.7 布尔查询 ###

boolQuery: 对多个查询条件连接，连接方式：

*  must(and)：条件必须成立。
 *  must\_not(not)：条件必须不成立。
 *  should(or)：条件可以成立。
 *  filter：条件必须成立，性能比must高，不会计算得分。

# 布尔查询 must
    GET person/_search
    { 
      "query": { 
        "bool": { 
          "must": [
            { 
              "term": { 
                "name": { 
                  "value": "赵七"
                }
              }
            },
            { 
              "match": { 
                "addr": "三星"
              }
            }
          ]
        }
      }
    }
    
    # 布尔查询 filter
    GET person/_search
    { 
      "query": { 
        "bool": { 
          "filter": [
            { 
              "term": { 
                "name": { 
                  "value": "赵七"
                }
              }
            }
          ]
        }
      }
    }

### 2.8 聚合查询 ###

*  指标聚合：相当于Mysql的聚合函数，max，min，avg，sum等。
 *  桶聚合：相当于Mysql的group by操作。

注意：不要对text类型的数据进行group by，否则会失败。

# 指标聚合 聚合函数
    # 对查询的结果进行聚合
    # max_age 给聚合结果起个名字，max要聚合的算法 field要聚合的字段
    GET person/_search
    { 
      "query": { 
        "match": { 
          "addr": "北京"
        }
      },
      "aggs": { 
        "max_age": { 
          "max": { 
            "field": "age"
          }
        }
      }
    }
    
    # 桶聚合 分组
    # 对查询结果进行分组
    # age_group 聚合别名 terms要根据指定的字段进行分组，总共显示size条数
    
    GET person/_search
    { 
      "query": { 
        "match": { 
          "addr": "北京"
        }
      },
      "aggs": { 
        "age_group": { 
          "terms": { 
            "field": "age",
            "size": 100
          }
        }
      }
    }

### 2.9 高亮查询 ###

高亮三要素：高亮字段，高亮前缀，高亮后缀。

# 高亮查询
    # 后续可能需要高亮字段将原有字段替换
    GET person/_search
    { 
      "query": { 
        "match": { 
          "addr": "北京"
        }
      },
      "highlight": { 
        
        "fields": { 
          "addr": { 
            "pre_tags": "<font color ='red'>"
            , "post_tags": "</font>"
          }
        }
      }
    }

## 3. 索引别名和重建索引 ##

随着业务需求变更，索引的结构可能会发生变化。但是ES的索引一旦创建，只允许添加字段，不允许改变字段，因为需要重建倒排索引，影响内部缓存结构，性能太低。  
那么此时，就需要重新创建一个新的索引，并将原有的数据导入到新索引中。

#--------重建索引----------
    # 新建person1，注意：索引名称字母必须全部小写
    # 现在需求是将person的name字段类型由keyword改成text 直接无法修改，只能创建person1，然后将person数据导入到person1中。
    PUT person1
    { 
      "mappings": { 
        "properties": { 
          "name":{ 
            "type": "text",
            "analyzer": "ik_max_word"
          },
          "addr":{ 
            "type": "text",
            "analyzer": "ik_max_word"
          }
        }
      }
    }
     
    # 将person数据拷贝到person1
    # _reindex
    POST _reindex
    { 
      "source": { 
        "index": "person"
      },
      "dest": { 
        "index": "person1"
      }
    }

但是代码中仍然使用的是老的索引，我们的办法是要么修改代码，要么给索引起别名。

# 给person1起别名为person，前提是不存在person这个索引
    POST person1/_alias/person

## 4. java API操作 ##

在这里 我把所有各种复杂的查询都进行了尝试，详情如下：

@SpringBootTest
    class ElasticsearchdemoApplicationTests { 
    
    
        @Resource
        private RestHighLevelClient client;
    
        /** * 1. 手动创建索引，并指定字段以及字段所使用的分词器 */
        @Test
        void createIndex() throws Exception { 
            IndicesClient indices = client.indices();
            CreateIndexRequest createIndexRequest = new CreateIndexRequest("goods2");
    // createIndexRequest.alias(new Alias("goods1_copy"));//设置别名
            createIndexRequest.mapping(getMapping());
            CreateIndexResponse createIndexResponse = indices.create(createIndexRequest, RequestOptions.DEFAULT);
            System.out.println(createIndexResponse.toString());
        }
    
        public XContentBuilder getMapping() throws Exception{ 
    
            return XContentFactory.jsonBuilder()
                    .startObject()
                    .startObject("properties")
                    .startObject("type").field("type", "keyword").endObject()
                    .startObject("name")
                    .field("type", "text").field("analyzer", "ik_max_word")
                    .endObject()
                    .startObject("manufacturer").field("type", "keyword").endObject()
                    .startObject("desc")
                    .field("type", "text").field("analyzer", "ik_max_word")
                    .endObject()
                    .startObject("price").field("type", "float").endObject()
                    .startObject("id").field("type", "long").endObject()
                    .endObject()
                    .endObject();
        }
    
        /** * 2. 保存1条（默认就创建索引） 完毕 */
        @Test
        void saveOne() throws IOException { 
    
            IndexRequest indexRequest =new IndexRequest("goods");
            indexRequest.id("1");
            Goods goods = new Goods(1,"电脑", "联想电脑 thinkPad ", "联想", "国产之光", new BigDecimal(6600));
            indexRequest.source(JSONObject.toJSONString(goods),XContentType.JSON);
            IndexResponse index = client.index(indexRequest, RequestOptions.DEFAULT);
            System.out.println(index.toString());
        }
    
        /** * 3. 批量保存数据 完毕 */
        @Test
        void bulkSave() throws IOException { 
    
            BulkRequest bulkRequest = new BulkRequest("goods2");
            List<Goods> goodsList = new ArrayList<Goods>(){ { 
                add(new Goods(1,"电脑","华为电脑 鸿蒙1 ","华为","国产之光",new BigDecimal("6600.45")));
                add(new Goods(2,"电脑","小米电脑 战神","小米","国产之光",new BigDecimal("13000.7")));
                add(new Goods(3,"电脑","华为电脑 鸿蒙2","华为","国产之光",new BigDecimal("15000.5")));
                add(new Goods(4,"电脑","华为电脑 鸿蒙3","华为","国产之光",new BigDecimal("16000.1")));
                add(new Goods(5,"手环","康佳手环","康佳","国产之光",new BigDecimal(700)));
                add(new Goods(6,"手环","小米手环","小米","国产之光",new BigDecimal(500)));
            }};
    
            for (Goods goods : goodsList) { 
                bulkRequest.add(new IndexRequest().source(JSONObject.toJSONString(goods),XContentType.JSON)); 
            }
            BulkResponse bulk = client.bulk(bulkRequest, RequestOptions.DEFAULT);
            System.out.println(bulk.status());
    
        }
    
        /** * 根据Id查询 */
        @Test
        void findById() throws IOException { 
    
            GetRequest getRequest = new GetRequest("goods");
            getRequest.id("1");
            GetResponse getResponse = client.get(getRequest, RequestOptions.DEFAULT);
            System.out.println(getResponse.getSourceAsString());
    
        }
    
        /** * 根据Id批量查询(可以使用term批量查询) */
        @Test
        void findByIds() throws IOException { 
    
            List<String> ids = new ArrayList<String>(){ { 
    
                add("zJMZX3wBr9fydOl2wxVp");
                add("zZMZX3wBr9fydOl2wxVp");
            }};
            SearchRequest searchRequest = new SearchRequest("goods");
            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
            searchSourceBuilder.query(QueryBuilders.termsQuery("_id",ids));
    
            searchRequest.source(searchSourceBuilder);
            SearchResponse search = client.search(searchRequest, RequestOptions.DEFAULT);
            for (SearchHit hit : search.getHits()) { 
                System.out.println(hit.getSourceAsString());
            }
    
        }
    
        /** * 根据id删除文档 */
        @Test
        void deleteById() throws IOException { 
    
            DeleteRequest deleteRequest = new DeleteRequest("goods");
            deleteRequest.id("1");
            client.delete(deleteRequest,RequestOptions.DEFAULT);
        }
    
        /** * 根据ID修改 */
        @Test
        void updateById() throws IOException { 
    
            UpdateRequest updateRequest = new UpdateRequest("goods2","3JMgbXwBr9fydOl2SRXi");
            Goods goods = new Goods(5,"手环", "康佳手环 电脑控制 ", "康佳", "国产之光棒棒哒 电脑控制手环", new BigDecimal(6600));
            updateRequest.doc(JSONObject.toJSONString(goods),XContentType.JSON);
            UpdateResponse update = client.update(updateRequest, RequestOptions.DEFAULT);
            System.out.println(update);
        }
    
        /** * 根据ID批量更新 */
        @Test
        void updateByIds() throws IOException { 
    
            List<Goods> goodsList = new ArrayList<Goods>(){ { 
                add(new Goods(1,"笔记本电脑","华为电脑 鸿蒙1 ","华为","国产之光11",new BigDecimal("6600.45")));
                add(new Goods(2,"笔记本电脑","小米电脑 战神","小米","国产之光11",new BigDecimal("13000.7")));
                add(new Goods(3,"笔记本电脑","华为电脑 鸿蒙2","华为","国产之光11",new BigDecimal("15000.5")));
                add(new Goods(4,"笔记本电脑","华为电脑 鸿蒙3","华为","国产之光11",new BigDecimal("16000.1")));
                add(new Goods(5,"笔记手环","康佳手环","康佳","国产之光11",new BigDecimal(700)));
                add(new Goods(6,"笔记手环","小米手环","小米","国产之光11",new BigDecimal(500)));
            }};
    
            //首先根据ID值进行查询
            SearchRequest searchRequest = new SearchRequest("goods2");
    
            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
            List<String> ids = goodsList.stream().map(Goods::getId).map(String::valueOf).collect(Collectors.toList());
            searchSourceBuilder.query(QueryBuilders.termsQuery("id",ids));
            searchRequest.source(searchSourceBuilder);
            SearchResponse search = client.search(searchRequest, RequestOptions.DEFAULT);
    
            List<Goods> esGoodsList = new ArrayList<>();
            for (SearchHit hit : search.getHits()) { 
                Goods goods = JSONObject.parseObject(hit.getSourceAsString(), Goods.class);
                esGoodsList.add(goods);
            }
            for (Goods esGoods : esGoodsList) { 
                for (Goods goods : goodsList) { 
                    if(esGoods.getId() == goods.getId()){ 
                        BeanUtils.copyProperties(goods,esGoods);
                    }
                }
            }
    
        }
    
        /** *分页查询 */
        @Test
        void selectByPage() throws IOException { 
    
            SearchRequest searchRequest = new SearchRequest("goods");
            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
            searchSourceBuilder.query(QueryBuilders.matchAllQuery());
            searchSourceBuilder.from(0); //从第几个开始 不包含这个
            searchSourceBuilder.size(3); //查询个数 从第0个不算，往后数三个
            searchRequest.source(searchSourceBuilder);
            SearchResponse search = client.search(searchRequest, RequestOptions.DEFAULT);
            for (SearchHit hit : search.getHits()) { 
                System.out.println(hit.getSourceAsString());
            }
        }
    
        /** * 模糊查询 */
        @Test
        void selectByWildcard() throws IOException { 
    
            //模糊查询只能针对keyword, text类型字段
            SearchRequest searchRequest = new SearchRequest("goods2");
            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
            searchSourceBuilder.query(QueryBuilders.wildcardQuery("name","*鸿蒙*"));
            searchRequest.source(searchSourceBuilder);
            SearchResponse search = client.search(searchRequest, RequestOptions.DEFAULT);
            for (SearchHit hit : search.getHits()) { 
                System.out.println(hit.getSourceAsString());
            }
        }
    
        /** * 范围查询和排序 */
        @Test
        void selectByRange() throws IOException { 
    
            SearchRequest searchRequest = new SearchRequest("goods2");
            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
            searchSourceBuilder.query(QueryBuilders.rangeQuery("price").gte(100).lte(19000));
            searchSourceBuilder.sort("price", SortOrder.DESC);
            searchRequest.source(searchSourceBuilder);
            SearchResponse search = client.search(searchRequest, RequestOptions.DEFAULT);
            for (SearchHit hit : search.getHits()) { 
                System.out.println(hit.getSourceAsString());
            }
        }
    
        /** * queryString查询 */
        @Test
        void selectByQueryString() throws IOException { 
    
            SearchRequest searchRequest = new SearchRequest("goods2");
            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    
            // 根据值去每个字段进行模糊查询 +代表必须含有 -代表不能含有
            // queryString查询 首先对你的条件进行分词 然后一个字段一个字段取匹配
            // OR 表示的是在一个字段中 只要有一个条件满足就ok AND表示在一个字段中必须两个查询条件都满足才可以
            // OR和AND必须大写
            searchSourceBuilder.query(QueryBuilders.queryStringQuery("(+电脑 -华为) OR 手环").field("name").field("type"));
            searchRequest.source(searchSourceBuilder);
            SearchResponse search = client.search(searchRequest, RequestOptions.DEFAULT);
            for (SearchHit hit : search.getHits()) { 
                System.out.println(hit.getSourceAsString());
            }
        }
    
        /** * 布尔查询 * 筛选出名称中包含鸿蒙或者鸿蒙同时价格在100-16000之外的 */
        @Test
        void selectByBoolQuery() throws IOException { 
    
            SearchRequest searchRequest = new SearchRequest("goods2");
            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    
            searchSourceBuilder.query(QueryBuilders
                    .boolQuery()
                        .should(QueryBuilders.termQuery("name","鸿蒙"))
                        .should(QueryBuilders.termQuery("name","康佳"))
                        .mustNot(QueryBuilders.rangeQuery("price").gte(100).lte(16000)));
            searchRequest.source(searchSourceBuilder);
            SearchResponse search = client.search(searchRequest, RequestOptions.DEFAULT);
            for (SearchHit hit : search.getHits()) { 
                System.out.println(hit.getSourceAsString());
            }
        }
    
        /** * 聚合查询 * */
        @Test
        void selectBysign() throws IOException { 
    
            SearchRequest searchRequest = new SearchRequest("goods2");
            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    
            searchSourceBuilder.query(QueryBuilders.termQuery("name","鸿蒙"));
            AvgAggregationBuilder avgAggregationBuilder = AggregationBuilders.avg("totalPrice").field("price");
            searchSourceBuilder.aggregation(avgAggregationBuilder);
    
            searchRequest.source(searchSourceBuilder);
            SearchResponse search = client.search(searchRequest, RequestOptions.DEFAULT);
            for (SearchHit hit : search.getHits()) { 
                System.out.println(hit.getSourceAsString());
            }
    
            //下面获得聚合查询
            Avg avg = search.getAggregations().get("totalPrice");
            System.out.println(avg.getValue());
        }
    
        /** * 桶聚合查询 * */
        @Test
        void selectByGroupBy() throws IOException { 
    
            SearchRequest searchRequest = new SearchRequest("goods2");
            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    
            searchSourceBuilder.query(QueryBuilders.termQuery("name","鸿蒙"));
            TermsAggregationBuilder termsBuilder = AggregationBuilders.terms("groupPrice").field("price");
            searchSourceBuilder.aggregation(termsBuilder);
            searchRequest.source(searchSourceBuilder);
            SearchResponse search = client.search(searchRequest, RequestOptions.DEFAULT);
            for (SearchHit hit : search.getHits()) { 
                System.out.println(hit.getSourceAsString());
            }
    
            //下面获得聚合查询
            Aggregations aggregations = search.getAggregations();
            Terms terms = aggregations.get("groupPrice");
            for (Terms.Bucket bucket : terms.getBuckets()) { 
                System.out.println(bucket.getKeyAsString() +" : "+ bucket.getDocCount());
            }
        }
    }