springboot快速整合elasticsearch+elasticsearch简介

灰太狼 2022-11-15 11:43 208阅读 0赞

# #

## 1、ElasticSearch简介 ##

我们的应用经常需要添加检索功能，开源的ElasticSearch是目前全文搜索引擎的首选。他可以快速存储、搜索和分析海量数据。Spring Boot通过整合Spring Data Elasticsearch为我们提供了非常便捷的检索功能支持；

ElasticSearch是一个分布式搜索服务，提供Restful API,底层基于Lucene，采用多shard（分片）的方式保证数据安全，并且提供自动resharding的功能，github等大型的站点也是采用了ElasticSearch作为其搜索服务。

## 2、ElasticSearch镜像的安装与启动 ##

docker search elasticsearch
    docker pull elasticsearch:7.11.1
    docker run -e ES_JAVA_OPTS="-Xms256m -Xmx256m" -e "discovery.type=single-node" -d -p 9200:9200 -p 9300:9300 --name es002 bc3d45eba361

因为ElasticSearch占用内存为2g，我的虚拟机不支持，设立了默认内存256MB和最大内存256MB

如果不加 -e "discovery.type=single-node" 会自动停止ElasticSearch容器。-d 后台运行 -p 虚拟机和容器进行端口映射 -p 多节点分布式端口映射。

### 如果还是自动停止容器：请按照以下步骤操作： ###

通过`docker ps -a`可以发现elasticsearch自动退出

通过`docker logs -f 容器id`查看日志，可以看到提示空间不足

使用命令`find / -name jvm.options`找到jvm.options文件位置

通过`vim 文件位置`命令进入该文件

找到xms参数位置，将分配空间修改小一些，减小值到系统可用的值，如

-Xms512m
    -Xmx512m

有可能报**max\_map\_count \[65530\] is too low**之类的错误，需要重置max\_map\_count大小

如`sysctl -w vm.max_map_count=262144`

查看max\_map\_count

`sysctl -a|grep vm.max_map_count`

然后重新启动容器

ps.

容器启动失败后，需要进入`docker ps -a`并记下容器id

然后通过`docker rm 容器id`删除当前容器，才可重新启动

pps.

如果是通过**阿里云/腾讯云**等云服务器的docker部署，千万不要忘记开放端口号！！！！！！！

此时，在物理机浏览器访问虚拟机端口：[http://192.168.44.128:9200/][http_192.168.44.128_9200],返回json，证明启动成功

{
      "name" : "ee56c20ae504",
      "cluster_name" : "docker-cluster",
      "cluster_uuid" : "lO4zw-yiQxC6MMv9pDE9FQ",
      "version" : {
        "number" : "7.11.1",
        "build_flavor" : "default",
        "build_type" : "docker",
        "build_hash" : "ff17057114c2199c9c1bbecc727003a907c0db7a",
        "build_date" : "2021-02-15T13:44:09.394032Z",
        "build_snapshot" : false,
        "lucene_version" : "8.7.0",
        "minimum_wire_compatibility_version" : "6.8.0",
        "minimum_index_compatibility_version" : "6.0.0-beta1"
      },
      "tagline" : "You Know, for Search"
    }

##  通过官方文档操作elasticsearch ##

[https://www.elastic.co/guide/cn/elasticsearch/guide/current/\_indexing\_employee\_documents.html][https_www.elastic.co_guide_cn_elasticsearch_guide_current_indexing_employee_documents.html]

PUT /megacorp/employee/1
    {
        "first_name" : "John",
        "last_name" :  "Smith",
        "age" :        25,
        "about" :      "I love to go rock climbing",
        "interests": [ "sports", "music" ]
    }
    PUT /megacorp/employee/2
    {
        "first_name" :  "Jane",
        "last_name" :   "Smith",
        "age" :         32,
        "about" :       "I like to collect rock albums",
        "interests":  [ "music" ]
    }
    
    PUT /megacorp/employee/3
    {
        "first_name" :  "Douglas",
        "last_name" :   "Fir",
        "age" :         35,
        "about":        "I like to build cabinets",
        "interests":  [ "forestry" ]
    }
    GET /megacorp/employee/1
    GET /megacorp/employee/_search
    GET /megacorp/employee/_search?q=last_name:Smith
    
    
    GET /megacorp/employee/_search
    {
        "query" : {
            "match" : {
                "last_name" : "Smith"
            }
        }
    }
    
    
    GET /megacorp/employee/_search
    {
        "query" : {
            "bool": {
                "must": {
                    "match" : {
                        "last_name" : "smith" 
                    }
                },
                "filter": {
                    "range" : {
                        "age" : { "gt" : 30 } 
                    }
                }
            }
        }
    }
    //全文搜索
    GET /megacorp/employee/_search
    {
        "query" : {
            "match" : {
                "about" : "rock climbing"
            }
        }
    }
    
    
    //短语搜索
    GET /megacorp/employee/_search
    {
        "query" : {
            "match_phrase" : {
                "about" : "rock climbing"
            }
        }
    }
    //高亮所搜
    GET /megacorp/employee/_search
    {
        "query" : {
            "match_phrase" : {
                "about" : "rock climbing"
            }
        },
        "highlight": {
            "fields" : {
                "about" : {}
            }
        }
    }

ElasticSearch的数据结构是

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BzaGRoeA_size_16_color_FFFFFF_t_70][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BzaGRoeA_size_16_color_FFFFFF_t_70 1][]

# springboot整合ElasticSearch和REST搜索 #

### 以前是jest，现在是rest ，autoconfig包里边： ###

springBoot 2.3.0版本及以后版本不支持es查询工具jestClient自动注入

![20210331212846847.png][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BzaGRoeA_size_16_color_FFFFFF_t_70 2][]

springboot默认有两种方式来和ElasticSearch交互

1、jest默认不生效，因为没有引入pom依赖

2、springdata ElasticSearch 默认生效，但是springdata ElasticSearch版本得和ElasticSearch容器版本匹配

client节点信息，clusterNodes clusterName

ElasticsearchTemplate操作es

编写一个ElasticSearchRepository的子接口来操作es

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
        <modelVersion>4.0.0</modelVersion>
        <parent>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-parent</artifactId>
            <version>2.4.4</version>
            <relativePath/> 
        </parent>
        <groupId>com.pshdhx.ElasticSearch</groupId>
        <artifactId>demo</artifactId>
        <version>0.0.1-SNAPSHOT</version>
        <name>demo</name>
        <description>Demo project for Spring Boot</description>
        <properties>
            <java.version>1.8</java.version>
        </properties>
        <dependencies>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
            </dependency>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-web</artifactId>
            </dependency>
    
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-test</artifactId>
                <scope>test</scope>
            </dependency>
        </dependencies>
    
        <build>
            <plugins>
                <plugin>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>spring-boot-maven-plugin</artifactId>
                </plugin>
            </plugins>
        </build>
    
    </project>

package com.pshdhx.elasticsearch.demo;
    
    import com.pshdhx.elasticsearch.demo.bean.Article;
    import io.searchbox.client.JestClient;
    import io.searchbox.core.Index;
    import io.searchbox.core.Search;
    import io.searchbox.core.SearchResult;
    import org.elasticsearch.client.AsyncSearchClient;
    import org.elasticsearch.client.RestHighLevelClient;
    import org.junit.jupiter.api.Test;
    import org.springframework.beans.factory.annotation.Autowired;
    import org.springframework.boot.autoconfigure.elasticsearch.RestClientBuilderCustomizer;
    import org.springframework.boot.test.autoconfigure.web.client.RestClientTest;
    import org.springframework.boot.test.context.SpringBootTest;
    import org.springframework.stereotype.Service;
    import org.springframework.web.client.RestTemplate;
    
    import java.io.IOException;
    
    //@Service
    @SpringBootTest
    class DemoApplicationTests {
    
    
    //    RestTemplate restTemplate;
    //
    //    JestClient01 jestClient01;
    //
    //
    //    @Autowired
    //    RestClientTest restClientTest;
    //
    //    RestClientBuilderCustomizer restClientBuilderCustomizer;
    //
    //
    //
    //
    //
    //    @Autowired
    //    RestHighLevelClient restHighLevelClient;
    
        @Autowired
        JestClient jestClient;
    
        @Test
        void contextLoads() {
            Article article = new Article(111,"psd","消息","news hello");
            Index index = new Index.Builder(article).index("pshdhx").type("news").build();
            try {
    //            JestClient jestCline = jestClient01.getJestCline();
    //            jestCline.execute(index);
    //            restHighLevelClient.search
    //            AsyncSearchClient asyncSearchClient = restHighLevelClient.asyncSearch();
    //            String forObject = restTemplate.getForObject("http://192.168.44.128:9200?first_name=John", String.class);
    //            System.out.println(forObject);
                jestClient.execute(index);
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    
        @Test
        void Search(){
            String json = "{\n" +
                    "    \"query\" : {\n" +
                    "        \"match\" : {\n" +
                    "            \"last_name\" : \"Smith\"\n" +
                    "        }\n" +
                    "    }\n" +
                    "}\n";
            Search search = new Search.Builder(json).addIndex("pshdhx").addType("news").build();
            try {
                SearchResult result = jestClient.execute(search);
                System.out.println(result);
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    
    }

结果：

[http://192.168.44.128:9200/\_search][http_192.168.44.128_9200_search]

{
        "took": 307,
        "timed_out": false,
        "_shards": {
            "total": 2,
            "successful": 2,
            "skipped": 0,
            "failed": 0
        },
        "hits": {
            "total": {
                "value": 2,
                "relation": "eq"
            },
            "max_score": 1.0,
            "hits": [
                {
                    "_index": "megacorp",
                    "_type": "employee",
                    "_id": "1",
                    "_score": 1.0,
                    "_source": {
                        "first_name": "John",
                        "last_name": "Smith",
                        "age": 25,
                        "about": "I love to go rock climbing",
                        "interests": [
                            "sports",
                            "music"
                        ]
                    }
                },
                {
                    "_index": "pshdhx",
                    "_type": "news",
                    "_id": "111",
                    "_score": 1.0,
                    "_source": {
                        "id": 111,
                        "author": "psd",
                        "title": "消息",
                        "content": "news hello"
                    }
                }
            ]
        }
    }

## ElasticSearch ##

配置类：

package com.pshdhx.elasticsearch.demo.config;
    
    import io.searchbox.client.JestClient;
    import io.searchbox.client.JestClientFactory;
    import io.searchbox.client.config.HttpClientConfig;
    import org.apache.http.HttpHost;
    import org.elasticsearch.client.RestClient;
    import org.elasticsearch.client.RestHighLevelClient;
    import org.springframework.context.annotation.Bean;
    import org.springframework.context.annotation.Configuration;
    import org.springframework.data.elasticsearch.client.ClientConfiguration;
    import org.springframework.data.elasticsearch.client.RestClientFactoryBean;
    import org.springframework.data.elasticsearch.client.RestClients;
    
    /**
     * @Authtor pshdhx
     * @Date 2021/3/3121:58
     * @Version 1.0
     */
    @Configuration
    public class EsConfig {
        /**
         * 重新注入RestHighLevelClient
         *
         * @return
    //     */
    //    @Bean
    //    public RestHighLevelClient restHighLevelClient() {
    //        RestHighLevelClient client = new RestHighLevelClient(
    //                RestClient.builder(
    //                        new HttpHost("http://192.168.44.128:9200", 9200, "http")
    //                )
    //        );
    //        return client;
    //    }
    
        @Bean
        public JestClient getJestCline() {
            JestClientFactory factory = new JestClientFactory();
            factory.setHttpClientConfig(new HttpClientConfig
                    .Builder("http://192.168.44.128:9200/")
                    .multiThreaded(true)
                    .build());
            return factory.getObject();
        }
    //
    //    @Bean
    //    public RestClient geRestCline() {
    //        RestClientFactoryBean factoryBean = new RestClientFactoryBean();
    //        factoryBean.setHosts("http://192.168.44.128:9200");
    //        RestHighLevelClient object = factoryBean.getObject();
    //
    //        JestClientFactory factory = new JestClientFactory();
    //        factory.setHttpClientConfig(new HttpClientConfig
    //                .Builder("http://192.168.44.128:9200")
    //                .multiThreaded(true)
    //                .build());
    //        return factory.getObject();
    //    }
    
        //    @Bean
    //    RestHighLevelClient elasticsearchClient() {
    //        ClientConfiguration configuration = ClientConfiguration.builder()
    //                .connectedTo("http://192.168.44.128:9200/")
    //                //.withConnectTimeout(Duration.ofSeconds(5))
    //                //.withSocketTimeout(Duration.ofSeconds(3))
    //                //.useSsl()
    //                //.withDefaultHeaders(defaultHeaders)
    //                //.withBasicAuth(username, password)
    //                // ... other options
    //
    //                .build();
    //        RestHighLevelClient client = RestClients.create(configuration).rest();
    //        return client;
    //    }
        @Bean
        public RestHighLevelClient restHighLevelClient() {
            ClientConfiguration configuration = ClientConfiguration.builder(
            )
                    .connectedTo("192.168.44.128:9200")
                    //.withConnectTimeout(Duration.ofSeconds(5))
                    //.withSocketTimeout(Duration.ofSeconds(3))
                    //.useSsl()
                    //.withDefaultHeaders(defaultHeaders)
                    //.withBasicAuth(username, password)
                    // ... other options
                    .build();
            RestHighLevelClient client = RestClients.create(configuration).rest();
            return client;
        }
    
    }

实体类：

package com.pshdhx.elasticsearch.demo.bean;
    
    import org.springframework.data.elasticsearch.annotations.Document;
    
    /**
     * @Authtor pshdhx
     * @Date 2021/4/118:38
     * @Version 1.0
     */
    @Document(indexName = "pshdhx",indexStoreType = "book")
    public class Book {
        private Integer id;
        private String bookName;
        private String author;
    
        public Integer getId() {
            return id;
        }
    
        public void setId(Integer id) {
            this.id = id;
        }
    
        public String getBookName() {
            return bookName;
        }
    
        public void setBookName(String bookName) {
            this.bookName = bookName;
        }
    
        public String getAuthor() {
            return author;
        }
    
        public void setAuthor(String author) {
            this.author = author;
        }
    }

实现类：

package com.pshdhx.elasticsearch.demo.repository;
    
    import com.pshdhx.elasticsearch.demo.bean.Book;
    import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;
    
    import java.util.List;
    
    /**
     * @Authtor pshdhx
     * @Date 2021/4/118:37
     * @Version 1.0
     */
    public interface BookRepository extends ElasticsearchRepository<Book,Integer> {
        public List<Book> findByBookNameLike(String bookName);
    }

测试类：

package com.pshdhx.elasticsearch.demo;
    
    import com.pshdhx.elasticsearch.demo.bean.Article;
    import com.pshdhx.elasticsearch.demo.bean.Book;
    import com.pshdhx.elasticsearch.demo.repository.BookRepository;
    import io.searchbox.client.JestClient;
    import io.searchbox.core.Index;
    import io.searchbox.core.Search;
    import io.searchbox.core.SearchResult;
    import org.elasticsearch.client.AsyncSearchClient;
    import org.elasticsearch.client.RestHighLevelClient;
    import org.junit.jupiter.api.Test;
    import org.springframework.beans.factory.annotation.Autowired;
    import org.springframework.boot.autoconfigure.elasticsearch.RestClientBuilderCustomizer;
    import org.springframework.boot.test.autoconfigure.web.client.RestClientTest;
    import org.springframework.boot.test.context.SpringBootTest;
    import org.springframework.stereotype.Service;
    import org.springframework.web.client.RestTemplate;
    
    import java.io.IOException;
    
    //@Service
    @SpringBootTest
    class DemoApplicationTests {
    
    
    //    RestTemplate restTemplate;
    //
    //    JestClient01 jestClient01;
    //
    //
    //    @Autowired
    //    RestClientTest restClientTest;
    //
    //    RestClientBuilderCustomizer restClientBuilderCustomizer;
    //
    //
    //
    //
    //
    //    @Autowired
    //    RestHighLevelClient restHighLevelClient;
    
        @Autowired
        JestClient jestClient;
    
        @Test
        void contextLoads() {
            Article article = new Article(111,"psd","消息","news hello");
            Index index = new Index.Builder(article).index("pshdhx").type("news").build();
            try {
    //            JestClient jestCline = jestClient01.getJestCline();
    //            jestCline.execute(index);
    //            restHighLevelClient.search
    //            AsyncSearchClient asyncSearchClient = restHighLevelClient.asyncSearch();
    //            String forObject = restTemplate.getForObject("http://192.168.44.128:9200?first_name=John", String.class);
    //            System.out.println(forObject);
                jestClient.execute(index);
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    
        @Test
        void Search(){
            String json = "{\n" +
                    "    \"query\" : {\n" +
                    "        \"match\" : {\n" +
                    "            \"last_name\" : \"Smith\"\n" +
                    "        }\n" +
                    "    }\n" +
                    "}\n";
            Search search = new Search.Builder(json).addIndex("pshdhx").addType("news").build();
            try {
                SearchResult result = jestClient.execute(search);
                System.out.println(result);
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    
        @Autowired
        BookRepository repository;
        @Test
        void test02(){
            Book b = new Book();
            b.setAuthor("pshdhx");
            b.setId(1);
            b.setBookName("book1");
            repository.save(b);
        }
    
        @Test
        void Test3(){
            for(Book book:repository.findByBookNameLike("bo")){
                System.out.println(book);
            }
        }
    
    }

结果：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BzaGRoeA_size_16_color_FFFFFF_t_70 3][]

[http_192.168.44.128_9200]: http://192.168.44.128:9200/
[https_www.elastic.co_guide_cn_elasticsearch_guide_current_indexing_employee_documents.html]: https://www.elastic.co/guide/cn/elasticsearch/guide/current/_indexing_employee_documents.html
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BzaGRoeA_size_16_color_FFFFFF_t_70]: /images/20221022/136e665f026a4d809543c34c72100ec3.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BzaGRoeA_size_16_color_FFFFFF_t_70 1]: /images/20221022/4a9a10faffe14e15a193d53b22357a90.png
[20210331212846847.png]: https://img-blog.csdnimg.cn/20210331212846847.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BzaGRoeA_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/20210331212908635.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BzaGRoeA==,size_16,color_FFFFFF,t_70
[http_192.168.44.128_9200_search]: http://192.168.44.128:9200/_search
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BzaGRoeA_size_16_color_FFFFFF_t_70 3]: https://img-blog.csdnimg.cn/20210401193147444.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BzaGRoeA==,size_16,color_FFFFFF,t_70