elasticsearch-7.8.0 + ik中文分词，打造全文搜索

不念不忘少年蓝@ 2023-02-27 12:08 0阅读 0赞

我的环境是centos7 64位的系统，es7.6 已经内置了jdk,当前安装的是7.8版本，如果没有jdk也不用安装，当然你想安装的话也是可以的

yum install -y java-1.8.0-openjdk*

elasticsearch官网下载地址：

wget  https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.8.0-linux-x86_64.tar.gz &

下载ik分词插件：[https://github.com/medcl/elasticsearch-analysis-ik/releases][https_github.com_medcl_elasticsearch-analysis-ik_releases]

wget [https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.8.0/elasticsearch-analysis-ik-7.8.0.zip][https_github.com_medcl_elasticsearch-analysis-ik_releases_download_v7.8.0_elasticsearch-analysis-ik-7.8.0.zip]

下载完elasticsearch直接解压就行了，我是解压到/usr/local/elasticsearch-7.8.0文件夹

mkdir /usr/local/elasticsearch-7.8.0/plugins/ik

把ik分词插件解压到usr/local/elasticsearch-7.8.0/plugins/ik 文件夹

新建用户组 `groupadd esgroup`

新增用户 `useradd esroot -g esgroup -p esroot`

设置权限 `chown -R esroot:esgroup /usr/local/`elasticsearch-7.8.0

切换权限 `su esroot`

用root是无法启动的哈！要用`esroot用户启动：nohup /usr/local/elasticsearch-7.8.0/bin/elasticsearch&`

`curl 127.0.0.1:9200 查看`elasticsearch`是否启动成功`

[esroot@email plugins]$ curl 127.0.0.1:9200
    {
      "name" : "email.buruyouni.com",
      "cluster_name" : "elasticsearch",
      "cluster_uuid" : "2kk4PsMnQHevnK8w1kkUAQ",
      "version" : {
        "number" : "7.8.0",
        "build_flavor" : "default",
        "build_type" : "tar",
        "build_hash" : "757314695644ea9a1dc2fecd26d1a43856725e65",
        "build_date" : "2020-06-14T19:35:50.234439Z",
        "build_snapshot" : false,
        "lucene_version" : "8.5.1",
        "minimum_wire_compatibility_version" : "6.8.0",
        "minimum_index_compatibility_version" : "6.0.0-beta1"
      },
      "tagline" : "You Know, for Search"
    }

查看ik的分词，我这里用的是：ik\_max\_word模式

curl -H "Content-Type: application/json" http://localhost:9200/\_analyze  -X POST  --data '\{ "text":"广东省佛山市","analyzer": "ik\_max\_word"\}'

{
        "tokens": [
            {
                "token": "广东省",
                "start_offset": 0,
                "end_offset": 3,
                "type": "CN_WORD",
                "position": 0
            },
            {
                "token": "广东",
                "start_offset": 0,
                "end_offset": 2,
                "type": "CN_WORD",
                "position": 1
            },
            {
                "token": "省",
                "start_offset": 2,
                "end_offset": 3,
                "type": "CN_CHAR",
                "position": 2
            },
            {
                "token": "佛山市",
                "start_offset": 3,
                "end_offset": 6,
                "type": "CN_WORD",
                "position": 3
            },
            {
                "token": "佛山",
                "start_offset": 3,
                "end_offset": 5,
                "type": "CN_WORD",
                "position": 4
            },
            {
                "token": "市",
                "start_offset": 5,
                "end_offset": 6,
                "type": "CN_CHAR",
                "position": 5
            }
        ]
    }

添加，put请求类型，添加id=3的文档（我这里用的是postman啦）

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI3NTE3Mzc3_size_16_color_FFFFFF_t_70][]

post请求类型，更新id=5的文档的id字段和total\_price字段：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI3NTE3Mzc3_size_16_color_FFFFFF_t_70 1][]

delete请求类型，删除id=3的文档：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI3NTE3Mzc3_size_16_color_FFFFFF_t_70 2][]

### elasticsearch配置本地词库（修改后需要重启才能生效） ###

新建一个demo.dic文件，每个次之间用换行分开即可

![20200723160157266.png][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI3NTE3Mzc3_size_16_color_FFFFFF_t_70 3][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI3NTE3Mzc3_size_16_color_FFFFFF_t_70 4][]

### elasticsearch配置远程词库（修改后不需要重启，1分钟生效） ###

[https://www.buruyouni.com/ikdic.dic][https_www.buruyouni.com_ikdic.dic] **是我放自定义词典的地方，每次更改大概1分钟生效，里面也是用换行区分就行了**

![20200723160453383.png][]

## ##

### PHP连接elasticsearch ###

https://github.com/elastic/elasticsearch-php

### mysql数据批量导入到elasticsearch ###

PHP操作es文档位置：https://www.elastic.co/guide/cn/elasticsearch/php/current/\_indexing\_documents.html

我数据库里面有1378W条数据，需要插入到elasticsearch

jvm.options的配置：-Xms1g   -Xmx1g只给了1GB内存，数据库还是用的mysql5.5机械盘，还固态盘的话估计会

每次从数据库读取3W条数据,然后插入到elasticsearch，从数据库获取数据到插入到elasticsearch，其中获取3W条数据大约是0.5-0.7秒一次，获取数据+插入到elasticsearch的平均是2.5W条/秒

/**
         * mysql数据批量插入数据到elasticsearch written:yangxingyi
         * 文档位置：https://www.elastic.co/guide/cn/elasticsearch/php/current/_indexing_documents.html
         */
        public function mysql2esAction(){
            ini_set("memory_limit","4048M");
            $start = time();
            $hosts = [
                'http://elastic:changeme@localhost:9200',
            ];
            $client = ClientBuilder::create()->setHosts($hosts)->build();
    
            //从mysql获取数据
            $field = "ID_CDKEY_ORDER,PRODUCTS_NAME,STAMP,EMAIL,BESTELLWERT";
            $count = Db::name("cdkey_order")->count();
            p($count);
    
            $limit = 30000;
            $prevId = 0;
            for($i=1;$i<=ceil($count/$limit);$i++){
                echo ($i-1)*$limit.PHP_EOL;
                if($i==1){
                    $data = Db::name("cdkey_order")->field($field)->limit(0,$limit)->select();
                }else{
                    $data = Db::name("cdkey_order")->where("ID_CDKEY_ORDER",">",$prevId)->field($field)->limit(0,$limit)->select();
                }
    
                if(!$data){
                    break;
                }
    
                $params['index'] = 'cdkey_order';//poetry这个document
                foreach($data as $k=>$v){
                    $params['body'][]=array(
                        'index' => array(
                           //id我这里用的是数据库的自增id
                            '_id'=>$v['ID_CDKEY_ORDER']
                        ),
                    );
    
                    $params['body'][] = array(
                        'product_name'=>$v['PRODUCTS_NAME'],
                        'stamp'=>$v['STAMP'],
                        'email'=>$v['EMAIL'],
                        'amout'=>$v['BESTELLWERT'],
                    );
                }
    
                $prevId = $v["ID_CDKEY_ORDER"];//获取最后一个customer_id
    
                $res = $client->bulk($params);
                unset($params);//用完这个变量要删除掉,不然这个标量会变得非常大
            }
    }

安装elasticsearch可视化工具elasticsearch-head

需要安装node js 6以上的版本

npm install -g cnpm --registry=https://registry.npm.taobao.org

### **elasticsearch开启远程访问**：[https://blog.csdn.net/chinoukin/article/details/94720970][https_blog.csdn.net_chinoukin_article_details_94720970] ###

[https_github.com_medcl_elasticsearch-analysis-ik_releases]: https://github.com/medcl/elasticsearch-analysis-ik/releases
[https_github.com_medcl_elasticsearch-analysis-ik_releases_download_v7.8.0_elasticsearch-analysis-ik-7.8.0.zip]: https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.8.0/elasticsearch-analysis-ik-7.8.0.zip
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI3NTE3Mzc3_size_16_color_FFFFFF_t_70]: /images/20230209/c61f529cc19b49348a56002d0801712b.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI3NTE3Mzc3_size_16_color_FFFFFF_t_70 1]: /images/20230209/0982cd8ceb8740f897b6fd958d4d636c.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI3NTE3Mzc3_size_16_color_FFFFFF_t_70 2]: /images/20230209/098c48addc76429fb19f8c8bc3bf1c90.png
[20200723160157266.png]: https://img-blog.csdnimg.cn/20200723160157266.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI3NTE3Mzc3_size_16_color_FFFFFF_t_70 3]: https://img-blog.csdnimg.cn/20200723160042108.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI3NTE3Mzc3,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI3NTE3Mzc3_size_16_color_FFFFFF_t_70 4]: https://img-blog.csdnimg.cn/20200723160006279.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzI3NTE3Mzc3,size_16,color_FFFFFF,t_70
[https_www.buruyouni.com_ikdic.dic]: https://www.buruyouni.com/ikdic.dic
[20200723160453383.png]: https://img-blog.csdnimg.cn/20200723160453383.png
[https_blog.csdn.net_chinoukin_article_details_94720970]: https://blog.csdn.net/chinoukin/article/details/94720970