Elasticsearch 安装cerebro、kibana及安装logstash并导入csv数据

女爷i 2022-12-08 12:57 104阅读 0赞

# 一 cerebro  #

cerebro是一个Elasticsearch集群管理工具，源码地址：[https://github.com/lmenezes/cerebro][https_github.com_lmenezes_cerebro]

解压后进入conf目录下，修改application.conf

hosts = [
      {
         host = "http://127.0.0.1:9200"
         name = "my-application"
      #  headers-whitelist = [ "x-proxy-user", "x-proxy-roles", "X-Forwarded-For" ]
      }
      # Example of host with authentication
      #{
      #  host = "http://some-authenticated-host:9200"
      #  name = "Secured Cluster"
      #  auth = {
      #    username = "username"
      #    password = "secret-password"
      #  }
      #}
    ]

启动：

./bin/cerebro

默认端口为9000

# 二 Kibana #

在官网下载与elasticsearch相同版本的kibana

修改config目录下的kibana.yml

server.host: "0.0.0.0"  
  
  
elasticsearch.hosts: \["http://127.0.0.1:9200","http://127.0.0.1:9201"\]  
  
  
i18n.locale: "zh-CN"

启动：  
./bin/kibana

默认端口 5601

# 三 logstash #

在官网下载与elasticsearch相同版本的logstash

下载最MovieLens最小测试数据集：https://grouplens.org/datasets/movielens/

找到要导入的数据文件movies.csv

修改config目录下的logstash-sample.conf   重命名为logstash.conf

修改logstash.conf：

启动:./bin/logstash -f config/logstash.conf

input {
      file {
        path => "/Users/yiruan/dev/elk7/logstash-7.0.1/bin/movies.csv"
        start_position => "beginning"
        sincedb_path => "/dev/null"
      }
    }
    filter {
      csv {
        separator => ","
        columns => ["id","content","genre"]
      }
    
      mutate {
        split => { "genre" => "|" }
        remove_field => ["path", "host","@timestamp","message"]
      }
    
      mutate {
    
        split => ["content", "("]
        add_field => { "title" => "%{[content][0]}"}
        add_field => { "year" => "%{[content][1]}"}
      }
    
      mutate {
        convert => {
          "year" => "integer"
        }
        strip => ["title"]
        remove_field => ["path", "host","@timestamp","message","content"]
      }
    
    }
    output {
       elasticsearch {
         hosts => "http://localhost:9200"
         index => "movies"
         document_id => "%{id}"
       }
      stdout {}
    }

注意csv文件需要换成实际的位置。

输出如下日志，则导入成功，可以在cerebro上查看到索引

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpYW5nd2VubWFpbA_size_16_color_FFFFFF_t_70][]

[https_github.com_lmenezes_cerebro]: https://github.com/lmenezes/cerebro
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpYW5nd2VubWFpbA_size_16_color_FFFFFF_t_70]: /images/20221123/b17ea2073fd8445ca53e919c123cf8b5.png