通过Docker部署Elasticsearch集群

忘是亡心i 2023-03-13 09:16 28阅读 0赞

以7.6.2为例

# Elasticsearch #

## 挂载 ##

在宿主机上创建Elasticsearch的数据存储、config配置目录  
eg:

数据存储：
    mkdir /data/es/data-b
    配置文件
    mkdir /home/es/elasticsearch-7.6.2/config

> 新建用户 adduser es  
> 数据存储需要赋予权限：sudo chown -R 1000:1000 /data/es/data-b

## Elasticsearch配置 ##

在/home/es/elasticsearch-7.6.2/config文件夹下新建elasticsearch.yml、 jvm.options

具体内容可以参考：  
elasticsearch.yml

cluster.name: business-log
    node.name: es-b-188
    http.port: 9201
    transport.tcp.port: 9301
    discovery.seed_hosts: ["10.133.0.188","10.133.0.190","10.133.0.191"]
    cluster.initial_master_nodes: ["10.133.0.188","10.133.0.190","10.133.0.191"]
    network.bind_host: 10.133.0.188
    network.publish_host: 10.133.0.188

jvm.options

## JVM configuration
    
    ################################################################
    ## IMPORTANT: JVM heap size
    ################################################################
    ##
    ## You should always set the min and max JVM heap
    ## size to the same value. For example, to set
    ## the heap to 4 GB, set:
    ##
    ## -Xms4g
    ## -Xmx4g
    ##
    ## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
    ## for more information
    ##
    ################################################################
    
    # Xms represents the initial size of total heap space
    # Xmx represents the maximum size of total heap space
    
    -Xms26g
    -Xmx26g
    
    ################################################################
    ## Expert settings
    ################################################################
    ##
    ## All settings below this section are considered
    ## expert settings. Don't tamper with them unless
    ## you understand what you are doing
    ##
    ################################################################
    
    ## GC configuration
    #-XX:+UseConcMarkSweepGC
    #-XX:CMSInitiatingOccupancyFraction=75
    #-XX:+UseCMSInitiatingOccupancyOnly
    
    ## G1GC Configuration
    # NOTE: G1GC is only supported on JDK version 10 or later.
    # To use G1GC uncomment the lines below.
    #-XX:-UseConcMarkSweepGC
    #-XX:-UseCMSInitiatingOccupancyOnly
    -XX:+UseG1GC
    -XX:InitiatingHeapOccupancyPercent=75
    
    ## DNS cache policy
    # cache ttl in seconds for positive DNS lookups noting that this overrides the
    # JDK security property networkaddress.cache.ttl; set to -1 to cache forever
    -Des.networkaddress.cache.ttl=60
    # cache ttl in seconds for negative DNS lookups noting that this overrides the
    # JDK security property networkaddress.cache.negative ttl; set to -1 to cache
    # forever
    -Des.networkaddress.cache.negative.ttl=10
    
    ## optimizations
    
    # pre-touch memory pages used by the JVM during initialization
    -XX:+AlwaysPreTouch
    
    ## basic
    
    # explicitly set the stack size
    -Xss1m
    
    # set to headless, just in case
    -Djava.awt.headless=true
    
    # ensure UTF-8 encoding by default (e.g. filenames)
    -Dfile.encoding=UTF-8
    
    # use our provided JNA always versus the system one
    -Djna.nosys=true
    
    # turn off a JDK optimization that throws away stack traces for common
    # exceptions because stack traces are important for debugging
    -XX:-OmitStackTraceInFastThrow
    
    # flags to configure Netty
    -Dio.netty.noUnsafe=true
    -Dio.netty.noKeySetOptimization=true
    -Dio.netty.recycler.maxCapacityPerThread=0
    
    # log4j 2
    -Dlog4j.shutdownHookEnabled=false
    -Dlog4j2.disable.jmx=true
    
    -Djava.io.tmpdir=${ES_TMPDIR}
    
    ## heap dumps
    
    # generate a heap dump when an allocation from the Java heap fails
    # heap dumps are created in the working directory of the JVM
    -XX:+HeapDumpOnOutOfMemoryError
    
    # specify an alternative path for heap dumps; ensure the directory exists and
    # has sufficient space
    -XX:HeapDumpPath=data
    
    # specify an alternative path for JVM fatal error logs
    -XX:ErrorFile=logs/hs_err_pid%p.log
    
    ## JDK 8 GC logging
    
    8:-XX:+PrintGCDetails
    8:-XX:+PrintGCDateStamps
    8:-XX:+PrintTenuringDistribution
    8:-XX:+PrintGCApplicationStoppedTime
    8:-Xloggc:logs/gc.log
    8:-XX:+UseGCLogFileRotation
    8:-XX:NumberOfGCLogFiles=32
    8:-XX:GCLogFileSize=64m
    
    # JDK 9+ GC logging
    9-:-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m
    # due to internationalization enhancements in JDK 9 Elasticsearch need to set the provider to COMPAT otherwise
    # time/date parsing will break in an incompatible way for some date patterns and locals
    9-:-Djava.locale.providers=COMPAT

## 启动 ##

创建docker-compose.yml文件，内容如下：

version: '2.2'
    services:
      elasticsearch:
        image: elasticsearch:7.6.2
        restart: always
        container_name: es-b-188
        network_mode: host
        volumes:
          - /data/es/data-b:/usr/share/elasticsearch/data
          - /home/es/elasticsearch-7.6.2/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
          - /home/es/elasticsearch-7.6.2/config/jvm.options:/usr/share/elasticsearch/config/jvm.options
        environment:
          - bootstrap.memory_lock=true
        ulimits:
          memlock:
            soft: -1
            hard: -1

启动：

docker-compose up -d

## 备注 ##

elasticsearch.yml、jvm.options、docker-compose.yml等每个节点都需要配置，其中elasticsearch.yml、docker-compose.yml具体节点信息需要替换。

# Kibana #

## 挂载 ##

在宿主机上创建kibana config配置目录  
eg:

配置文件
    mkdir /home/es/kibana-7.6.2

## Kibana配置 ##

在/home/es/kibana-7.6.2目录下创建kibana.yml，具体内容如下：

# Kibana is served by a back end server. This setting specifies the port to use.
    server.port: 5602
    
    # Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
    # The default is 'localhost', which usually means remote machines will not be able to connect.
    # To allow connections from remote users, set this parameter to a non-loopback address.
    server.host: 0.0.0.0
    
    # Enables you to specify a path to mount Kibana at if you are running behind a proxy.
    # Use the `server.rewriteBasePath` setting to tell Kibana if it should remove the basePath
    # from requests it receives, and to prevent a deprecation warning at startup.
    # This setting cannot end in a slash.
    #server.basePath: ""
    
    # Specifies whether Kibana should rewrite requests that are prefixed with
    # `server.basePath` or require that they are rewritten by your reverse proxy.
    # This setting was effectively always `false` before Kibana 6.3 and will
    # default to `true` starting in Kibana 7.0.
    #server.rewriteBasePath: false
    
    # The maximum payload size in bytes for incoming server requests.
    #server.maxPayloadBytes: 1048576
    
    # The Kibana server's name.  This is used for display purposes.
    #server.name: "your-hostname"
    
    # The URLs of the Elasticsearch instances to use for all your queries.
    elasticsearch.hosts: ["http://10.133.0.188:9201"]
    
    # When this setting's value is true Kibana uses the hostname specified in the server.host
    # setting. When the value of this setting is false, Kibana uses the hostname of the host
    # that connects to this Kibana instance.
    #elasticsearch.preserveHost: true
    
    # Kibana uses an index in Elasticsearch to store saved searches, visualizations and
    # dashboards. Kibana creates a new index if the index doesn't already exist.
    #kibana.index: ".kibana"
    
    # The default application to load.
    #kibana.defaultAppId: "home"
    
    # If your Elasticsearch is protected with basic authentication, these settings provide
    # the username and password that the Kibana server uses to perform maintenance on the Kibana
    # index at startup. Your Kibana users still need to authenticate with Elasticsearch, which
    # is proxied through the Kibana server.
    #elasticsearch.username: "kibana"
    #elasticsearch.password: "pass"
    
    # Enables SSL and paths to the PEM-format SSL certificate and SSL key files, respectively.
    # These settings enable SSL for outgoing requests from the Kibana server to the browser.
    #server.ssl.enabled: false
    #server.ssl.certificate: /path/to/your/server.crt
    #server.ssl.key: /path/to/your/server.key
    
    # Optional settings that provide the paths to the PEM-format SSL certificate and key files.
    # These files are used to verify the identity of Kibana to Elasticsearch and are required when
    # xpack.security.http.ssl.client_authentication in Elasticsearch is set to required.
    #elasticsearch.ssl.certificate: /path/to/your/client.crt
    #elasticsearch.ssl.key: /path/to/your/client.key
    
    # Optional setting that enables you to specify a path to the PEM file for the certificate
    # authority for your Elasticsearch instance.
    #elasticsearch.ssl.certificateAuthorities: [ "/path/to/your/CA.pem" ]
    
    # To disregard the validity of SSL certificates, change this setting's value to 'none'.
    #elasticsearch.ssl.verificationMode: full
    
    # Time in milliseconds to wait for Elasticsearch to respond to pings. Defaults to the value of
    # the elasticsearch.requestTimeout setting.
    #elasticsearch.pingTimeout: 1500
    
    # Time in milliseconds to wait for responses from the back end or Elasticsearch. This value
    # must be a positive integer.
    #elasticsearch.requestTimeout: 30000
    
    # List of Kibana client-side headers to send to Elasticsearch. To send *no* client-side
    # headers, set this value to [](an empty list).
    #elasticsearch.requestHeadersWhitelist: [ authorization ]
    
    # Header names and values that are sent to Elasticsearch. Any custom headers cannot be overwritten
    # by client-side headers, regardless of the elasticsearch.requestHeadersWhitelist configuration.
    #elasticsearch.customHeaders: {}
    
    # Time in milliseconds for Elasticsearch to wait for responses from shards. Set to 0 to disable.
    #elasticsearch.shardTimeout: 30000
    
    # Time in milliseconds to wait for Elasticsearch at Kibana startup before retrying.
    #elasticsearch.startupTimeout: 5000
    
    # Logs queries sent to Elasticsearch. Requires logging.verbose set to true.
    #elasticsearch.logQueries: false
    
    # Specifies the path where Kibana creates the process ID file.
    #pid.file: /var/run/kibana.pid
    
    # Enables you specify a file where Kibana stores log output.
    #logging.dest: stdout
    
    # Set the value of this setting to true to suppress all logging output.
    #logging.silent: false
    
    # Set the value of this setting to true to suppress all logging output other than error messages.
    #logging.quiet: false
    
    # Set the value of this setting to true to log all events, including system usage information
    # and all requests.
    #logging.verbose: false
    
    # Set the interval in milliseconds to sample system and process performance
    # metrics. Minimum is 100ms. Defaults to 5000.
    #ops.interval: 5000
    
    # Specifies locale to be used for all localizable strings, dates and number formats.
    # Supported languages are the following: English - en , by default , Chinese - zh-CN .
    #i18n.locale: "en"

## 启动 ##

创建docker-compose.yml文件，内容如下：

version: '2'
    services:
      kibana:
        image: kibana:7.6.2
        volumes:
          - ./kibana.yml:/usr/share/kibana/config/kibana.yml
        ports:
          - '5602:5602/tcp'

启动：

docker-compose up -d

# 官方文档 #

https://www.elastic.co/guide/en/elasticsearch/reference/7.5/docker.html  
https://www.elastic.co/guide/en/kibana/current/docker.html

# 个人微信公众号 #

![在这里插入图片描述][20200512210044537.jpg_pic_center]

[20200512210044537.jpg_pic_center]: /images/20230312/5a8f7a2666e243e38b9ecd209eb358c4.png