prometheus服务发现-consul生产环境服务部署

深藏阁楼爱情的钟 2022-01-16 06:53 478阅读 0赞

[2019独角兽企业重金招聘Python工程师标准>>> ][2019_Python_] ![hot3.png][]

模拟生产环境多节点部署，使用linux服务方式部署consul集群，保证服务高可用即异常挂掉或主机重启后能自动回复，同时启用agent通信加密和接口认证。

# 前期准备 #

1.  安装

将下载解压得到二进制文件consul拷贝到/usr/local/bin目录

sudo chown root:root /usr/local/bin/consul
    consul --version # 验证

1.  自动补全

consul -autocomplete-install
    complete -C /usr/local/bin/consul consul

1.  准备目录

创建/opt/consul目录，并在其下准备如下3个server的目录

/opt/consul$ tree
    .
    ├── server1
    │   ├── config
    │   └── data
    ├── server2
    │   ├── config
    │   └── data
    └── server3
        ├── config
        └── data

1.  生成秘钥

用于集群间通信加密，需要保证集群中所有节点都配置该秘钥

$ consul keygen
    mz8Con27P34D9fiPG1bjHA==

# 配置 #

## server1 ##

### service unit ###

准备文件：/lib/systemd/system/consul-server1.service，内容如下：

[Unit]
    Description="consul server1"
    Requires=network-online.target
    After=network-online.target
    [Service]
    ExecStart=/usr/local/bin/consul agent -config-dir=/opt/consul/server1/config
    ExecReload=/usr/local/bin/consul reload
    KillMode=process
    Restart=on-failure
    LimitNOFILE=65536
    [Install]
    WantedBy=multi-user.target

### 配置 ###

准备配置文件：/opt/consul/server1/config/config.json，内容如下：

{
      "datacenter": "prometheus",
      "bind_addr":"10.106.169.121",
      "log_level": "INFO",
      "node_id":"09d82408-bc4f-49e0-4208-61ef1d4842f7",
      "node_name": "server1",
      "data_dir":"/opt/consul/server1/data",
      "server": true,
      "bootstrap_expect":3,
      "encrypt": "mz8Con27P34D9fiPG1bjHA==",
      "ui":true,
      "client_addr":"0.0.0.0",
      "retry_join":["10.106.169.121:18301","10.106.169.121:28301","10.106.169.121:38301"],
      "ports": {
         "http": 18500,
         "dns": 18600,
         "serf_lan":18301,
         "serf_wan":18302,
         "server":18300,
         "grpc":-1
      },
      "acl": {
        "enabled": true,
        "default_policy": "deny",
        "down_policy": "extend-cache",
        "tokens":{
            "master":"47eca91b-a5e7-e82d-6424-dba7637e0737",
            "agent":"47eca91b-a5e7-e82d-6424-dba7637e0737"
        }
      }
    }

## server2 ##

### service unit ###

准备文件：/lib/systemd/system/consul-server2.service，内容如下：

[Unit]
    Description="consul server2"
    Requires=network-online.target
    After=network-online.target
    [Service]
    ExecStart=/usr/local/bin/consul agent -config-dir=/opt/consul/server2/config
    ExecReload=/usr/local/bin/consul reload
    KillMode=process
    Restart=on-failure
    LimitNOFILE=65536
    [Install]
    WantedBy=multi-user.target

### 配置 ###

准备配置文件：/opt/consul/server2/config/config.json，内容如下：

{
      "datacenter": "prometheus",
      "bind_addr":"10.106.169.121",
      "log_level": "INFO",
      "node_id":"613ccd6e-68d1-3bbd-b2a4-3cbc450f019d",
      "node_name": "server2",
      "data_dir":"/opt/consul/server2/data",
      "server": true,
      "bootstrap_expect":3,
      "encrypt": "mz8Con27P34D9fiPG1bjHA==",
      "ui":true,
      "client_addr":"0.0.0.0",
      "retry_join":["10.106.169.121:18301","10.106.169.121:28301","10.106.169.121:38301"],
      "ports": {
         "http": 28500,
         "dns": 28600,
         "serf_lan":28301,
         "serf_wan":28302,
         "server":28300,
         "grpc":-1
      },
      "acl": {
        "enabled": true,
        "default_policy": "deny",
        "down_policy": "extend-cache",
        "tokens":{
            "master":"47eca91b-a5e7-e82d-6424-dba7637e0737",
            "agent":"47eca91b-a5e7-e82d-6424-dba7637e0737"
        }
      }
    }

## server3 ##

### service unit ###

准备文件：/lib/systemd/system/consul-server3.service，内容如下：

[Unit]
    Description="consul server3"
    Requires=network-online.target
    After=network-online.target
    [Service]
    ExecStart=/usr/local/bin/consul agent -config-dir=/opt/consul/server3/config
    ExecReload=/usr/local/bin/consul reload
    KillMode=process
    Restart=on-failure
    LimitNOFILE=65536
    [Install]
    WantedBy=multi-user.target

### 配置 ###

准备配置文件：/opt/consul/server3/config/config.json，内容如下：

{
      "datacenter": "prometheus",
      "bind_addr":"10.106.169.121",
      "log_level": "INFO",
      "node_id":"d8a09ffd-7ccb-84bd-7231-8d8b7a01951e",
      "node_name": "server3",
      "data_dir":"/opt/consul/server3/data",
      "server": true,
      "bootstrap_expect":3,
      "encrypt": "mz8Con27P34D9fiPG1bjHA==",
      "ui":true,
      "client_addr":"0.0.0.0",
      "retry_join":["10.106.169.121:18301","10.106.169.121:28301","10.106.169.121:38301"],
      "ports": {
         "http": 38500,
         "dns": 38600,
         "serf_lan":38301,
         "serf_wan":38302,
         "server":38300,
         "grpc":-1
      },
      "acl": {
        "enabled": true,
        "default_policy": "deny",
        "down_policy": "extend-cache",
        "tokens":{
            "master":"47eca91b-a5e7-e82d-6424-dba7637e0737",
            "agent":"47eca91b-a5e7-e82d-6424-dba7637e0737"
        }
      }
    }

# 启动 #

通过命令 ： sudo systemctl enable consul-server1 consul-server2 consul-server3将服务设置为开机自动启动，再使用命令启动服务 ： sudo systemctl restart consul-server1 consul-server2 consul-server3

# 验证 #

## UI ##

在页面[http://127.0.0.1:18500/ui/prometheus/acls/tokens输入配置中的master][http_127.0.0.1_18500_ui_prometheus_acls_tokens_master] token，再刷新界面可以在services和nodes中查看到信息

## API ##

$ curl http://127.0.0.1:18500/v1/catalog/nodes #未带token，返回空的节点列表
    []
    $ curl http://127.0.0.1:18500/v1/catalog/nodes -H 'x-consul-token: 47eca91b-a5e7-e82d-6424-dba7637e0737' # 通过在header中增加x-consul-token则可返回节点列表
    [{"ID":"09d82408-bc4f-49e0-4208-61ef1d4842f7","Node":"server1","Address":"10.106.169.121","Datacenter":"prometheus","TaggedAddresses":null,"Meta":null,"CreateIndex":9,"ModifyIndex":9},{"ID":"613ccd6e-68d1-3bbd-b2a4-3cbc450f019d","Node":"server2","Address":"10.106.169.121","Datacenter":"prometheus","TaggedAddresses":null,"Meta":null,"CreateIndex":7,"ModifyIndex":7},{"ID":"d8a09ffd-7ccb-84bd-7231-8d8b7a01951e","Node":"server3","Address":"10.106.169.121","Datacenter":"prometheus","TaggedAddresses":null,"Meta":null,"CreateIndex":8,"ModifyIndex":8}]

# 参考 #

[https://learn.hashicorp.com/consul/advanced/day-1-operations/deployment-guide][https_learn.hashicorp.com_consul_advanced_day-1-operations_deployment-guide] \[官方部署方式\]

[https://www.consul.io/docs/agent/acl-system.html][https_www.consul.io_docs_agent_acl-system.html] \[acl介绍\]

[https://learn.hashicorp.com/consul/advanced/day-1-operations/acl-guide][https_learn.hashicorp.com_consul_advanced_day-1-operations_acl-guide] \[acl配置\]

[https://learn.hashicorp.com/consul/advanced/day-1-operations/agent-encryption][https_learn.hashicorp.com_consul_advanced_day-1-operations_agent-encryption] \[agent通信加密\]

转载于:https://my.oschina.net/sannychan/blog/3008165

[2019_Python_]: https://my.oschina.net/u/2663968/blog/3061697
[hot3.png]: /images/20220114/d1cb958027d1454c9ee1b3cbf4580a3e.png
[http_127.0.0.1_18500_ui_prometheus_acls_tokens_master]: http://127.0.0.1:18500/ui/prometheus/acls/tokens%E8%BE%93%E5%85%A5%E9%85%8D%E7%BD%AE%E4%B8%AD%E7%9A%84master
[https_learn.hashicorp.com_consul_advanced_day-1-operations_deployment-guide]: https://learn.hashicorp.com/consul/advanced/day-1-operations/deployment-guide
[https_www.consul.io_docs_agent_acl-system.html]: https://www.consul.io/docs/agent/acl-system.html
[https_learn.hashicorp.com_consul_advanced_day-1-operations_acl-guide]: https://learn.hashicorp.com/consul/advanced/day-1-operations/acl-guide
[https_learn.hashicorp.com_consul_advanced_day-1-operations_agent-encryption]: https://learn.hashicorp.com/consul/advanced/day-1-operations/agent-encryption