docker篇-(docker-compose安装etcd高可用集群)

客官°小女子只卖身不卖艺 2022-11-17 15:58 532阅读 0赞

### docker-compose安装etcd高可用集群 ###

*  构建etcd镜像
 *  生成ssl证书
 *   *  安装cfssl证书生成工具
     *  编写生成证书的配置文件
     *  编写ca根证书配置文件
     *  编写服务端认证证书配置文件
     *  编写客户端证书配置
 *  编写docker-compose.yml文件
 *   *  返回上级目录
     *  docker-compose.yml如下
 *  运行etcd集群
 *  查看集群状态

# 构建etcd镜像 #

这里我安装的etcd集群使用的是3.4.9版本的  
1.下载etcd安装包,并解压

# cd到当前用户所在目录并创建etcd然后到etcd目录
    cd ~ && mkdir etcd && cd etcd
    # 下载etcd二进制压缩包并解压
    wget https://repo.huaweicloud.com/etcd/v3.4.9/etcd-v3.4.9-linux-amd64.tar.gz && tar zxvf etcd-v3.4.9-linux-amd64.tar.gz
    # 删除下载的etcd压缩包
    rm -rf etcd-v3.4.9-linux-amd64.tar.gz
    # 创建bin目录，并将etcd解压出来的二进制文件复制到bin目录
    mkdir bin && mv etcd-v3.4.9-linux-amd64/{ etcd,etcdctl} bin/
    # 删除解压出来的etcd目录
    rm -rf etcd-v3.4.9-linux-amd64/

2.编写Dockerfile文件

cat > Dockerfile <<EOF FROM alpine MAINTAINER lhstack@foxmail.com RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories \\ && apk update \\ && apk add --no-cache tzdata \\ && cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime ENV TZ Asia/Shanghai VOLUME /work WORKDIR /word ADD bin/etcd /usr/sbin/etcd ADD bin/etcdctl /usr/sbin/etcdctl EXPOSE 2379 2380 CMD etcd EOF

3.构建etcd镜像，启动并测试是否能够运行

# 构建镜像
    docker build -t etcd:3.4.9-alpine .
    # 启动etcd
    docker run --rm -ti etcd:3.4.9-alpine

启动成功同如下页面  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQyNDEzMDEx_size_16_color_FFFFFF_t_70]

# 生成ssl证书 #

## 安装cfssl证书生成工具 ##

wget -O /usr/local/sbin/certinfo https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 && chmod +x /usr/local/sbin/certinfo
    wget -O /usr/local/sbin/cfssl https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 && chmod +x /usr/local/sbin/cfssl
    wget -O /usr/local/sbin/cfssljson https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 && chmod +x /usr/local/sbin/cfssljson

## 编写生成证书的配置文件 ##

1.创建证书文件存放的目录

# 后续生成的证书都会在ssl目录
    mkdir ssl && cd ssl

2.目录结构如下  
![在这里插入图片描述][20210410220809360.png]  
3.打印默认证书配置文件

cfssl print-defaults config > ca-config.json

4.修改ca-config.json中的配置

cat > ca-config.json <<EOF { "signing": { "default": { "expiry": "876000h" }, "profiles": { "etcd": { "expiry": "876000h", "usages": [ "signing", "key encipherment", "server auth", "client auth" ] } } } } EOF

## 编写ca根证书配置文件 ##

1.打印默认ca证书配置

cfssl print-defaults csr > ca-csr.json

2.修改ca-csr.json配置文件

cat > ca-csr.json <<EOF { "CN": "www.etcd.com.cn", "key": { "algo": "ecdsa", "size": 256 }, "names": [ { "City": "CN", "Country": "ChengDu", "Author": "Lhstack" } ] } EOF

3.生成ca-key.pem,ca.pem文件

cfssl gencert -initca ca-csr.json | cfssljson -bare ca

## 编写服务端认证证书配置文件 ##

1.打印默认server端证书配置

cfssl print-defaults csr > server-csr.json

2.修改server-csr.json配置文件  
hosts为etcd每一台集群的ip地址，如果后续要扩容，可以多预留几个ip

cat > server-csr.json <<EOF { "CN": "www.etcd.com.cn", "hosts":[ "10.100.110.80", "10.100.110.81", "10.100.110.82" ], "key": { "algo": "ecdsa", "size": 256 }, "names": [ { "City": "CN", "Country": "ChengDu", "Author": "Lhstack" } ] } EOF

3.生成server端认证证书server.pem,server-key.pem

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=etcd server-csr.json | cfssljson -bare server

## 编写客户端证书配置 ##

1.打印默认server端证书配置

cfssl print-defaults csr > client-csr.json

2.修改client-csr.json配置文件

cat > client-csr.json <<EOF { "CN": "www.etcd.com.cn", "key": { "algo": "ecdsa", "size": 256 }, "names": [ { "City": "CN", "Country": "ChengDu", "Author": "Lhstack" } ] } EOF

3.生成客户端端认证证书client.pem,client-key.pem

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=etcd client-csr.json | cfssljson -bare client

# 编写docker-compose.yml文件 #

## 返回上级目录 ##

cd ..

![在这里插入图片描述][20210410231233579.png]

## docker-compose.yml如下 ##

version: '3'
    services:
      etcd-1:
        image: etcd:3.4.9-alpine
        build: .
        container_name: etcd-1
        volumes:
        - ./ssl:/opt/etcd/ssl
        - ./data/etcd-1:/opt/etcd/data
        command:
        - sh
        - -c
        - | etcd -name etcd-1 --cert-file=/opt/etcd/ssl/server.pem \ --listen-peer-urls https://10.100.110.80:2380 --listen-client-urls https://10.100.110.80:2379,http://10.100.110.80:2379 \ --key-file=/opt/etcd/ssl/server-key.pem --peer-cert-file=/opt/etcd/ssl/server.pem \ --peer-key-file=/opt/etcd/ssl/server-key.pem --initial-advertise-peer-urls https://10.100.110.80:2380 \ --advertise-client-urls https://10.100.110.80:2379 --initial-cluster-token etcd-cluster \ --trusted-ca-file=/opt/etcd/ssl/ca.pem --peer-trusted-ca-file=/opt/etcd/ssl/ca.pem \ --initial-cluster etcd-1=https://10.100.110.80:2380,etcd-2=https://10.100.110.81:2380,etcd-3=https://10.100.110.82:2380 \ --enable-v2=true \ --data-dir=/opt/etcd/data --initial-cluster-state new
        networks:
          etcd:
            ipv4_address: "10.100.110.80"
      etcd-2:
        image: etcd:3.4.9-alpine
        container_name: etcd-2
        volumes:
        - ./ssl:/opt/etcd/ssl
        - ./data/etcd-2:/opt/etcd/data
        command:
        - sh
        - -c
        - | etcd -name etcd-2 --cert-file=/opt/etcd/ssl/server.pem \ --listen-peer-urls https://10.100.110.81:2380 --listen-client-urls https://10.100.110.81:2379,http://10.100.110.81:2379 \ --key-file=/opt/etcd/ssl/server-key.pem --peer-cert-file=/opt/etcd/ssl/server.pem \ --peer-key-file=/opt/etcd/ssl/server-key.pem --initial-advertise-peer-urls https://10.100.110.81:2380 \ --advertise-client-urls https://10.100.110.81:2379 --initial-cluster-token etcd-cluster \ --trusted-ca-file=/opt/etcd/ssl/ca.pem --peer-trusted-ca-file=/opt/etcd/ssl/ca.pem \ --initial-cluster etcd-1=https://10.100.110.80:2380,etcd-2=https://10.100.110.81:2380,etcd-3=https://10.100.110.82:2380 \ --enable-v2=true \ --data-dir=/opt/etcd/data --initial-cluster-state new
        networks:
          etcd:
            ipv4_address: "10.100.110.81"
      etcd-3:
        image: etcd:3.4.9-alpine
        container_name: etcd-3
        volumes:
        - ./ssl:/opt/etcd/ssl
        - ./data/etcd-3:/opt/etcd/data
        command:
        - sh
        - -c
        - | etcd -name etcd-3 --cert-file=/opt/etcd/ssl/server.pem \ --listen-peer-urls https://10.100.110.82:2380 --listen-client-urls https://10.100.110.82:2379,http://10.100.110.82:2379 \ --key-file=/opt/etcd/ssl/server-key.pem --peer-cert-file=/opt/etcd/ssl/server.pem \ --peer-key-file=/opt/etcd/ssl/server-key.pem --initial-advertise-peer-urls https://10.100.110.82:2380 \ --advertise-client-urls https://10.100.110.82:2379 --initial-cluster-token etcd-cluster \ --trusted-ca-file=/opt/etcd/ssl/ca.pem --peer-trusted-ca-file=/opt/etcd/ssl/ca.pem \ --initial-cluster etcd-1=https://10.100.110.80:2380,etcd-2=https://10.100.110.81:2380,etcd-3=https://10.100.110.82:2380 \ --enable-v2=true \ --data-dir=/opt/etcd/data --initial-cluster-state new
        networks:
          etcd:
            ipv4_address: "10.100.110.82"
    networks:
      etcd:
        driver: bridge
        ipam:
          driver: default
          config:
          - subnet: 10.100.0.0/16

# 运行etcd集群 #

目录结构如下  
![在这里插入图片描述][2021041023141370.png]  
执行docker-compose

docker-compose up -d

运行结果如下  
![在这里插入图片描述][20210410231450472.png]

# 查看集群状态 #

进入任意一个etcd集群节点

docker exec -ti etcd-1 sh

查看集群状态

etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/client.pem --key=/opt/etcd/ssl/client-key.pem --endpoints="https://10.100.110.80:2379,https://10.100.110.81:2379,https://10.100.110.82:2379" endpoint status -w table

将etcdctl封装成脚本使用

cat > etcdctl.sh << EOF #!/bin/sh SSL_DIR=/opt/etcd/ssl ENDPOINTS="https://10.100.110.80:2379,https://10.100.110.81:2379,https://10.100.110.82:2379" etcdctl --cacert=\$SSL_DIR/ca.pem --cert=\$SSL_DIR/client.pem --key=\$SSL_DIR/client-key.pem --endpoints=\$ENDPOINTS \$@ EOF

使用脚本查看集群状态

sh etcdctl.sh endpoint status -w table

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQyNDEzMDEx_size_16_color_FFFFFF_t_70 1]  
使用脚本添加数据

sh etcdctl.sh put msg hello

使用脚本获取数据

sh etcdctl.sh get msg

使用脚本删除数据

sh etcdctl.sh del msg

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQyNDEzMDEx_size_16_color_FFFFFF_t_70]: /images/20221022/5aa64d549e684841a4a403e063219f73.png
[20210410220809360.png]: /images/20221022/174424d778d14f4cac575cfe64f8df19.png
[20210410231233579.png]: /images/20221022/3bc6dd85bad74ab0a5b5b05e01332d8e.png
[2021041023141370.png]: /images/20221022/9ed48acbddf34857ba74b91131140121.png
[20210410231450472.png]: /images/20221022/55d6e8d8b62842eca8ba1d28534e83b9.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQyNDEzMDEx_size_16_color_FFFFFF_t_70 1]: /images/20221022/580b184aecb749e4a0c54e8dd1ccb73d.png