如何在Ubuntu 18.04上使用Ansible设置和保护etcd群集

超、凢脫俗 2022-11-28 15:28 253阅读 0赞

*The author selected the [Wikimedia Foundation][] to receive a donation as part of the [Write for DOnations][] program.*

*作者选择[Wikimedia Foundation][]接受捐赠，这是[Write for DOnations][]计划的一部分。*

### 介绍 **(**Introduction**)** ###

[etcd][] is a distributed key-value store relied on by many platforms and tools, including [Kubernetes][], [Vulcand][], and [Doorman][]. Within Kubernetes, etcd is used as a global configuration store that stores the state of the cluster. Knowing how to administer etcd is essential to administering a Kubernetes cluster. Whilst there are many managed Kubernetes offerings, also known as *Kubernetes-as-a-Service*, that remove this administrative burden away from you, many companies still choose to run self-managed Kubernetes clusters on-premises because of the flexibility it brings.

[etcd][]是许多平台和工具(包括[Kubernetes][] ， [Vulcand][]和[Doorman)][Doorman]所依赖的分布式键值存储。 在Kubernetes中，etcd用作存储群集状态的全局配置存储。 知道如何管理etcd对于管理Kubernetes集群至关重要。 尽管有许多托管的Kubernetes产品(也称为*Kubernetes即服务* )可以减轻您的管理负担，但许多公司仍选择在本地运行自行管理的Kubernetes集群，因为它带来了灵活性。

The first half of this article will guide you through setting up a 3-node etcd cluster on Ubuntu 18.04 servers. The second half will focus on securing the cluster using [*Transport Layer Security*, or *TLS*][Transport Layer Security_ or _TLS]. To run each setup in an automated manner, we will use [Ansible][] throughout. Ansible is a *configuration management* tool similar to [Puppet][], [Chef][], and [SaltStack][]; it allows us to define each setup step in a declarative manner, inside files called *playbooks*.

本文的前半部分将指导您在Ubuntu 18.04服务器上设置3节点etcd集群。 下半年将重点介绍使用[*传输层安全性* ( *TLS)*][Transport Layer Security_ or _TLS]保护群集的[*安全*][Transport Layer Security_ or _TLS] 。 要以自动化方式运行每个设置，我们将在整个过程中使用[Ansible][] 。 Ansible是类似于[Puppet][] ， [Chef][]和[SaltStack][]的*配置管理*工具； 它允许我们以声明的方式在称为*playbooks的*文件内定义每个设置步骤。

At the end of this tutorial, you will have a secure 3-node etcd cluster running on your servers. You will also have an Ansible playbook that allows you to repeatedly and consistently recreate the same setup on a fresh set of servers.

在本教程的最后，您将在服务器上运行一个安全的3节点etcd集群。 您还将拥有Ansible剧本，可让您在全新的服务器上反复一致地重新创建相同的设置。

## 先决条件 **(**Prerequisites**)** ##

Before you begin this guide you’ll need the following:

在开始本指南之前，您需要满足以下条件：

*  [Python][], `pip`, and the [`pyOpenSSL`][pyOpenSSL] package installed on your local machine. To learn how to install Python3, pip, and Python packages, refer to [How To Install Python 3 and Set Up a Local Programming Environment on Ubuntu 18.04][].
    
    在本地计算机上安装的[Python][] ， `pip`和[`pyOpenSSL`][pyOpenSSL]软件包。 要了解如何安装Python3，pip和Python软件包，请参阅[如何在Ubuntu 18.04上安装Python 3和设置本地编程环境][How To Install Python 3 and Set Up a Local Programming Environment on Ubuntu 18.04] 。
 *  Three Ubuntu 18.04 servers on the same local network, with at least 2GB of RAM and root SSH access. You should also configure the servers to have the hostnames **etcd1**, **etcd2**, and **etcd3**. The steps outlined in this article would work on any generic server, not necessarily DigitalOcean Droplets. However, if you’d like to host your servers on DigitalOcean, you can follow the [How to Create a Droplet from the DigitalOcean Control Panel][] guide to fulfil this requirement. Note that you must enable the **Private Networking** option when creating your Droplet. To enable private networking on existing Droplets, refer to [How to Enable Private Networking on Droplets][].
    
    同一本地网络上的三台Ubuntu 18.04服务器，至少具有2GB的RAM和root SSH访问权限。 您还应该将服务器配置为具有主机名**etcd1** ， **etcd2**和**etcd3** 。 本文概述的步骤将适用于任何通用服务器，而不必适用于DigitalOcean Droplet。 但是，如果要在DigitalOcean上托管服务器，则可以按照“ [如何从DigitalOcean控制面板中创建液滴”][How to Create a Droplet from the DigitalOcean Control Panel]指南来满足此要求。 请注意，创建Droplet时必须启用“ **专用网络”**选项。 要在现有Droplet上启用专用网络，请参阅[如何在Droplet上启用专用网络][How to Enable Private Networking on Droplets] 。

**Warning:** Since the purpose of this article is to provide an introduction to setting up an etcd cluster on a private network, the three Ubuntu 18.04 servers in this setup were not tested with a firewall and are accessed as the **root** user. In a production setup, any node exposed to the public internet would require a firewall and a sudo user to adhere to security best practices. For more information, check out the [Initial Server Setup with Ubuntu 18.04][] tutorial.

**警告：**由于本文的目的是为了介绍如何在专用网络上设置etcd群集，因此未使用防火墙测试此设置中的三台Ubuntu 18.04服务器，并以**root**用户身份对其进行访问。 在生产设置中，任何暴露于公共Internet的节点都将需要防火墙和sudo用户来遵守安全最佳实践。 有关更多信息，请查看《 [Ubuntu 18.04][Initial Server Setup with Ubuntu 18.04]的[初始服务器设置》][Initial Server Setup with Ubuntu 18.04]教程。

*  An SSH key pair allowing your local machine access to the **etcd1**, **etcd2**, and **etcd3** servers. If you do not know what SSH is, or do not have an SSH key pair, you can learn about it by reading [SSH Essentials: Working with SSH Servers, Clients, and Keys][SSH Essentials_ Working with SSH Servers_ Clients_ and Keys].
    
    SSH密钥对，允许您的本地计算机访问**etcd1** ， **etcd2**和**etcd3**服务器。 如果您不知道什么是SSH或没有SSH密钥对，则可以阅读[SSH Essentials：使用SSH服务器，客户端和密钥][SSH Essentials_ Working with SSH Servers_ Clients_ and Keys]来了解它。
 *  Ansible installed on your local machine. For example, if you’re running Ubuntu 18.04, you can install Ansible by following **Step 1** of the [How to Install and Configure Ansible on Ubuntu 18.04][] article. This will make the `ansible` and `ansible-playbook` commands available on your machine. You may also want to keep this [How to Use Ansible: A Reference Guide][How to Use Ansible_ A Reference Guide] handy. The commands in this tutorial should work with Ansible v2.x; we have tested it on Ansible v2.9.7 running Python v3.8.2.
    
    Ansible安装在本地计算机上。 例如，如果您正在运行Ubuntu 18.04，则可以按照[如何在Ubuntu 18.04上安装和配置Ansible][How to Install and Configure Ansible on Ubuntu 18.04]文章中的**步骤1** [来安装Ansible][How to Install and Configure Ansible on Ubuntu 18.04] 。 这将使`ansible`和`ansible-playbook`命令在您的计算机上可用。 您可能还希望保留此“ [如何使用Ansible：参考指南”][How to Use Ansible_ A Reference Guide] 。 本教程中的命令应与Ansible v2.x一起使用。 我们已经在运行Python v3.8.2的Ansible v2.9.7上对其进行了测试。

## 步骤1 —为控制节点配置Ansible **(**Step 1 — Configuring Ansible for the Control Node**)** ##

Ansible is a tool used to manage servers. The servers Ansible is managing are called the *managed nodes*, and the machine that is running Ansible is called the *control node*. Ansible works by using the SSH keys on the control node to gain access to the managed nodes. Once an SSH session is established, Ansible will run a set of scripts to provision and configure the managed nodes. In this step, we will test that we are able to use Ansible to connect to the managed nodes and run the [`hostname` command][hostname_ command].

Ansible是用于管理服务器的工具。 Ansible管理的服务器称为受*管节点* ，运行Ansible的计算机称为*控制节点* 。 Ansible通过使用控制节点上的SSH密钥来访问受管节点来工作。 建立SSH会话后，Ansible将运行一组脚本来配置和配置受管节点。 在此步骤中，我们将测试是否可以使用Ansible连接到受管节点并运行[`hostname`命令][hostname_ command] 。

A typical day for a system administrator may involve managing different sets of nodes. For instance, you may use Ansible to provision some new servers, but later on use it to reconfigure another set of servers. To allow administrators to better organize the set of managed nodes, Ansible provides the concept of *host inventory* (or *inventory* for short). You can define every node that you wish to manage with Ansible inside an *inventory file*, and organize them into *groups*. Then, when running the `ansible` and `ansible-playbook` commands, you can specify which hosts or groups the command applies to.

对于系统管理员而言，典型的一天可能涉及管理不同的节点集。 例如，您可以使用Ansible来配置一些新服务器，但稍后使用它来重新配置另一组服务器。 为了使管理员能够更好地组织受管节点集，Ansible提供了*主机清单* (或简称*清单* )的概念。 您可以在*清单文件中*定义要使用Ansible管理的每个节点，并将它们组织成*组* 。 然后，在运行`ansible`和`ansible-playbook`命令时，可以指定该命令适用于哪些主机或组。

By default, Ansible reads the inventory file from `/etc/ansible/hosts`; however, we can specify a different inventory file by using the `--inventory` flag (or `-i` for short).

默认情况下，Ansible从`/etc/ansible/hosts`读取清单文件； 但是，我们可以使用`--inventory`标志(或简称`-i` )来指定其他清单文件。

To get started, create a new directory on your local machine (the **control node**) to house all the files for this tutorial:

首先，请在本地计算机( **控制节点** )上创建一个新目录，以容纳本教程的所有文件：

*  mkdir -p $HOME/playground/etcd-ansible
    
    mkdir -p $ HOME /操场/ etcd-ansible

Then, enter into the directory you just created:

然后，进入刚刚创建的目录：

*  cd $HOME/playground/etcd-ansible
    
    cd $ HOME / playground / etcd-ansible

Inside the directory, create and open a blank inventory file named `hosts` using your editor:

在目录中，使用编辑器创建并打开一个名为`hosts`的空白清单文件：

*  nano $HOME/playground/etcd-ansible/hosts
    
    nano $ HOME / playground / etcd-ansible / hosts

Inside the `hosts` file, list out each of your managed nodes in the following format, replacing the public IP addresses highlighted with the actual public IP addresses of your servers:

在`hosts`文件中，以以下格式列出每个受管节点，将突出显示的公用IP地址替换为服务器的实际公用IP地址：

~/playground/etcd-ansible/hosts

〜/ playground / etcd-ansible / hosts

[etcd]
    etcd1 ansible_host=etcd1_public_ip  ansible_user=root
    etcd2 ansible_host=etcd2_public_ip  ansible_user=root
    etcd3 ansible_host=etcd3_public_ip  ansible_user=root

The `[etcd]` line defines a group called `etcd`. Under the group definition, we list all our managed nodes. Each line begins with an alias (e.g., `etcd1`), which allows us to refer to each host using an easy-to-remember name instead of a long IP address. The `ansible_host` and `ansible_user` are Ansible *variables*. In this case, they are used to provide Ansible with the public IP addresses and SSH usernames to use when connecting via SSH.

`[etcd]`行定义了一个名为`etcd`的组。 在组定义下，我们列出了所有受管节点。 每行以一个别名(例如`etcd1` ) `etcd1` ，这使我们可以使用易于记忆的名称而不是长IP地址来引用每个主机。 `ansible_host`和`ansible_user`是Ansible *变量* 。 在这种情况下，它们用于为Ansible提供通过SSH连接时使用的公共IP地址和SSH用户名。

To ensure Ansible is able to connect with our managed nodes, we can test for connectivity by using Ansible to run the `hostname` command on each of the hosts within the `etcd` group:

为了确保Ansible能够与我们的受管节点连接，我们可以使用Ansible在`etcd`组中的每个`hostname`上运行`hostname`命令来测试连接性：

*  ansible etcd -i hosts -m command -a hostname
    
    ansible etcd -i hosts -m命令-a主机名

Let us break down this command to learn what each part means:

让我们分解此命令以了解每个部分的含义：

*  `etcd`: specifies the *host pattern* to use to determine which hosts from the inventory are being managed with this command. Here, we are using the group name as the host pattern.
    
    `etcd` ：指定*主机模式* ，该*主机模式*用于确定通过此命令管理清单中的哪些主机。 在这里，我们使用组名作为主机模式。
 *  `-i hosts`: specifies the inventory file to use.
    
    `-i hosts` ：指定要使用的清单文件。
 *  `-m command`: the functionality behind Ansible is provided by *modules*. The `command` module takes the argument passed in and executes it as a command on each of the managed nodes. This tutorial will introduce a few more Ansible modules as we progress.
    
    `-m command` ：Ansible背后的功能由*模块*提供。 `command`模块采用传入的参数，并在每个受管节点上将其作为命令执行。 随着我们的进步，本教程将介绍更多的Ansible模块。
 *  `-a hostname`: the argument to pass into the module. The number and types of arguments depend on the module.
    
    `-a hostname` ：传递给模块的参数。 参数的数量和类型取决于模块。

After running the command, you will find the following output, which means Ansible is configured correctly:

运行该命令后，您将找到以下输出，这意味着Ansible已正确配置：

Each command that Ansible runs is called a *task*. Using `ansible` on the command line to run tasks is called running *ad-hoc* commands. The upside of ad-hoc commands is that they are quick and require little setup; the downside is that they run manually, and thus cannot be committed to a version control system like [Git][].

Ansible运行的每个命令称为一个*任务* 。 在命令行上使用`ansible`来运行任务称为运行*临时*命令。 临时命令的优点是它们快速且几乎不需要设置。 缺点是它们是手动运行的，因此不能提交给像[Git][]这样的版本控制系统。

A slight improvement would be to write a shell script and run our commands using Ansible’s [`script` module][script_ module]. This would allow us to record the configuration steps we took into version control. However, shell scripts are *imperative*, which means we are responsible for figuring out the commands to run (the “how"s) to configure the system to the desired state. Ansible, on the other hand, advocates for a *declarative* approach, where we define "what” the desired state of our server should be inside configuration files, and Ansible is responsible for getting the server to that desired state.

稍有改进就是编写一个Shell脚本并使用Ansible的[`script`模块][script_ module]运行我们的命令。 这将使我们能够记录进行版本控制的配置步骤。 但是，shell脚本是*必不可少的* ，这意味着我们有责任弄清楚将系统配置为所需状态所需的运行命令(“方式”)；另一方面，Ansible提倡采用*声明式*方法，在配置文件中定义“什么”服务器的期望状态，Ansible负责使服务器达到该期望状态。

The declarative approach is preferred because the intent of the configuration file is immediately conveyed, meaning it’s easier to understand and maintain. It also places the onus of handling edge cases on Ansible instead of the administrator, saving us a lot of work.

声明性方法是首选，因为配置文件的意图会立即传达，这意味着更易于理解和维护。 它还将处理边缘案例的责任放在了Ansible而非管理员身上，从而为我们节省了很多工作。

Now that you have configured the Ansible control node to communicate with the managed nodes, in the next step, we will introduce you to Ansible *playbooks*, which allow you to specify tasks in a declarative way.

现在您已经配置了Ansible控制节点与受管节点进行通信，在下一步中，我们将向您介绍Ansible *剧本* ，使您可以以声明的方式指定任务。

## 第2步-使用Ansible剧本获取受管节点的主机名 **(**Step 2 — Getting the Hostnames of Managed Nodes with Ansible Playbooks**)** ##

In this step, we will replicate what was done in Step 1—printing out the hostnames of the managed nodes—but instead of running ad-hoc tasks, we will define each task declaratively as an Ansible playbook and run it. The purpose of this step is to demonstrate how Ansible playbooks work; we will carry out much more substantial tasks with playbooks in later steps.

在此步骤中，我们将复制在步骤1中所做的工作(打印出受管节点的主机名)，但是不是运行临时任务，而是声明性地将每个任务定义为Ansible剧本并运行它。 此步骤的目的是演示Ansible剧本的工作方式； 我们将在以后的步骤中使用剧本执行更多实质性任务。

Inside your project directory, create a new file named `playbook.yaml` using your editor:

在项目目录中，使用编辑器创建一个名为`playbook.yaml`的新文件：

*  nano $HOME/playground/etcd-ansible/playbook.yaml
    
    纳米$ HOME / playground / etcd-ansible / playbook.yaml

Inside `playbook.yaml`, add the following lines:

在`playbook.yaml`内部，添加以下行：

~/playground/etcd-ansible/playbook.yaml

〜/ playground / etcd-ansible / playbook.yaml

- hosts: etcd
      tasks:
        - name: "Retrieve hostname"
          command: hostname
          register: output
        - name: "Print hostname"
          debug: var=output.stdout_lines

Close and save the `playbook.yaml` file by pressing `CTRL+X` followed by `Y`.

通过按`CTRL+X`然后按`Y`关闭并保存`playbook.yaml`文件。

The playbook contains a list of *plays*; each play contains a list of tasks that should be run on all hosts matching the host pattern specified by the `hosts` key. In this playbook, we have one play that contains two tasks. The first task runs the `hostname` command using the `command` module and registers the output to a variable named `output`. In the second task, we use the [`debug`][debug] module to print out the `stdout_lines` property of the `output` variable.

该剧本包含*戏剧*清单; 每个播放包含一个任务列表，应在与`hosts`键指定的主机模式匹配的所有主机上运行。 在这本剧本中，我们有一部戏剧包含两个任务。 第一个任务使用`command`模块运行`hostname`命令，并将输出注册到名为`output`的变量中。 在第二个任务中，我们使用[`debug`][debug]模块打印`output`变量的`stdout_lines`属性。

We can now run this playbook using the `ansible-playbook` command:

现在，我们可以使用`ansible-playbook`命令运行此剧本：

*  ansible-playbook -i hosts playbook.yaml
    
    ansible-playbook -i托管playbook.yaml

You will find the following output, which means your playbook is working correctly:

您将找到以下输出，这意味着您的剧本正在正常运行：

Output PLAY [etcd] ***********************************************************************************************************************
    
    TASK [Gathering Facts] ************************************************************************************************************
    ok: [etcd2]
    ok: [etcd3]
    ok: [etcd1]
    
    TASK [Retrieve hostname] **********************************************************************************************************
    changed: [etcd2]
    changed: [etcd3]
    changed: [etcd1]
    
    TASK [Print hostname] *************************************************************************************************************
    ok: [etcd1] => {
        "output.stdout_lines": [
            "etcd1"
        ]
    }
    ok: [etcd2] => {
        "output.stdout_lines": [
            "etcd2"
        ]
    }
    ok: [etcd3] => {
        "output.stdout_lines": [
            "etcd3"
        ]
    }
    
    PLAY RECAP ************************************************************************************************************************
    etcd1                      : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
    etcd2                      : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
    etcd3                      : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

**Note:** `ansible-playbook` sometimes uses `cowsay` as a playful way to print the headings. If you find a lot of ASCII-art cows printed on your terminal, now you know why. To disable this feature, set the `ANSIBLE_NOCOWS` environment variable to `1` prior to running `ansible-playbook` by running `export ANSIBLE_NOCOWS=1` in your shell.

**注意：** `ansible-playbook`有时会使用`cowsay`作为打印标题的好玩方式。 如果您发现终端上印有很多ASCII艺术字的母牛，现在您知道原因了。 要禁用此功能，请在外壳程序中运行`export ANSIBLE_NOCOWS=1`在运行`export ANSIBLE_NOCOWS=1` `ansible-playbook`之前将`ANSIBLE_NOCOWS`环境变量设置为`1` 。

In this step, we’ve moved from running imperative ad-hoc tasks to running declarative playbooks. In the next step, we will replace these two demo tasks with tasks that will set up our etcd cluster.

在这一步中，我们已经从运行命令式临时任务转变为运行声明式剧本。 在下一步中，我们将用设置etcd集群的任务替换这两个演示任务。

## 第3步-在受管节点上安装etcd **(**Step 3 — Installing etcd on the Managed Nodes**)** ##

In this step, we will show you the commands to install `etcd` manually and demonstrate how to translate these same commands into tasks inside our Ansible playbook.

在这一步中，我们将向您`etcd`手动安装`etcd`的命令，并演示如何将这些相同的命令转换为Ansible剧本中的任务。

`etcd` and its client `etcdctl` are available as binaries, which we’ll download, extract, and move to a directory that’s part of the `PATH` environment variable. When configured manually, these are the steps we would take on each of the managed nodes:

`etcd`及其客户端`etcdctl`可作为二进制文件使用，我们将下载它们，将其提取并移动到`PATH`环境变量的一部分中。 手动配置后，以下是我们将在每个受管节点上执行的步骤：

*  mkdir -p /opt/etcd/bin
    
    mkdir -p / opt / etcd / bin
 *  cd /opt/etcd/bin
    
    cd / opt / etcd / bin
 *  wget -qO- https://storage.googleapis.com/etcd/v3.3.13/etcd-v3.3.13-linux-amd64.tar.gz | tar --extract --gzip --strip-components=1
    
    wget -qO- https://storage.googleapis.com/etcd/v 3.3.13 / etcd-v 3.3.13 -linux-amd64.tar.gz | tar-提取--gzip --strip-components = 1
 *  echo 'export PATH="$PATH:/opt/etcd/bin"' >> ~/.profile
    
    echo'export PATH =“ $ PATH：/ opt / etcd / bin”'>>〜/ .profile
 *  echo 'export ETCDCTL\_API=3" >> ~/.profile
    
    echo'export ETCDCTL\_API = 3“ >>〜/ .profile

The first four commands download and extract the binaries to the `/opt/etcd/bin/` directory. By default, the `etcdctl` client will use API v2 to communicate with the `etcd` server. Since we are running etcd v3.x, the last command sets the `ETCDCTL_API` environment variable to `3`.

前四个命令将二进制文件下载并解压缩到`/opt/etcd/bin/`目录。 默认情况下， `etcdctl`客户端将使用API v2与`etcd`服务器进行通信。 由于我们正在运行etcd v3.x，因此最后一个命令将`ETCDCTL_API`环境变量设置为`3` 。

**Note:** Here, we are using etcd v3.3.13 built for a machine with processors that use the AMD64 instruction set. You can find binaries for other systems and other versions on the the official GitHub [Releases][] page.

**注意：**在这里，我们使用为机器上使用AMD64指令集的计算机而构建的etcd v3.3.13。 您可以在官方的GitHub [Releases][]页面上找到其他系统和其他版本的二进制文件。

To replicate the same steps in a standardized format, we can add tasks to our playbook. Open the `playbook.yaml` playbook file in your editor:

要以标准化格式复制相同的步骤，我们可以将任务添加到剧本中。 在编辑器中打开`playbook.yaml`剧本文件：

*  nano $HOME/playground/etcd-ansible/playbook.yaml
    
    纳米$ HOME / playground / etcd-ansible / playbook.yaml

Replace the entirety of the `playbook.yaml` file with the following contents:

用以下内容替换整个`playbook.yaml`文件：

~/playground/etcd-ansible/playbook.yaml

〜/ playground / etcd-ansible / playbook.yaml

- hosts: etcd
      become: True
      tasks:
        - name: "Create directory for etcd binaries"
          file:
            path: /opt/etcd/bin
            state: directory
            owner: root
            group: root
            mode: 0700
        - name: "Download the tarball into the /tmp directory"
          get_url:
            url: https://storage.googleapis.com/etcd/v3.3.13/etcd-v3.3.13-linux-amd64.tar.gz
            dest: /tmp/etcd.tar.gz
            owner: root
            group: root
            mode: 0600
            force: True
        - name: "Extract the contents of the tarball"
          unarchive:
            src: /tmp/etcd.tar.gz
            dest: /opt/etcd/bin/
            owner: root
            group: root
            mode: 0600
            extra_opts:
              - --strip-components=1
            decrypt: True
            remote_src: True
        - name: "Set permissions for etcd"
          file:
            path: /opt/etcd/bin/etcd
            state: file
            owner: root
            group: root
            mode: 0700
        - name: "Set permissions for etcdctl"
          file:
            path: /opt/etcd/bin/etcdctl
            state: file
            owner: root
            group: root
            mode: 0700
        - name: "Add /opt/etcd/bin/ to the $PATH environment variable"
          lineinfile:
            path: /etc/profile
            line: export PATH="$PATH:/opt/etcd/bin"
            state: present
            create: True
            insertafter: EOF
        - name: "Set the ETCDCTL_API environment variable to 3"
          lineinfile:
            path: /etc/profile
            line: export ETCDCTL_API=3
            state: present
            create: True
            insertafter: EOF

Each task uses a module; for this set of tasks, we are making use of the following modules:

每个任务使用一个模块； 对于这组任务，我们使用以下模块：

*  [`file`][file]: to create the `/opt/etcd/bin` directory, and to later set the file permissions for the `etcd` and `etcdctl` binaries.
    
    [`file`][file] ：创建`/opt/etcd/bin`目录，并在以后设置`etcd`和`etcdctl`二进制文件的文件权限。
 *  [`get_url`][get_url]: to download the gzipped tarball onto the managed nodes.
    
    [`get_url`][get_url] ：将压缩的tarball下载到受管节点上。
 *  [`unarchive`][unarchive]: to extract and unpack the `etcd` and `etcdctl` binaries from the gzipped tarball.
    
    [`unarchive`][unarchive] ：从压缩的tarball中提取并解压缩`etcd`和`etcdctl`二进制文件。
 *  [`lineinfile`][lineinfile]: to add an entry into the `.profile` file.
    
    [`lineinfile`][lineinfile] ：在`.profile`文件中添加一个条目。

To apply these changes, close and save the `playbook.yaml` file by pressing `CTRL+X` followed by `Y`. Then, on the terminal, run the same `ansible-playbook` command again:

要应用这些更改，请按`CTRL+X`然后按`Y` ，关闭并保存`playbook.yaml`文件。 然后，在终端上，再次运行相同的`ansible-playbook`命令：

*  ansible-playbook -i hosts playbook.yaml
    
    ansible-playbook -i托管playbook.yaml

The `PLAY RECAP` section of the output will show only `ok` and `changed`:

输出的`PLAY RECAP`部分将仅显示`ok`并已`changed` ：

Output ...
    PLAY RECAP ************************************************************************************************************************
    etcd1                      : ok=8    changed=7    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
    etcd2                      : ok=8    changed=7    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
    etcd3                      : ok=8    changed=7    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

To confirm a correct installation of etcd, manually SSH into one of the managed nodes and run `etcd` and `etcdctl`:

要确认正确安装了etcd，请手动将SSH SSH到一个受管节点中，然后运行`etcd`和`etcdctl` ：

*  ssh root@etcd1\_public\_ip
    
    ssh root @ etcd1\_public\_ip

`etcd1_public_ip` is the public IP addresses of the server named **etcd1**. Once you have gained SSH access, run `etcd --version` to print out the version of etcd installed:

`etcd1_public_ip`是名为**etcd1**的服务器的公共IP地址。 获得SSH访问权限后，运行`etcd --version`打印出已安装的etcd版本：

*  etcd --version
    
    etcd-版本

You will find output similar to what’s shown in the following, which means the `etcd` binary is successfully installed:

您将找到类似于以下内容的输出，这意味着`etcd`二进制文件已成功安装：

Output etcd Version: 3.3.13
    Git SHA: 98d3084
    Go Version: go1.10.8
    Go OS/Arch: linux/amd64

To confirm `etcdctl` is successfully installed, run `etcdctl version`:

要确认`etcdctl`成功安装后，运行`etcdctl version` ：

*  etcdctl version
    
    etcdctl版本

You will find output similar to the following:

您将找到类似于以下内容的输出：

Output etcdctl version: 3.3.13
    API version: 3.3

Note that the output says `API version: 3.3`, which also confirms that our `ETCDCTL_API` environment variable was set correctly.

请注意，输出显示`API version: 3.3` ，这还确认我们的`ETCDCTL_API`环境变量设置正确。

Exit out of the **etcd1** server to return to your local environment.

退出**etcd1**服务器，以返回到本地环境。

We have now successfully installed `etcd` and `etcdctl` on all of our managed nodes. In the next step, we will add more tasks to our play to run etcd as a background service.

现在，我们已经在所有受管节点上成功安装了`etcd`和`etcdctl` 。 在下一步中，我们将在游戏中添加更多任务以将etcd作为后台服务运行。

## 步骤4 —为etcd创建一个单位文件 **(**Step 4 — Creating a Unit File for etcd**)** ##

The quickest way to run etcd with Ansible may appear to be to use the `command` module to run `/opt/etcd/bin/etcd`. However, this will not work because it will make `etcd` run as a foreground process. Using the `command` module will cause Ansible to hang as it waits for the `etcd` command to return, which it never will. So in this step, we are going to update our playbook to run our `etcd` binary as a background *service* instead.

使用Ansible运行etcd的最快方法似乎是使用`command`模块来运行`/opt/etcd/bin/etcd` 。 但是，这将不起作用，因为它将使`etcd`作为前台进程运行。 使用`command`模块将导致Ansible挂起，因为它等待`etcd`命令返回，而它永远不会挂起。 因此，在这一步中，我们将更新剧本，以将`etcd`二进制文件作为后台*服务*运行。

Ubuntu 18.04 uses [*systemd*][systemd] as its *init system*, which means we can create new services by writing *unit files* and placing them inside the `/etc/systemd/system/` directory.

Ubuntu 18.04使用[*systemd*][systemd]作为其*初始系统* ，这意味着我们可以通过编写*单位文件*并将它们放置在`/etc/systemd/system/`目录中来创建新服务。

First, inside our project directory, create a new directory named `files/`:

首先，在我们的项目目录中，创建一个名为`files/`的新目录：

*  mkdir files
    
    mkdir文件

Then, using your editor, create a new file named `etcd.service` within that directory:

然后，使用编辑器在该目录中创建一个名为`etcd.service`的新文件：

*  nano files/etcd.service
    
    纳米文件/ etcd.service

Next, copy the following code block into the `files/etcd.service` file:

接下来，将以下代码块复制到`files/etcd.service`文件中：

~/playground/etcd-ansible/files/etcd.service

〜/ playground / etcd-ansible / files / etcd.service

[Unit]
    Description=etcd distributed reliable key-value store
    
    [Service]
    Type=notify
    ExecStart=/opt/etcd/bin/etcd
    Restart=always

This unit file defines a service that runs the executable at `/opt/etcd/bin/etcd`, notifies systemd when it has finished initializing, and always restarts if it ever exits.

该单元文件定义了一个服务，该服务在`/opt/etcd/bin/etcd`中运行可执行文件，并在完成初始化后通知systemd，并在退出时始终重新启动。

**Note:** If you’d like to understand more about systemd and unit files, or want to tailor the unit file to your needs, read the [Understanding Systemd Units and Unit Files][] guide.

**注意：**如果您想了解有关systemd和单位文件的更多信息，或者想根据需要定制单位文件，请阅读《 [了解系统单位和单位文件》][Understanding Systemd Units and Unit Files]指南。

Close and save the `files/etcd.service` file by pressing `CTRL+X` followed by `Y`.

通过按`CTRL+X`然后按`Y`关闭并保存`files/etcd.service`文件。

Next, we need to add a task inside our playbook that will copy the `files/etcd.service` local file into the `/etc/systemd/system/etcd.service` directory for every managed node. We can do this using the [`copy`][copy] module.

接下来，我们需要在剧本内添加一个任务，该任务会将每个受管节点的`files/etcd.service`本地文件复制到`/etc/systemd/system/etcd.service`目录中。 我们可以使用[`copy`][copy]模块来做到这一点。

Open up your playbook:

打开您的剧本：

*  nano $HOME/playground/etcd-ansible/playbook.yaml
    
    纳米$ HOME / playground / etcd-ansible / playbook.yaml

Append the following highlighted task to the end of our existing tasks:

将以下突出显示的任务追加到我们现有任务的末尾：

~/playground/etcd-ansible/playbook.yaml

〜/ playground / etcd-ansible / playbook.yaml

- hosts: etcd
      become: True
      tasks:
        ...
        - name: "Set the ETCDCTL_API environment variable to 3"
          lineinfile:
            path: /etc/profile
            line: export ETCDCTL_API=3
            state: present
            create: True
            insertafter: EOF
        - name: "Create a etcd service"
          copy:
            src: files/etcd.service
            remote_src: False
            dest: /etc/systemd/system/etcd.service
            owner: root
            group: root
            mode: 0644

By copying the unit file into the `/etc/systemd/system/etcd.service`, a service is now defined.

通过将单位文件复制到`/etc/systemd/system/etcd.service` ，现在可以定义一个服务。

Save and exit the playbook.

保存并退出剧本。

Run the same `ansible-playbook` command again to apply the new changes:

再次运行相同的`ansible-playbook`命令以应用新更改：

*  ansible-playbook -i hosts playbook.yaml
    
    ansible-playbook -i托管playbook.yaml

To confirm the changes have been applied, first SSH into one of the managed nodes:

要确认已应用更改，请首先SSH进入一个受管节点：

*  ssh root@etcd1\_public\_ip
    
    SSH 根 @ etcd1\_public\_ip

Then, run `systemctl status etcd` to query systemd about the status of the `etcd` service:

然后，运行`systemctl status etcd`来查询systemd有关`etcd`服务的状态：

*  systemctl status etcd
    
    systemctl状态等

You will find the following output, which states that the service is loaded:

您将找到以下输出，指出该服务已加载：

Output ● etcd.service - etcd distributed reliable key-value store
       Loaded: loaded (/etc/systemd/system/etcd.service; static; vendor preset: enabled)
       Active: inactive (dead)
    ...

**Note:** The last line (`Active: inactive (dead)`) of the output states that the service is inactive, which means it would not be automatically run when the system starts. This is expected and not an error.

**注意：输出**的最后一行( `Active: inactive (dead)` )指出该服务是不活动的，这意味着在系统启动时该服务不会自动运行。 这是预期的，不是错误。

Press `q` to return to the shell, and then run `exit` to exit out of the managed node and back to your local shell:

按`q`返回到外壳程序，然后运行`exit`退出受管节点并返回到本地外壳程序：

*  exit
    
    出口

In this step, we updated our playbook to run the `etcd` binary as a systemd service. In the next step, we will continue to set up etcd by providing it space to store its data.

在这一步中，我们更新了剧本以将`etcd`二进制文件作为systemd服务运行。 在下一步中，我们将通过为其提供存储数据的空间来继续设置etcd。

## 步骤5 —配置数据目录 **(**Step 5 — Configuring the Data Directory**)** ##

etcd is a key-value data store, which means we must provide it with space to store its data. In this step, we are going to update our playbook to define a dedicated data directory for etcd to use.

etcd是键值数据存储，这意味着我们必须为其提供空间来存储其数据。 在这一步中，我们将更新我们的剧本以定义供etcd使用的专用数据目录。

Open up your playbook:

打开您的剧本：

*  nano $HOME/playground/etcd-ansible/playbook.yaml
    
    纳米$ HOME / playground / etcd-ansible / playbook.yaml

Append the following task to the end of the list of tasks:

将以下任务追加到任务列表的末尾：

~/playground/etcd-ansible/playbook.yaml

〜/ playground / etcd-ansible / playbook.yaml

- hosts: etcd
      become: True
      tasks:
        ...
        - name: "Create a etcd service"
          copy:
            src: files/etcd.service
            remote_src: False
            dest: /etc/systemd/system/etcd.service
            owner: root
            group: root
            mode: 0644
        - name: "Create a data directory"
          file:
            path: /var/lib/etcd/{
        { inventory_hostname }}.etcd
            state: directory
            owner: root
            group: root
            mode: 0755

Here, we are using `/var/lib/etcd/hostname.etcd` as the data directory, where `hostname` is the hostname of the current managed node. `inventory_hostname` is a variable that represents the hostname of the current managed node; its value is populated by Ansible automatically. The curly-braces syntax (i.e., `{ { inventory_hostname }}`) is used for *variable substitution*, supported by the [Jinja2][] template engine, which is the default templating engine for Ansible.

在这里，我们使用`/var/lib/etcd/ hostname .etcd`作为数据目录，其中`hostname`是当前受管节点的主机名。 `inventory_hostname`主机名是一个变量，表示当前受管节点的主机名。 它的值由Ansible自动填充。 大括号语法(即`{ { inventory_hostname }}` stock\_hostname `{ { inventory_hostname }}` )用于*变量替换* ，由[Jinja2][]模板引擎支持，后者是Ansible的默认模板引擎。

Close the text editor and save the file.

关闭文本编辑器并保存文件。

Next, we need to instruct etcd to use this data directory. We do this by passing in the `data-dir` parameter to etcd. To set etcd parameters, we can use a combination of environment variables, command-line flags, and configuration files. For this tutorial, we will use a configuration file, as it is much neater to isolate all configurations into a file, rather than have configuration littered across our playbook.

接下来，我们需要指示etcd使用此数据目录。 为此，我们将`data-dir`参数传递给etcd。 要设置etcd参数，我们可以结合使用环境变量，命令行标志和配置文件。 在本教程中，我们将使用配置文件，因为将所有配置隔离到一个文件中比较整洁，而不是在我们的剧本中乱七八糟。

In your project directory, create a new directory named `templates/`:

在您的项目目录中，创建一个名为`templates/`的新目录：

*  mkdir templates
    
    mkdir模板

Then, using your editor, create a new file named `etcd.conf.yaml.j2` within the directory:

然后，使用编辑器在目录中创建一个名为`etcd.conf.yaml.j2`的新文件：

*  nano templates/etcd.conf.yaml.j2
    
    纳米模板/etcd.conf.yaml.j2

Next, copy the following line and paste it into the file:

接下来，复制以下行并将其粘贴到文件中：

~/playground/etcd-ansible/templates/etcd.conf.yaml.j2

〜/ playground / etcd-ansible / templates / etcd.conf.yaml.j2

data-dir: /var/lib/etcd/{
        { inventory_hostname }}.etcd

This file uses the same Jinja2 variable substitution syntax as our playbook. To substitute the variables and upload the result to each managed host, we can use the [`template`][template] module. It works in a similar way to `copy`, except it will perform variable substitution prior to upload.

该文件使用与我们的剧本相同的Jinja2变量替换语法。 要替换变量并将结果上传到每个托管主机，我们可以使用[`template`][template]模块。 它的`copy`方式类似，不同之处在于它将在上传之前执行变量替换。

Exit from `etcd.conf.yaml.j2`, then open up your playbook:

从`etcd.conf.yaml.j2`退出，然后打开您的剧本：

*  nano $HOME/playground/etcd-ansible/playbook.yaml
    
    纳米$ HOME / playground / etcd-ansible / playbook.yaml

Append the following tasks to the list of tasks to create a directory and upload the templated configuration file into it:

将以下任务添加到任务列表以创建目录并将模板化的配置文件上载到其中：

~/playground/etcd-ansible/playbook.yaml

〜/ playground / etcd-ansible / playbook.yaml

- hosts: etcd
      become: True
      tasks:
        ...
        - name: "Create a data directory"
          file:
            ...
            mode: 0755
        - name: "Create directory for etcd configuration"
          file:
            path: /etc/etcd
            state: directory
            owner: root
            group: root
            mode: 0755
        - name: "Create configuration file for etcd"
          template:
            src: templates/etcd.conf.yaml.j2
            dest: /etc/etcd/etcd.conf.yaml
            owner: root
            group: root
            mode: 0600

Save and close this file.

保存并关闭此文件。

Because we’ve made this change, we need to update our service’s unit file to pass it the location of our configuration file (i.e., `/etc/etcd/etcd.conf.yaml`).

因为已经进行了更改，所以我们需要更新服务的单位文件，以将其传递到配置文件的位置(即`/etc/etcd/etcd.conf.yaml` )。

Open the etcd service file on your local machine:

在本地计算机上打开etcd服务文件：

*  nano files/etcd.service
    
    纳米文件/ etcd.service

Update the `files/etcd.service` file by adding the `--config-file` flag highlighted in the following:

通过添加以下突出显示的`--config-file`标志来更新`files/etcd.service`文件：

~/playground/etcd-ansible/files/etcd.service

〜/ playground / etcd-ansible / files / etcd.service

[Unit]
    Description=etcd distributed reliable key-value store
    
    [Service]
    Type=notify
    ExecStart=/opt/etcd/bin/etcd --config-file /etc/etcd/etcd.conf.yaml
    Restart=always

Save and close this file.

保存并关闭此文件。

In this step, we used our playbook to provide a data directory for etcd to store its data. In the next step, we will add a couple more tasks to restart the `etcd` service and have it run on startup.

在此步骤中，我们使用了剧本为etcd提供了一个数据目录来存储其数据。 在下一步中，我们将添加更多任务以重新启动`etcd`服务并使其在启动时运行。

## 第6步—启用和启动etcd服务 **(**Step 6 — Enabling and Starting the etcd Service**)** ##

Whenever we make changes to the unit file of a service, we need to restart the service to have it take effect. We can do this by running the `systemctl restart etcd` command. Furthermore, to make the `etcd` service start automatically on system startup, we need to run `systemctl enable etcd`. In this step, we will run those two commands using the playbook.

每当我们更改服务的单位文件时，我们都需要重新启动服务以使其生效。 我们可以通过运行`systemctl restart etcd`命令来做到这一点。 此外，要使`etcd`服务在系统启动时自动启动，我们需要运行`systemctl enable etcd` 。 在这一步中，我们将使用剧本运行这两个命令。

To run commands, we can use the [`command`][command] module:

要运行命令，我们可以使用[`command`][command]模块：

*  nano $HOME/playground/etcd-ansible/playbook.yaml
    
    纳米$ HOME / playground / etcd-ansible / playbook.yaml

Append the following tasks to the end of the task list:

将以下任务追加到任务列表的末尾：

~/playground/etcd-ansible/playbook.yaml

〜/ playground / etcd-ansible / playbook.yaml

- hosts: etcd
      become: True
      tasks:
        ...
        - name: "Create configuration file for etcd"
          template:
            ...
            mode: 0600
        - name: "Enable the etcd service"
          command: systemctl enable etcd
        - name: "Start the etcd service"
          command: systemctl restart etcd

Save and close the file.

保存并关闭文件。

Run `ansible-playbook -i hosts playbook.yaml` once more:

运行`ansible-playbook -i hosts playbook.yaml`再次`ansible-playbook -i hosts playbook.yaml` ：

*  ansible-playbook -i hosts playbook.yaml
    
    ansible-playbook -i托管playbook.yaml

To check that the `etcd` service is now restarted and enabled, SSH into one of the managed nodes:

要检查`etcd`服务现在已重新启动并启用，请通过SSH进入以下受管节点之一：

*  ssh root@etcd1\_public\_ip
    
    SSH 根 @ etcd1\_public\_ip

Then, run `systemctl status etcd` to check the status of the `etcd` service:

然后，运行`systemctl status etcd`来检查`etcd`服务的状态：

*  systemctl status etcd
    
    systemctl状态等

You will find `enabled` and `active (running)` as highlighted in the following; this means the changes we made in our playbook have taken effect:

您将发现`enabled`和`active (running)`如以下突出显示； 这意味着我们在剧本中所做的更改已生效：

Output ● etcd.service - etcd distributed reliable key-value store
       Loaded: loaded (/etc/systemd/system/etcd.service; static; vendor preset: enabled)
       Active: active (running)
     Main PID: 19085 (etcd)
        Tasks: 11 (limit: 2362)

In this step, we used the `command` module to run `systemctl` commands that restart and enable the `etcd` service on our managed nodes. Now that we have set up an etcd installation, we will, in the next step, test out its functionality by carry out some basic create, read, update, and delete (CRUD) operations.

在此步骤中，我们使用`command`模块来运行`systemctl`命令，这些命令可在`etcd`管节点上重新启动并启用`etcd`服务。 现在我们已经设置了etcd安装，接下来，我们将通过执行一些基本的创建，读取，更新和删除(CRUD)操作来测试其功能。

## 第7步-测试etcd **(**Step 7 — Testing etcd**)** ##

Although we have a working etcd installation, it is insecure and not yet ready for production use. But before we secure our etcd setup in later steps, let’s first understand what etcd can do in terms of functionality. In this step, we are going to manually send requests to etcd to add, retrieve, update, and delete data from it.

尽管我们已经安装了etcd，但它并不安全，还没有准备好用于生产环境。 但是在以后的步骤中保护etcd设置之前，让我们首先了解etcd在功能方面可以做什么。 在这一步中，我们将手动将请求发送到etcd，以从中添加，检索，更新和删除数据。

By default, etcd exposes an API that listens on port `2379` for client communication. This means we can send raw API requests to etcd using an HTTP client. However, it’s quicker to use the official etcd client `etcdctl`, which allows you to create/update, retrieve, and delete key-value pairs using the `put`, `get`, and `del` subcommands, respectively.

默认情况下，etcd公开一个侦听端口`2379`进行客户端通信的API。 这意味着我们可以使用HTTP客户端将原始API请求发送到etcd。 但是，使用官方的etcd客户端`etcdctl` ，它允许您分别使用`put` ， `get`和`del`子命令来创建/更新，检索和删除键值对。

Make sure you’re still inside the **etcd1** managed node, and run the following `etcdctl` commands to confirm your etcd installation is working.

确保您仍在**etcd1受**管节点内，并运行以下`etcdctl`命令以确认您的etcd安装正在运行。

First, create a new entry using the `put` subcommand.

首先，使用`put`子命令创建一个新条目。

The `put` subcommand has the following syntax:

`put`子命令具有以下语法：

etcdctl put key value

On **etcd1**, run the following command:

在**etcd1上** ，运行以下命令：

*  etcdctl put foo "bar"
    
    etcdctl把foo放到“ bar”

The command we just ran instructs etcd to write the value `"bar"` to the key `foo` in the store.

我们刚刚运行的命令指示etcd将值`"bar"`写入存储区中的键`foo` 。

You will then find `OK` printed in the output, which indicates the data persisted:

然后，您将在输出中找到`OK`打印，这表明数据已持久：

Output OK

We can then retrieve this entry using the `get` subcommand, which has the syntax `etcdctl get key`:

然后，我们可以使用`get`子命令检索该条目，该命令的语法为`etcdctl get key` ：

*  etcdctl get foo
    
    etcdctl获取foo

You will find this output, which shows the key on the first line and the value you inserted earlier on the second line:

您将找到此输出，该输出在第一行显示键，并在第二行显示您先前插入的值：

Output foo
    bar

We can delete the entry using the `del` subcommand, which has the syntax `etcdctl del key`:

我们可以使用`del`子命令删除条目，该命令的语法为`etcdctl del key` ：

*  etcdctl del foo
    
    etcdctl del foo

You will find the following output, which indicates the number of entries deleted:

您将找到以下输出，该输出指示已删除的条目数：

Output 1

Now, let’s run the `get` subcommand once more in an attempt to retrieve a deleted key-value pair:

现在，让我们再次运行`get`子命令，以尝试检索已删除的键值对：

*  etcdctl get foo
    
    etcdctl获取foo

You will not receive an output, which means `etcdctl` is unable to retrieve the key-value pair. This confirms that after the entry is deleted, it can no longer be retrieved.

您将不会收到输出，这意味着`etcdctl`无法检索键值对。 这确认删除条目后，将无法再检索它。

Now that you’ve tested the basic operations of etcd and `etcdctl`, let’s exit out of our managed node and back to your local environment:

现在，您已经测试了etcd和`etcdctl`的基本操作，让我们退出托管节点，回到本地环境：

*  exit
    
    出口

In this step, we used the `etcdctl` client to send requests to etcd. At this point, we are running three separate instances of etcd, each acting independently from each other. However, etcd is designed as a distributed key-value store, which means multiple etcd instances can group up to form a single *cluster*; each instance then becomes a *member* of the cluster. After forming a cluster, you would be able to retrieve a key-value pair that was inserted from a different member of the cluster. In the next step, we will use our playbook to transform our 3 single-node clusters into a single 3-node cluster.

在这一步中，我们使用`etcdctl`客户端将请求发送到etcd。 此时，我们正在运行三个单独的etcd实例，每个实例彼此独立。 但是，etcd被设计为分布式键值存储，这意味着可以将多个etcd实例组成一个*集群* 。 然后，每个实例都成为集群的*成员* 。 形成集群后，您将能够检索从集群的其他成员插入的键值对。 下一步，我们将使用剧本将3个单节点群集转换为3个节点群集。

## 步骤8 —使用静态发现形成集群 **(**Step 8 — Forming a Cluster Using Static Discovery**)** ##

To create one 3-node cluster instead of three 1-node clusters, we must configure these etcd installations to communicate with each other. This means each one must know the IP addresses of the others. This process is called *discovery*. Discovery can be done using either *static configuration* or *dynamic service discovery*. In this step, we will discuss the difference between the two, as well as update our playbook to set up an etcd cluster using static discovery.

要创建一个3节点群集而不是3个1节点群集，我们必须配置这些etcd安装以相互通信。 这意味着每个人都必须知道其他人的IP地址。 此过程称为*发现* 。 可以使用*静态配置*或*动态服务发现*来完成*发现* 。 在这一步中，我们将讨论两者之间的区别，并更新我们的剧本以使用静态发现来建立一个etcd集群。

Discovery by static configuration is the method that requires the least setup; this is where the endpoints of each member are passed into the `etcd` command before it is executed. To use static configuration, the following conditions must be met prior to the initialization of the cluster:

通过静态配置发现是需要最少设置的方法。 这是每个成员的端点在执行之前传递到`etcd`命令中的位置。 要使用静态配置，必须在集群初始化之前满足以下条件：

*  the number of members are known
    
    成员数量已知
 *  the endpoints of each member are known
    
    每个成员的端点是已知的
 *  the IP addresses for all endpoints are static
    
    所有端点的IP地址都是静态的

If these conditions cannot be met, then you can use a dynamic discovery service. With dynamic service discovery, all instances would register with the discovery service, which allows each member to retrieve information about the location of other members.

如果不能满足这些条件，则可以使用动态发现服务。 使用动态服务发现，所有实例都会在发现服务中注册，这允许每个成员检索有关其他成员位置的信息。

Since we know we want a 3-node etcd cluster, and all our servers have static IP addresses, we will use static discovery. To initiate our cluster using static discovery, we must add several parameters to our configuration file. Use an editor to open up the `templates/etcd.conf.yaml.j2` template file:

由于我们知道我们需要一个3节点的etcd集群，并且我们所有的服务器都具有静态IP地址，因此我们将使用静态发现。 要使用静态发现启动集群，我们必须向配置文件中添加几个参数。 使用编辑器打开`templates/etcd.conf.yaml.j2`模板文件：

*  nano templates/etcd.conf.yaml.j2
    
    纳米模板/etcd.conf.yaml.j2

Then, add the following highlighted lines:

然后，添加以下突出显示的行：

~/playground/etcd-ansible/templates/etcd.conf.yaml.j2

〜/ playground / etcd-ansible / templates / etcd.conf.yaml.j2

data-dir: /var/lib/etcd/{
        { inventory_hostname }}.etcd
    name: {
        { inventory_hostname }}
    initial-advertise-peer-urls: http://{
        { hostvars[inventory_hostname]['ansible_facts']['eth1']['ipv4']['address'] }}:2380
    listen-peer-urls: http://{
        { hostvars[inventory_hostname]['ansible_facts']['eth1']['ipv4']['address'] }}:2380,http://127.0.0.1:2380
    advertise-client-urls: http://{
        { hostvars[inventory_hostname]['ansible_facts']['eth1']['ipv4']['address'] }}:2379
    listen-client-urls: http://{
        { hostvars[inventory_hostname]['ansible_facts']['eth1']['ipv4']['address'] }}:2379,http://127.0.0.1:2379
    initial-cluster-state: new
    initial-cluster: {% for host in groups['etcd'] %}{
        { hostvars[host]['ansible_facts']['hostname'] }}=http://{
        { hostvars[host]['ansible_facts']['eth1']['ipv4']['address'] }}:2380{% if not loop.last %},{% endif %}{% endfor %}

Close and save the `templates/etcd.conf.yaml.j2` file by pressing `CTRL+X` followed by `Y`.

关闭并保存`templates/etcd.conf.yaml.j2`文件，方法是按`CTRL+X`然后按`Y`

Here’s a brief explanation of each parameter:

这是每个参数的简要说明：

*  `name` \- a human-readable name for the member. By default, etcd uses a unique, randomly-generated ID to identify each member; however, a human-readable name allows us to reference it more easily inside configuration files and on the command line. Here, we will use the hostnames as the member names (i.e., `etcd1`, `etcd2`, and `etcd3`).
    
    `name` \-成员的易读名称。 默认情况下，etcd使用唯一的，随机生成的ID来标识每个成员。 但是，人类可读的名称使我们可以在配置文件内部和命令行中更轻松地引用它。 在这里，我们将使用主机名作为成员名(即`etcd1` ， `etcd2`和`etcd3` )。
 *  `initial-advertise-peer-urls` \- a list of IP address/port combinations that other members can use to communicate with this member. In addition to the API port (`2379`), etcd also exposes port `2380` for peer communication between etcd members, which allows them to send messages to each other and exchange data. Note that these URLs must be reachable by its peers (and not be a local IP address).
    
    `initial-advertise-peer-urls`其他成员可以用来与该成员通信的IP地址/端口组合的列表。 除了API端口( `2379` )之外，etcd还公开了端口`2380`以便etcd成员之间进行对等通信，从而允许它们彼此之间发送消息并交换数据。 请注意，这些URL的对等方必须可以访问(而不是本地IP地址)。
 *  `listen-peer-urls` \- a list of IP address/port combinations where the current member will listen for communication from other members. This must include all the URLs from the `--initial-advertise-peer-urls` flag, but also local URLs like `127.0.0.1:2380`. The destination IP address/port of incoming peer messages must match one of the URLs listed here.
    
    `listen-peer-urls` \-IP地址/端口组合的列表，当前成员将在其中侦听其他成员的通信。 这必须包括`--initial-advertise-peer-urls`标志中的所有URL，还必须包括`127.0.0.1:2380`类的本地URL。 传入对等消息的目标IP地址/端口必须与此处列出的URL之一匹配。
 *  `advertise-client-urls` \- a list of IP address/port combinations that clients should use to communicate with this member. These URLs must be reachable by the client (and not be a local address). If the client is accessing the cluster over public internet, this must be a public IP address.
    
    `advertise-client-urls`客户端用于与该成员通信的IP地址/端口组合的列表。 客户端必须可以访问这些URL(而不是本地地址)。 如果客户端通过公共Internet访问群集，则该地址必须是公共IP地址。
 *  `listen-client-urls` \- a list of IP address/port combinations where the current member will listen for communication from clients. This must include all the URLs from the `--advertise-client-urls` flag, but also local URLs like `127.0.0.1:2379`. The destination IP address/port of incoming client messages must match one of the URLs listed here.
    
    `listen-client-urls` \-IP地址/端口组合的列表，当前成员将在其中侦听来自客户端的通信。 这必须包括`--advertise-client-urls`标志中的所有URL，还必须包括`127.0.0.1:2379`类的本地URL。 传入客户端消息的目标IP地址/端口必须与此处列出的URL之一匹配。
 *  `initial-cluster` \- a list of endpoints for each member of the cluster. Each endpoint must match one of the corresponding member’s `initial-advertise-peer-urls` URLs.
    
    `initial-cluster`中每个成员的端点列表。 每个端点必须匹配相应成员的`initial-advertise-peer-urls` URL中的一个。
 *  `initial-cluster-state` \- either `new` or `existing`.
    
    `initial-cluster-state` state- `new`或`existing` 。

To ensure consistency, etcd can only make decisions when a majority of the nodes are healthy. This is known as establishing *quorum*. In other words, in a three-member cluster, quorum is reached if two or more of the members are healthy.

为了确保一致性，etcd仅在大多数节点运行状况良好时才能做出决策。 这称为建立*仲裁* 。 换句话说，在三成员群集中，如果两个或两个以上成员健康，则达到法定人数。

If the `initial-cluster-state` parameter is set to `new`, `etcd` will know that this is a new cluster being bootstrapped, and will allow members to start in parallel, without waiting for quorum to be reached. More concretely, after the first member is started, it will not have quorum because one third (33.33%) is less than or equal to 50%. Normally, etcd will halt and refuse to commit any more actions and the cluster will never be formed. However, with `initial-cluster-state` set to `new`, it will ignore the initial lack of quorum.

如果将`initial-cluster-state`参数设置为`new` ，则`etcd`将知道这是一个正在引导的新集群，并将允许成员并行启动，而无需等待达到法定人数。 更具体地讲，第一个成员启动后就不会达到法定人数，因为三分之一(33.33％)小于或等于50％。 通常，etcd将停止并拒绝执行任何其他操作，并且该集群将永远不会形成。 但是，将`initial-cluster-state`设置为`new` ，它将忽略最初缺乏仲裁的情况。

If set to `existing`, the member will try to join an existing cluster, and expects quorum to already be established.

如果设置为`existing` ，则成员将尝试加入现有集群，并期望已经建立仲裁。

**Note:** You can find more details about all supported configuration flags in the [Configuration][] section of etcd’s documentation.

**注意：**您可以在etcd文档的“ [配置”][Configuration]部分中找到有关所有受支持的配置标志的更多详细信息。

In the updated `templates/etcd.conf.yaml.j2` template file, there are a few instances of `hostvars`. When Ansible runs, it will collect variables from a variety of sources. We have already made use of the `inventory_hostname` variable before, but there are a lot more available. These variables are available under `hostvars[inventory_hostname]['ansible_facts']`. Here, we are extracting the private IP addresses of each node and using it to construct our parameter value.

在更新的`templates/etcd.conf.yaml.j2`模板文件中，有一些`hostvars`实例。 当Ansible运行时，它将从各种来源收集变量。 我们之前已经使用了`inventory_hostname`变量，但是还有更多可用的方法。 这些变量在`hostvars[inventory_hostname]['ansible_facts']`下可用。 在这里，我们提取每个节点的私有IP地址，并使用它来构造我们的参数值。

**Note:** Because we enabled the **Private Networking** option when we created our servers, each server would have three IP addresses associated with them:

**注意：**因为在创建服务器时启用了“ **专用网络”**选项，所以每个服务器将具有与之关联的三个IP地址：

*  A *loopback* IP address - an address that is only valid inside the same machine. It is used for the machine to refer to itself, e.g., `127.0.0.1`
    
    *环回* IP地址-仅在同一台机器内部有效的地址。 它用于机器引用自身，例如`127.0.0.1`
 *  A *public* IP address - an address that is routable over the public internet, e.g., `178.128.169.51`
    
    *公共* IP地址-可通过公共Internet路由的地址，例如`178.128.169.51`
 *  A *private* IP address - an address that is routable only within the private network; in the case of DigitalOcean Droplets, there’s a private network within each datacenter, e.g., `10.131.82.225`
    
    *专用* IP地址-只能在专用网络内路由的地址； 对于DigitalOcean Droplet，每个数据中心内都有一个专用网络，例如`10.131.82.225`

Each of these IP addresses are associated with a different network interface—the loopback address is associated with the `lo` interface, the public IP address is associated with the `eth0` interface, and the private IP address with the `eth1` interface. We are using the `eth1` interface so that all traffic stays within the private network, without ever reaching the internet.

这些IP地址中的每一个都与不同的网络接口相关联-环回地址与`lo`接口相关联，公用IP地址与`eth0`接口相关联，专用IP地址与`eth1`接口相关联。 我们正在使用`eth1`接口，以便所有流量都保持在专用网络内，而永远不会到达Internet。

Understanding of network interfaces is not required for this article, but if you’d like to learn more, [An Introduction to Networking Terminology, Interfaces, and Protocols][An Introduction to Networking Terminology_ Interfaces_ and Protocols] is a great place to start.

本文不需要了解网络接口，但是如果您想了解更多信息，那么[网络术语，接口和协议简介][An Introduction to Networking Terminology_ Interfaces_ and Protocols]是一个很好的起点。

The `{% %}` Jinja2 syntax defines the `for` loop structure that iterates through every node in the `etcd` group to build up the `initial-cluster` string into a format required by etcd.

`{% %}` Jinja2语法定义了`for`循环结构，该结构循环遍历`etcd`组中的每个节点，以将`initial-cluster`字符串构建为etcd所需的格式。

To form the new three-member cluster, you must first stop the `etcd` service and clear the data directory before launching the cluster. To do this, use an editor to open up the `playbook.yaml` file on your local machine:

要形成新的三成员集群，必须先停止`etcd`服务并清除数据目录，然后再启动集群。 为此，请使用编辑器在本地计算机上打开`playbook.yaml`文件：

*  nano $HOME/playground/etcd-ansible/playbook.yaml
    
    纳米$ HOME / playground / etcd-ansible / playbook.yaml

Then, before the `"Create a data directory"` task, add a task to stop the `etcd` service:

然后，在`"Create a data directory"`任务之前，添加一个任务以停止`etcd`服务：

~/playground/etcd-ansible/playbook.yaml

〜/ playground / etcd-ansible / playbook.yaml

- hosts: etcd
      become: True
      tasks:
        ...
            group: root
            mode: 0644
        - name: "Stop the etcd service"
          command: systemctl stop etcd
        - name: "Create a data directory"
          file:
        ...

Next, update the `"Create a data directory"` task to first delete the data directory and recreate it:

接下来，更新`"Create a data directory"`任务以首先删除数据目录并重新创建它：

~/playground/etcd-ansible/playbook.yaml

〜/ playground / etcd-ansible / playbook.yaml

- hosts: etcd
      become: True
      tasks:
        ...
        - name: "Stop the etcd service"
          command: systemctl stop etcd
        - name: "Create a data directory"
          file:
            path: /var/lib/etcd/{
        { inventory_hostname }}.etcd
            state: "{
        { item }}"
            owner: root
            group: root
            mode: 0755
          with_items:
            - absent
            - directory
        - name: "Create directory for etcd configuration"
          file:
        ...

The `with_items` property defines a list of strings that this task will iterate over. It is equivalent to repeating the same task twice but with different values for the `state` property. Here, we are iterating over the list with items `absent` and `directory`, which ensures that the data directory is deleted first and then re-created after.

`with_items`属性定义此任务将迭代的字符串列表。 这等效于重复两次相同的任务，但是对`state`属性使用不同的值。 在这里，我们遍历该列表，其中`absent`项目和`directory` ，这可确保首先删除数据目录，然后再重新创建。

Close and save the `playbook.yaml` file by pressing `CTRL+X` followed by `Y`. Then, run `ansible-playbook` again. Ansible will now create a single, 3-member `etcd` cluster:

通过按`CTRL+X`然后按`Y`关闭并保存`playbook.yaml`文件。 然后，再次运行`ansible-playbook` 。 Ansible现在将创建一个由3个成员组成的`etcd`集群：

*  ansible-playbook -i hosts playbook.yaml
    
    ansible-playbook -i托管playbook.yaml

You can check this by SSH-ing into any etcd member node:

您可以通过SSH进入任何etcd成员节点来进行检查：

*  ssh root@etcd1\_public\_ip
    
    SSH 根 @ etcd1\_public\_ip

Then run `etcdctl endpoint health --cluster`:

然后运行`etcdctl endpoint health --cluster`运行`etcdctl endpoint health --cluster` ：

*  etcdctl endpoint health --cluster
    
    etcdctl端点运行状况-集群

This will list out the health of each member of the cluster:

这将列出集群中每个成员的运行状况：

Output http://etcd2_private_ip:2379 is healthy: successfully committed proposal: took = 2.517267ms
    http://etcd1_private_ip:2379 is healthy: successfully committed proposal: took = 2.153612ms
    http://etcd3_private_ip:2379 is healthy: successfully committed proposal: took = 2.639277ms

We have now successfully created a 3-node etcd cluster. We can confirm this by adding an entry to etcd on one member node, and retrieving it on another member node. On one of the member nodes, run `etcdctl put`:

*  etcdctl put foo "bar"
    
    etcdctl put foo "bar"

Then, use a new terminal to SSH into a different member node:

*  ssh root@etcd2\_public\_ip
    
    ssh root @ etcd2\_public\_ip

Next, attempt to retrieve the same entry using the key:

*  etcdctl get foo
    
    etcdctl get foo

You will be able to retrieve the entry, which proves that the cluster is working:

Output foo
    bar

Lastly, exit out of each of the managed nodes and back to your local machine:

*  exit
    
    出口

In this step, we provisioned a new 3-node cluster. At the moment, communication between `etcd` members and their peers and clients are conducted through HTTP. This means the communication is unencrypted and any party who can intercept the traffic can read the messages. This is not a big issue if the `etcd` cluster and clients are all deployed within a private network or *virtual private network* (VPN) which you fully control. However, if any of the traffic needs to travel through a shared network (private or public), then you should ensure this traffic is encrypted. Furthermore, a mechanism needs to be put in place for a client or peer to verify the authenticity of the server.

In the next step, we will look at how to secure client-to-server as well as peer communication using TLS.

## Step 9 — Obtaining the Private IP Addresses of Managed Nodes **(**Step 9 — Obtaining the Private IP Addresses of Managed Nodes**)** ##

To encrypt messages between member nodes, etcd uses *Hypertext Transfer Protocol Secure*, or *HTTPS*, which is a layer on top of the *Transport Layer Security*, or *TLS*, protocol. TLS uses a system of private keys, certificates, and trusted entities called *Certificate Authorities* (CAs) to authenticate with, and send encrypted messages to, each other.

To encrypt messages between member nodes, etcd uses *Hypertext Transfer Protocol Secure* , or *HTTPS* , which is a layer on top of the *Transport Layer Security* , or *TLS* , protocol. TLS uses a system of private keys, certificates, and trusted entities called *Certificate Authorities* (CAs) to authenticate with, and send encrypted messages to, each other.

In this tutorial, each member node needs to generate a certificate to identify itself, and have this certificate signed by a CA. We will configure all member nodes to trust this CA, and thus also trust any certificates it signs. This allows member nodes to mutually authenticate with each other.

The certificate that a member node generates must allow other member nodes to identify itself. All certificates include the *Common Name* (CN) of the entity it is associated with. This is often used as the identity of the entity. However, when verifying a certificate, client implementations may compare whether the information it collected about the entity match what was given in the certificate. For example, when a client downloads the TLS certificate with the subject of `CN=foo.bar.com`, but the client is actually connecting to the server using an IP address (e.g., `167.71.129.110`), then there’s a mismatch and the client may not trust the certificate. By specifying a *subject alternative name* (SAN) in the certificate, it informs the verifier that both names belong to the same entity.

The certificate that a member node generates must allow other member nodes to identify itself. All certificates include the *Common Name* (CN) of the entity it is associated with. This is often used as the identity of the entity. However, when verifying a certificate, client implementations may compare whether the information it collected about the entity match what was given in the certificate. For example, when a client downloads the TLS certificate with the subject of `CN=foo.bar.com` , but the client is actually connecting to the server using an IP address (eg, `167.71.129.110` ), then there's a mismatch and the client may not trust the certificate. By specifying a *subject alternative name* (SAN) in the certificate, it informs the verifier that both names belong to the same entity.

Because our etcd members are peering with each other using their private IP addresses, when we define our certificates, we’ll need to provide these private IP addresses as the subject alternative names.

Because our etcd members are peering with each other using their private IP addresses, when we define our certificates, we'll need to provide these private IP addresses as the subject alternative names.

To find out the private IP address of a managed node, SSH into it:

*  ssh root@etcd1\_public\_ip
    
    ssh root @ etcd1\_public\_ip

Then run the following command:

*  ip -f inet addr show eth1
    
    ip -f inet addr show eth1

You’ll find output similar to the following lines:

You'll find output similar to the following lines:

Output 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
        inet 10.131.255.176/16 brd 10.131.255.255 scope global eth1
           valid_lft forever preferred_lft forever

In our example output, `10.131.255.176` is the private IP address of the managed node, and the only information we are interested in. To filter out everything else apart from the private IP, we can pipe the output of the `ip` command to the [`sed` utility][sed_ utility], which is used to filter and transform text.

*  ip -f inet addr show eth1 | sed -En -e 's/.\*inet (\[0-9.\]+).\*/\\1/p'
    
    ip -f inet addr show eth1 | sed -En -e 's/.\*inet (\[0-9.\]+).\*/\\1/p'

Now, the only output is the private IP address itself:

Output 10.131.255.176

Once you’re satisfied that the preceding command works, exit out of the managed node:

Once you're satisfied that the preceding command works, exit out of the managed node:

*  exit
    
    出口

To incorporate the preceding commands into our playbook, first open up the `playbook.yaml` file:

*  nano $HOME/playground/etcd-ansible/playbook.yaml
    
    nano $HOME/playground/etcd-ansible/playbook.yaml

Then, add a new play with a single task before our existing play:

~/playground/etcd-ansible/playbook.yaml

...
    - hosts: etcd
      tasks:
        - shell: ip -f inet addr show eth1 | sed -En -e 's/.*inet ([0-9.]+).*/\1/p'
          register: privateIP
    - hosts: etcd
      tasks:
    ...

The task uses the `shell` module to run the `ip` and `sed` commands, which fetches the private IP address of the managed node. It then *registers* the return value of the shell command inside a variable named `privateIP`, which we will use later.

In this step, we added a task to the playbook to obtain the private IP address of the managed nodes. In the next step, we are going to use this information to generate certificates for each member node, and have these certificates signed by a Certificate Authority (CA).

## Step 10 — Generating etcd Members' Private Keys and CSRs **(**Step 10 — Generating etcd Members’ Private Keys and CSRs**)** ##

In order for a member node to receive encrypted traffic, the sender must use the member node’s public key to encrypt the data, and the member node must use its private key to decrypt the ciphertext and retrieve the original data. The public key is packaged into a certificate and signed by a CA to ensure that it is genuine.

In order for a member node to receive encrypted traffic, the sender must use the member node's public key to encrypt the data, and the member node must use its private key to decrypt the ciphertext and retrieve the original data. The public key is packaged into a certificate and signed by a CA to ensure that it is genuine.

Therefore, we will need to generate a private key and certificate signing request (CSR) for each etcd member node. To make it easier for us, we will generate all key pairs and sign all certificates locally, on the control node, and then copy the relevant files to the managed hosts.

First, create a directory called `artifacts/`, where we’ll place the files (keys and certificates) generated during the process. Open the `playbook.yaml` file with an editor:

First, create a directory called `artifacts/` , where we'll place the files (keys and certificates) generated during the process. Open the `playbook.yaml` file with an editor:

*  nano $HOME/playground/etcd-ansible/playbook.yaml
    
    nano $HOME/playground/etcd-ansible/playbook.yaml

In it, use the `file` module to create the `artifacts/` directory:

~/playground/etcd-ansible/playbook.yaml

...
        - shell: ip -f inet addr show eth1 | sed -En -e 's/.*inet ([0-9.]+).*/\1/p'
          register: privateIP
    - hosts: localhost
      gather_facts: False
      become: False
      tasks:
        - name: "Create ./artifacts directory to house keys and certificates"
          file:
            path: ./artifacts
            state: directory
    - hosts: etcd
      tasks:
    ...

Next, add another task to the end of the play to generate the private key:

~/playground/etcd-ansible/playbook.yaml

...
    - hosts: localhost
      gather_facts: False
      become: False
      tasks:
            ...
        - name: "Generate private key for each member"
          openssl_privatekey:
            path: ./artifacts/{
        {item}}.key
            type: RSA
            size: 4096
            state: present
            force: True
          with_items: "{
        { groups['etcd'] }}"
    - hosts: etcd
      tasks:
    ...

Creating private keys and CSRs can be done using the [`openssl_privatekey`][openssl_privatekey] and [`openssl_csr`][openssl_csr] modules, respectively.

The `force: True` attribute ensures that the private key is regenerated each time, even if it exists already.

Similarly, append the following new task to the same play to generate the CSRs for each member, using the `openssl_csr` module:

~/playground/etcd-ansible/playbook.yaml

...
    - hosts: localhost
      gather_facts: False
      become: False
      tasks:
        ...
        - name: "Generate private key for each member"
          openssl_privatekey:
            ...
          with_items: "{
        { groups['etcd'] }}"
        - name: "Generate CSR for each member"
          openssl_csr:
            path: ./artifacts/{
        {item}}.csr
            privatekey_path: ./artifacts/{
        {item}}.key
            common_name: "{
        {item}}"
            key_usage:
              - digitalSignature
            extended_key_usage:
              - serverAuth
            subject_alt_name:
              - IP:{
        { hostvars[item]['privateIP']['stdout']}}
              - IP:127.0.0.1
            force: True
          with_items: "{
        { groups['etcd'] }}"

We are specifying that this certificate can be involved in a digital signature mechanism for the purpose of server authentication. This certificate is associated with the hostname (e.g., `etcd1`), but the verifier should also treat each node’s private and local loopback IP addresses as alternative names. Note that we are using the `privateIP` variable that we registered in the previous play.

We are specifying that this certificate can be involved in a digital signature mechanism for the purpose of server authentication. This certificate is associated with the hostname (eg, `etcd1` ), but the verifier should also treat each node's private and local loopback IP addresses as alternative names. Note that we are using the `privateIP` variable that we registered in the previous play.

Close and save the `playbook.yaml` file by pressing `CTRL+X` followed by `Y`. Then, run our playbook again:

Close and save the `playbook.yaml` file by pressing `CTRL+X` followed by `Y` . Then, run our playbook again:

*  ansible-playbook -i hosts playbook.yaml
    
    ansible-playbook -i hosts playbook.yaml

We will now find a new directory called `artifacts` within our project directory; use `ls` to list out its contents:

*  ls artifacts
    
    ls artifacts

You will find the private keys and CSRs for each of the etcd members:

Output etcd1.csr  etcd1.key  etcd2.csr  etcd2.key  etcd3.csr  etcd3.key

In this step, we used several Ansible modules to generate private keys and public key certificates for each of the member nodes. In the next step, we will look at how to sign a certificate signing request (CSR).

## Step 11 — Generating CA Certificates **(**Step 11 — Generating CA Certificates**)** ##

Within an etcd cluster, member nodes encrypt messages using the receiver’s public key. To ensure the public key is genuine, the receiver packages the public key into a certificate signing request (CSR) and has a trusted entity (i.e., the CA) sign the CSR. Since we control all the member nodes and the CAs they trust, we don’t need to use an external CA and can act as our own CA. In this step, we are going to act as our own CA, which means we’ll need to generate a private key and a self-signed certificate to function as the CA.

Within an etcd cluster, member nodes encrypt messages using the receiver's public key. To ensure the public key is genuine, the receiver packages the public key into a certificate signing request (CSR) and has a trusted entity (ie, the CA) sign the CSR. Since we control all the member nodes and the CAs they trust, we don't need to use an external CA and can act as our own CA. In this step, we are going to act as our own CA, which means we'll need to generate a private key and a self-signed certificate to function as the CA.

First, open the `playbook.yaml` file with your editor:

*  nano $HOME/playground/etcd-ansible/playbook.yaml
    
    nano $HOME/playground/etcd-ansible/playbook.yaml

Then, similar to the previous step, append a task to the `localhost` play to generate a private key for the CA:

~/playground/etcd-ansible/playbook.yaml

- hosts: localhost
      ...
      tasks:
        ...
      - name: "Generate CSR for each member"
        ...
        with_items: "{
        { groups['etcd'] }}"
        - name: "Generate private key for CA"
          openssl_privatekey:
            path: ./artifacts/ca.key
            type: RSA
            size: 4096
            state: present
            force: True
    - hosts: etcd
      become: True
      tasks:
        - name: "Create directory for etcd binaries"
    ...

Next, use the `openssl_csr` module to generate a new CSR. This is similar to the previous step, but in this CSR, we are adding the basic constraint and key usage extension to indicate that this certificate can be used as a CA certificate:

~/playground/etcd-ansible/playbook.yaml

- hosts: localhost
      ...
      tasks:
        ...
        - name: "Generate private key for CA"
          openssl_privatekey:
            path: ./artifacts/ca.key
            type: RSA
            size: 4096
            state: present
            force: True
        - name: "Generate CSR for CA"
          openssl_csr:
            path: ./artifacts/ca.csr
            privatekey_path: ./artifacts/ca.key
            common_name: ca
            organization_name: "Etcd CA"
            basic_constraints:
              - CA:TRUE
              - pathlen:1
            basic_constraints_critical: True
            key_usage:
              - keyCertSign
              - digitalSignature
            force: True
    - hosts: etcd
      become: True
      tasks:
        - name: "Create directory for etcd binaries"
    ...

Lastly, use the [`openssl_certificate`][openssl_certificate] module to self-sign the CSR:

~/playground/etcd-ansible/playbook.yaml

- hosts: localhost
      ...
      tasks:
        ...
        - name: "Generate CSR for CA"
          openssl_csr:
            path: ./artifacts/ca.csr
            privatekey_path: ./artifacts/ca.key
            common_name: ca
            organization_name: "Etcd CA"
            basic_constraints:
              - CA:TRUE
              - pathlen:1
            basic_constraints_critical: True
            key_usage:
              - keyCertSign
              - digitalSignature
            force: True
        - name: "Generate self-signed CA certificate"
          openssl_certificate:
            path: ./artifacts/ca.crt
            privatekey_path: ./artifacts/ca.key
            csr_path: ./artifacts/ca.csr
            provider: selfsigned
            force: True
    - hosts: etcd
      become: True
      tasks:
        - name: "Create directory for etcd binaries"
    ...

Close and save the `playbook.yaml` file by pressing `CTRL+X` followed by `Y`. Then, run our playbook again to apply the changes:

Close and save the `playbook.yaml` file by pressing `CTRL+X` followed by `Y` . Then, run our playbook again to apply the changes:

*  ansible-playbook -i hosts playbook.yaml
    
    ansible-playbook -i hosts playbook.yaml

You can also run `ls` to check the contents of the `artifacts/` directory:

*  ls artifacts/
    
    ls artifacts/

You will now find the freshly generated CA certificate (`ca.crt`):

You will now find the freshly generated CA certificate ( `ca.crt` ):

Output ca.crt  ca.csr  ca.key  etcd1.csr  etcd1.key  etcd2.csr  etcd2.key  etcd3.csr  etcd3.key

In this step, we generated a private key and a self-signed certificate for the CA. In the next step, we will use the CA certificate to sign each member’s CSR.

In this step, we generated a private key and a self-signed certificate for the CA. In the next step, we will use the CA certificate to sign each member's CSR.

## Step 12 — Signing the etcd members' CSRs **(**Step 12 — Signing the etcd members’ CSRs**)** ##

In this step, we are going to sign each member node’s CSR. This will be similar to how we used the `openssl_certificate` module to self-sign the CA certificate, but instead of using the `selfsigned` provider, we will use the `ownca` provider, which allows us to sign using our own CA certificate.

In this step, we are going to sign each member node's CSR. This will be similar to how we used the `openssl_certificate` module to self-sign the CA certificate, but instead of using the `selfsigned` provider, we will use the `ownca` provider, which allows us to sign using our own CA certificate.

Open up your playbook:

*  nano $HOME/playground/etcd-ansible/playbook.yaml
    
    nano $HOME/playground/etcd-ansible/playbook.yaml

Append the following highlighted task to the `"Generate self-signed CA certificate"` task:

~/playground/etcd-ansible/playbook.yaml

- hosts: localhost
      ...
      tasks:
        ...
        - name: "Generate self-signed CA certificate"
          openssl_certificate:
            path: ./artifacts/ca.crt
            privatekey_path: ./artifacts/ca.key
            csr_path: ./artifacts/ca.csr
            provider: selfsigned
            force: True
        - name: "Generate an `etcd` member certificate signed with our own CA certificate"
          openssl_certificate:
            path: ./artifacts/{
        {item}}.crt
            csr_path: ./artifacts/{
        {item}}.csr
            ownca_path: ./artifacts/ca.crt
            ownca_privatekey_path: ./artifacts/ca.key
            provider: ownca
            force: True
          with_items: "{
        { groups['etcd'] }}"
    - hosts: etcd
      become: True
      tasks:
        - name: "Create directory for etcd binaries"
    ...

Close and save the `playbook.yaml` file by pressing `CTRL+X` followed by `Y`. Then, run the playbook again to apply the changes:

Close and save the `playbook.yaml` file by pressing `CTRL+X` followed by `Y` . Then, run the playbook again to apply the changes:

*  ansible-playbook -i hosts playbook.yaml
    
    ansible-playbook -i hosts playbook.yaml

Now, list out the contents of the `artifacts/` directory:

*  ls artifacts/
    
    ls artifacts/

You will find the private key, CSR, and certificate for every etcd member and the CA:

Output ca.crt  ca.csr  ca.key  etcd1.crt  etcd1.csr  etcd1.key  etcd2.crt  etcd2.csr  etcd2.key  etcd3.crt  etcd3.csr  etcd3.key

In this step, we have signed each member node’s CSRs using the CA’s key. In the next step, we are going to copy the relevant files into each managed node, so that etcd has access to the relevant keys and certificates to set up TLS connections.

In this step, we have signed each member node's CSRs using the CA's key. In the next step, we are going to copy the relevant files into each managed node, so that etcd has access to the relevant keys and certificates to set up TLS connections.

## Step 13 — Copying Private Keys and Certificates **(**Step 13 — Copying Private Keys and Certificates**)** ##

Every node needs to have a copy of the CA’s self-signed certificate (`ca.crt`). Each `etcd` member node also needs to have its own private key and certificate. In this step, we are going to upload these files and place them in a new `/etc/etcd/ssl/` directory.

Every node needs to have a copy of the CA's self-signed certificate ( `ca.crt` ). Each `etcd` member node also needs to have its own private key and certificate. In this step, we are going to upload these files and place them in a new `/etc/etcd/ssl/` directory.

To start, open the `playbook.yaml` file with your editor:

*  nano $HOME/playground/etcd-ansible/playbook.yaml
    
    nano $HOME/playground/etcd-ansible/playbook.yaml

To make these changes on our Ansible playbook, first update the `path` property of the `Create directory for etcd configuration` task to create the `/etc/etcd/ssl/` directory:

~/playground/etcd-ansible/playbook.yaml

- hosts: etcd
      ...
      tasks:
        ...
          with_items:
            - absent
            - directory
        - name: "Create directory for etcd configuration"
          file:
            path: "{
        { item }}"
            state: directory
            owner: root
            group: root
            mode: 0755
          with_items:
            - /etc/etcd
            - /etc/etcd/ssl
        - name: "Create configuration file for etcd"
          template:
    ...

Then, following the modified task, add three more tasks to copy the files over:

~/playground/etcd-ansible/playbook.yaml

- hosts: etcd
      ...
      tasks:
        ...
        - name: "Copy over the CA certificate"
          copy:
            src: ./artifacts/ca.crt
            remote_src: False
            dest: /etc/etcd/ssl/ca.crt
            owner: root
            group: root
            mode: 0644
        - name: "Copy over the `etcd` member certificate"
          copy:
            src: ./artifacts/{
        {inventory_hostname}}.crt
            remote_src: False
            dest: /etc/etcd/ssl/server.crt
            owner: root
            group: root
            mode: 0644
        - name: "Copy over the `etcd` member key"
          copy:
            src: ./artifacts/{
        {inventory_hostname}}.key
            remote_src: False
            dest: /etc/etcd/ssl/server.key
            owner: root
            group: root
            mode: 0600
        - name: "Create configuration file for etcd"
          template:
    ...

Close and save the `playbook.yaml` file by pressing `CTRL+X` followed by `Y`.

Close and save the `playbook.yaml` file by pressing `CTRL+X` followed by `Y` .

Run `ansible-playbook` again to make these changes:

*  ansible-playbook -i hosts playbook.yaml
    
    ansible-playbook -i hosts playbook.yaml

In this step, we have successfully uploaded the private keys and certificates to the managed nodes. Having copied the files over, we now need to update our etcd configuration file to make use of them.

## Step 14 — Enabling TLS on etcd **(**Step 14 — Enabling TLS on etcd**)** ##

In the last step of this tutorial, we are going to update some Ansible configurations to enable TLS in an etcd cluster.

First, open up the `templates/etcd.conf.yaml.j2` template file using your editor:

*  nano $HOME/playground/etcd-ansible/templates/etcd.conf.yaml.j2
    
    nano $HOME/playground/etcd-ansible/templates/etcd.conf.yaml.j2

Once inside, change all URLs to use `https` as the protocol instead of `http`. Additionally, add a section at the end of the template to specify the location of the CA certificate, server certificate, and server key:

Once inside, change all URLs to use `https` as the protocol instead of `http` . Additionally, add a section at the end of the template to specify the location of the CA certificate, server certificate, and server key:

~/playground/etcd-ansible/templates/etcd.conf.yaml.j2

data-dir: /var/lib/etcd/{
        { inventory_hostname }}.etcd
    name: {
        { inventory_hostname }}
    initial-advertise-peer-urls: https://{
        { hostvars[inventory_hostname]['ansible_facts']['eth1']['ipv4']['address'] }}:2380
    listen-peer-urls: https://{
        { hostvars[inventory_hostname]['ansible_facts']['eth1']['ipv4']['address'] }}:2380,https://127.0.0.1:2380
    advertise-client-urls: https://{
        { hostvars[inventory_hostname]['ansible_facts']['eth1']['ipv4']['address'] }}:2379
    listen-client-urls: https://{
        { hostvars[inventory_hostname]['ansible_facts']['eth1']['ipv4']['address'] }}:2379,https://127.0.0.1:2379
    initial-cluster-state: new
    initial-cluster: {% for host in groups['etcd'] %}{
        { hostvars[host]['ansible_facts']['hostname'] }}=https://{
        { hostvars[host]['ansible_facts']['eth1']['ipv4']['address'] }}:2380{% if not loop.last %},{% endif %}{% endfor %}
    
    client-transport-security:
      cert-file: /etc/etcd/ssl/server.crt
      key-file: /etc/etcd/ssl/server.key
      trusted-ca-file: /etc/etcd/ssl/ca.crt
    peer-transport-security:
      cert-file: /etc/etcd/ssl/server.crt
      key-file: /etc/etcd/ssl/server.key
      trusted-ca-file: /etc/etcd/ssl/ca.crt

Close and save the `templates/etcd.conf.yaml.j2` file.

Next, run your Ansible playbook:

*  ansible-playbook -i hosts playbook.yaml
    
    ansible-playbook -i hosts playbook.yaml

Then, SSH into one of the managed nodes:

*  ssh root@etcd1\_public\_ip
    
    ssh root @ etcd1\_public\_ip

Once inside, run the `etcdctl endpoint health` command to check whether the endpoints are using HTTPS, and if all members are healthy:

*  etcdctl --cacert /etc/etcd/ssl/ca.crt endpoint health --cluster
    
    etcdctl --cacert /etc/etcd/ssl/ca.crt endpoint health --cluster

Because our CA certificate is not, by default, a trusted root CA certificate installed in the `/etc/ssl/certs/` directory, we need to pass it to `etcdctl` using the `--cacert` flag.

This will give the following output:

这将给出以下输出：

Output https://etcd3_private_ip:2379 is healthy: successfully committed proposal: took = 19.237262ms
    https://etcd1_private_ip:2379 is healthy: successfully committed proposal: took = 4.769088ms
    https://etcd2_private_ip:2379 is healthy: successfully committed proposal: took = 5.953599ms

To confirm that the `etcd` cluster is actually working, we can, once again, create an entry on one member node, and retrieve it from another member node:

*  etcdctl --cacert /etc/etcd/ssl/ca.crt put foo "bar"
    
    etcdctl --cacert /etc/etcd/ssl/ca.crt put foo "bar"

Use a new terminal to SSH into a different node:

*  ssh root@etcd2\_public\_ip
    
    ssh root @ etcd2\_public\_ip

Now retrieve the same entry using the key `foo`:

Now retrieve the same entry using the key `foo` :

*  etcdctl --cacert /etc/etcd/ssl/ca.crt get foo
    
    etcdctl --cacert /etc/etcd/ssl/ca.crt get foo

This will return the entry, showing the output below:

Output foo
    bar

You can do the same on the third node to ensure all three members are operational.

## 结论 **(**Conclusion**)** ##

You have now successfully provisioned a 3-node etcd cluster, secured it with TLS, and confirmed that it is working.

etcd is a tool originally created by [CoreOS][]. To understand etcd’s usage in relation to CoreOS, you can read [How To Use Etcdctl and Etcd, CoreOS’s Distributed Key-Value Store][How To Use Etcdctl and Etcd_ CoreOS_s Distributed Key-Value Store]. The article also guides you through setting up a dynamic discovery model, something which was discussed but not demonstrated in this tutorial.

etcd is a tool originally created by [CoreOS][] . To understand etcd's usage in relation to CoreOS, you can read [How To Use Etcdctl and Etcd, CoreOS's Distributed Key-Value Store][How To Use Etcdctl and Etcd_ CoreOS_s Distributed Key-Value Store] . The article also guides you through setting up a dynamic discovery model, something which was discussed but not demonstrated in this tutorial.

As mentioned at the beginning of this tutorial, etcd is an important part of the Kubernetes ecosystem. To learn more about Kubernetes and etcd’s role within it, you can read [An Introduction to Kubernetes][]. If you are deploying etcd as part of a Kubernetes cluster, know that there are other tools available, such as [kubespray][] and [`kubeadm`][kubeadm]. For more details on the latter, you can read [How To Create a Kubernetes Cluster Using Kubeadm on Ubuntu 18.04][].

As mentioned at the beginning of this tutorial, etcd is an important part of the Kubernetes ecosystem. To learn more about Kubernetes and etcd's role within it, you can read [An Introduction to Kubernetes][] . If you are deploying etcd as part of a Kubernetes cluster, know that there are other tools available, such as [kubespray][] and [`kubeadm`][kubeadm] . For more details on the latter, you can read [How To Create a Kubernetes Cluster Using Kubeadm on Ubuntu 18.04][] .

Finally, this tutorial made use of many tools, but could not dive into each in too much detail. In the following you’ll find links that will provide a more detailed examination of each tool:

Finally, this tutorial made use of many tools, but could not dive into each in too much detail. In the following you'll find links that will provide a more detailed examination of each tool:

*  To learn more advanced syntax of Ansible playbooks, you can read [Configuration Management 101: Writing Ansible Playbooks][Configuration Management 101_ Writing Ansible Playbooks]. Ansible’s official [Intro to Playbooks][] is also a great resource.
    
    To learn more advanced syntax of Ansible playbooks, you can read [Configuration Management 101: Writing Ansible Playbooks][Configuration Management 101_ Writing Ansible Playbooks] . Ansible's official [Intro to Playbooks][] is also a great resource.
 *  To learn more about OpenSSL, you can read [OpenSSL Essentials: Working with SSL Certificates, Private Keys and CSRs][Transport Layer Security_ or _TLS].
    
    To learn more about OpenSSL, you can read [OpenSSL Essentials: Working with SSL Certificates, Private Keys and CSRs][Transport Layer Security_ or _TLS] .

> 翻译自: [https://www.digitalocean.com/community/tutorials/how-to-set-up-and-secure-an-etcd-cluster-with-ansible-on-ubuntu-18-04][https_www.digitalocean.com_community_tutorials_how-to-set-up-and-secure-an-etcd-cluster-with-ansible-on-ubuntu-18-04]

[Wikimedia Foundation]: https://wikimediafoundation.org/
[Write for DOnations]: https://do.co/w4do-cta
[etcd]: https://etcd.io/
[Kubernetes]: https://kubernetes.io/
[Vulcand]: https://vulcand.github.io/
[Doorman]: https://github.com/youtube/doorman
[Transport Layer Security_ or _TLS]: https://www.digitalocean.com/community/tutorials/openssl-essentials-working-with-ssl-certificates-private-keys-and-csrs
[Ansible]: https://www.digitalocean.com/community/conceptual_articles/an-introduction-to-configuration-management-with-ansible
[Puppet]: https://puppet.com/
[Chef]: https://www.chef.io/
[SaltStack]: https://www.saltstack.com/
[Python]: https://www.python.org/
[pyOpenSSL]: https://pypi.org/project/pyOpenSSL/
[How To Install Python 3 and Set Up a Local Programming Environment on Ubuntu 18.04]: https://www.digitalocean.com/community/tutorials/how-to-install-python-3-and-set-up-a-local-programming-environment-on-ubuntu-18-04
[How to Create a Droplet from the DigitalOcean Control Panel]: https://www.digitalocean.com/docs/droplets/how-to/create/
[How to Enable Private Networking on Droplets]: https://www.digitalocean.com/docs/networking/private-networking/how-to/enable/
[Initial Server Setup with Ubuntu 18.04]: https://www.digitalocean.com/community/tutorials/initial-server-setup-with-ubuntu-18-04
[SSH Essentials_ Working with SSH Servers_ Clients_ and Keys]: https://www.digitalocean.com/community/tutorials/ssh-essentials-working-with-ssh-servers-clients-and-keys#generating-and-working-with-ssh-keys
[How to Install and Configure Ansible on Ubuntu 18.04]: https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-ansible-on-ubuntu-18-04
[How to Use Ansible_ A Reference Guide]: https://www.digitalocean.com/community/tutorials/how-to-use-ansible-cheat-sheet-guide
[hostname_ command]: https://docs.ansible.com/ansible/2.5/modules/hostname_module.html
[Git]: https://git-scm.com/
[script_ module]: https://docs.ansible.com/ansible/latest/modules/script_module.html
[debug]: https://docs.ansible.com/ansible/latest/modules/debug_module.html
[Releases]: https://github.com/etcd-io/etcd/releases
[file]: https://docs.ansible.com/ansible/latest/modules/file_module.html
[get_url]: https://docs.ansible.com/ansible/latest/modules/get_url_module.html
[unarchive]: https://docs.ansible.com/ansible/latest/modules/unarchive_module.html
[lineinfile]: https://docs.ansible.com/ansible/latest/modules/lineinfile_module.html
[systemd]: https://www.digitalocean.com/community/tutorials/systemd-essentials-working-with-services-units-and-the-journal
[Understanding Systemd Units and Unit Files]: https://www.digitalocean.com/community/tutorials/understanding-systemd-units-and-unit-files
[copy]: https://docs.ansible.com/ansible/latest/modules/copy_module.html
[Jinja2]: https://palletsprojects.com/p/jinja/
[template]: https://docs.ansible.com/ansible/latest/modules/template_module.html
[command]: https://docs.ansible.com/ansible/latest/modules/command_module.html
[Configuration]: https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/configuration.md
[An Introduction to Networking Terminology_ Interfaces_ and Protocols]: https://www.digitalocean.com/community/tutorials/an-introduction-to-networking-terminology-interfaces-and-protocols
[sed_ utility]: https://www.digitalocean.com/community/tutorials/the-basics-of-using-the-sed-stream-editor-to-manipulate-text-in-linux
[openssl_privatekey]: https://docs.ansible.com/ansible/latest/modules/openssl_privatekey_module.html
[openssl_csr]: https://docs.ansible.com/ansible/latest/modules/openssl_csr_module.html
[openssl_certificate]: https://docs.ansible.com/ansible/2.4/openssl_certificate_module.html
[CoreOS]: https://coreos.com/
[How To Use Etcdctl and Etcd_ CoreOS_s Distributed Key-Value Store]: https://www.digitalocean.com/community/tutorials/how-to-use-etcdctl-and-etcd-coreos-s-distributed-key-value-store
[An Introduction to Kubernetes]: https://www.digitalocean.com/community/tutorials/an-introduction-to-kubernetes
[kubespray]: https://github.com/kubernetes-sigs/kubespray
[kubeadm]: https://kubernetes.io/docs/reference/setup-tools/kubeadm/
[How To Create a Kubernetes Cluster Using Kubeadm on Ubuntu 18.04]: https://www.digitalocean.com/community/tutorials/how-to-create-a-kubernetes-cluster-using-kubeadm-on-ubuntu-18-04
[Configuration Management 101_ Writing Ansible Playbooks]: https://www.digitalocean.com/community/tutorials/configuration-management-101-writing-ansible-playbooks
[Intro to Playbooks]: https://docs.ansible.com/ansible/latest/user_guide/playbooks_intro.html
[https_www.digitalocean.com_community_tutorials_how-to-set-up-and-secure-an-etcd-cluster-with-ansible-on-ubuntu-18-04]: https://www.digitalocean.com/community/tutorials/how-to-set-up-and-secure-an-etcd-cluster-with-ansible-on-ubuntu-18-04