CentOS-7安装kubernetes+etcd

￡神魔★判官ぃ 2022-03-26 09:46 231阅读 0赞

## 前期准备 ##

### 配置hosts ###

>     192.168.245.227 iot-node1 \# master节点(etcd,kubernetes-master)
>     192.168.245.228 iot-node2 \# slave节点(etcd,kubernetes-node,docker)
>     192.168.245.229 iot-node3 \# slave节点(etcd,kubernetes-node,docker)

### 其他 ###

同CentOS-7安装mesos+marathon+zk前期准备

## 添加安装源 ##

### 添加etcd安装源 ###

> sudo rpm -Uvh  
> [https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm][https_dl.fedoraproject.org_pub_epel_epel-release-latest-7.noarch.rpm]

> sudo rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6

### 添加kubernetes安装源 ###

> sudo tee /etc/yum.repos.d/kubernetes.repo <<-‘EOF’  
> \[kubernetesrepo\]  
> name=Kubernetes Repository  
> baseurl=http://cbs.centos.org/repos/virt7-kubernetes-110-release/x86\_64/os  
> enabled=1  
> gpgcheck=1  
> gpgkey=https://yum.dockerproject.org/gpg  
> EOF

## 在线安装etcd ##

### 安装etcd ###

> sudo yum install -y etcd

### 配置etcd ###

> sudo vi /etc/etcd/etcd.conf

1）节点iot-node1中/etc/etcd/etcd.conf内容：

> \#\[Member\]  
> ETCD\_DATA\_DIR="/var/lib/etcd"  
> ETCD\_LISTEN\_PEER\_URLS=“[http://192.168.245.227:2380][http_192.168.245.227_2380]”  
> ETCD\_LISTEN\_CLIENT\_URLS=“[http://192.168.245.227:2379][http_192.168.245.227_2379],[http://127.0.0.1:2379][http_127.0.0.1_2379]”  
> ETCD\_NAME=“etcd1”  
> \#\[Clustering\]  
> ETCD\_INITIAL\_ADVERTISE\_PEER\_URLS=“[http://192.168.245.227:2380][http_192.168.245.227_2380]”  
> ETCD\_ADVERTISE\_CLIENT\_URLS=“[http://192.168.245.227:2379][http_192.168.245.227_2379]”  
> ETCD\_INITIAL\_CLUSTER=“etcd1=http://192.168.245.227:2380,etcd2=http://192.168.245.228:2380, etcd3=http://192.168.245.229:2380”  
> ETCD\_INITIAL\_CLUSTER\_TOKEN=“etcd-cluster”  
> ETCD\_INITIAL\_CLUSTER\_STATE=“new”

2）节点iot-node2中/etc/etcd/etcd.conf内容：

> \#\[Member\]  
> ETCD\_DATA\_DIR="/var/lib/etcd"  
> ETCD\_LISTEN\_PEER\_URLS=“[http://192.168.245.228:2380][http_192.168.245.228_2380]”  
> ETCD\_LISTEN\_CLIENT\_URLS="[http://192.168.245.228:2379][http_192.168.245.228_2379],[http://127.0.0.1:2379][http_127.0.0.1_2379] "  
> ETCD\_NAME=“etcd1”  
> \#\[Clustering\]  
> ETCD\_INITIAL\_ADVERTISE\_PEER\_URLS=“[http://192.168.245.228:2380][http_192.168.245.228_2380]”  
> ETCD\_ADVERTISE\_CLIENT\_URLS=“[http://192.168.245.228:2379][http_192.168.245.228_2379]”  
> ETCD\_INITIAL\_CLUSTER=“etcd1=http://192.168.245.227:2380,etcd2=http://192.168.245.228:2380,  
> etcd3=http://192.168.245.229:2380”  
> ETCD\_INITIAL\_CLUSTER\_TOKEN=“etcd-cluster”  
> ETCD\_INITIAL\_CLUSTER\_STATE=“new”

3）节点iot-node3中/etc/etcd/etcd.conf内容：

> \#\[Member\]  
> ETCD\_DATA\_DIR="/var/lib/etcd"  
> ETCD\_LISTEN\_PEER\_URLS=“[http://192.168.245.229:2380][http_192.168.245.229_2380]”  
> ETCD\_LISTEN\_CLIENT\_URLS=“[http://192.168.245.229:2379][http_192.168.245.229_2379],[http://127.0.0.1:2379][http_127.0.0.1_2379]”  
> ETCD\_NAME=“etcd1”  
> \#\[Clustering\]  
> ETCD\_INITIAL\_ADVERTISE\_PEER\_URLS=“[http://192.168.245.229:2380][http_192.168.245.229_2380]”  
> ETCD\_ADVERTISE\_CLIENT\_URLS=“[http://192.168.245.229:2379][http_192.168.245.229_2379]”  
> ETCD\_INITIAL\_CLUSTER=“etcd1=http://192.168.245.227:2380,etcd2=http://192.168.245.228:2380,  
> etcd3=http://192.168.245.229:2380”  
> ETCD\_INITIAL\_CLUSTER\_TOKEN=“etcd-cluster”  
> ETCD\_INITIAL\_CLUSTER\_STATE=“new”

顺序启动3个节点的etcd，iot-node1、iot-node2两个节点启动etcd会报超  
时，等所有节点都启动后就没问题了。

针对几个URLS做下简单的解释：

> \[member\]  
> ETCD\_NAME ：ETCD的节点名  
> ETCD\_DATA\_DIR：ETCD的数据存储目录  
> ETCD\_SNAPSHOT\_COUNTER：多少次的事务提交将触发一次快照  
> ETCD\_HEARTBEAT\_INTERVAL：ETCD节点之间心跳传输的间隔，单位毫秒  
> ETCD\_ELECTION\_TIMEOUT：该节点参与选举的最大超时时间，单位毫秒  
> ETCD\_LISTEN\_PEER\_URLS：该节点与其他节点通信时所监听的地址列表，多个地址使用逗号隔开，其格式可以划分为scheme://IP:PORT，这里的scheme可以是http、https  
> ETCD\_LISTEN\_CLIENT\_URLS：该节点与客户端通信时监听的地址列表  
> \[cluster\]  
> ETCD\_INITIAL\_ADVERTISE\_PEER\_URLS：该成员节点在整个集群中的通信地址列表，这个地址用来传输集群数据的地址。因此这个地址必须是可以连接集群中所有的成员的。  
> ETCD\_INITIAL\_CLUSTER：配置集群内部所有成员地址，其格式为：ETCD\_NAME=ETCD\_INITIAL\_ADVERTISE\_PEER\_URLS，如果有多个使用逗号隔开  
> ETCD\_ADVERTISE\_CLIENT\_URLS：广播给集群中其他成员自己的客户端地址列表

### 启动/停止/查看etcd ###

> sudo systemctl start etcd

> sudo systemctl stop etcd

> sudo systemctl status etcd

### 设置开机启动 ###

> sudo systemctl enable etcd

### 查看etcd启动日志 ###

> sudo journalctl -xef

### 查看etcd集群节点 ###

> etcdctl member list

### 查看etcd集群状态 ###

> etcdctl cluster-health

## 在线安装kubernetes ##

### 安装kubernetes ###

1）master节点安装

> sudo yum install -y kubernetes-master

2）slave节点安装

> sudo yum install -y kubernetes-node

### 配置kubernetes ###

1）所有节点配置config

> sudo vi /etc/kubernetes/config

/etc/kubernetes/config内容：

> \#\#\#  
> \# kubernetes system config  
> \#  
> \# The following values are used to configure various aspects of all  
> \# kubernetes services, including  
> \#  
> \# kube-apiserver.service  
> \# kube-controller-manager.service  
> \# kube-scheduler.service  
> \# kubelet.service  
> \# kube-proxy.service  
> \# logging to stderr means we get it in the systemd journal  
> KUBE\_LOGTOSTDERR="–logtostderr=true"  
> \# journal message level, 0 is debug  
> KUBE\_LOG\_LEVEL="–v=0"  
> \# Should this cluster be allowed to run privileged docker containers  
> KUBE\_ALLOW\_PRIV="–allow-privileged=false"  
> \# How the controller-manager, scheduler, and proxy find the apiserver  
> KUBE\_MASTER="–master=http://192.168.245.227:8080"

2）master节点配置apiserver

> sudo vi /etc/kubernetes/apiserver

/etc/kubernetes/apiserver内容：

> \# kubernetes system config  
> \#  
> \# The following values are used to configure the kube-apiserver  
> \#  
> \# The address on the local server to listen to.  
> KUBE\_API\_ADDRESS="–address=0.0.0.0"  
> \# The port on the local server to listen on.  
> KUBE\_API\_PORT="–port=8080"  
> \# Port minions listen on  
> KUBELET\_PORT="–kubelet-port=10250"  
> \# Comma separated list of nodes in the etcd cluster  
> KUBE\_ETCD\_SERVERS="–etcd-servers=http:// 192.168.245.227:2379, http://  
> 192.168.245.228:2379,http:// 192.168.245.229:2379 "  
> \# Address range to use for services  
> KUBE\_SERVICE\_ADDRESSES="–service-cluster-ip-range=172.16.0.0/24"  
> \# default admission control policies  
> KUBE\_ADMISSION\_CONTROL="–admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,ResourceQuota"  
> \# Add your own!  
> KUBE\_API\_ARGS="–service-node-port-range=1-65535 --insecure-bind-address=0.0.0.0"

2）master节点配置controller-manager

> sudo vi /etc/kubernetes/controller-manager

/etc/kubernetes/controller-manager内容：

> \#\#\#  
> \# The following values are used to configure the kubernetes controller-manager  
> \# defaults from config and apiserver should be adequate  
> \# Add your own!  
> KUBE\_CONTROLLER\_MANAGER\_ARGS=""

3）master节点配置scheduler

> sudo vi /etc/kubernetes/scheduler

/etc/kubernetes/scheduler：

> \#\#\#  
> \# The following values are used to configure the kubernetes scheduler  
> \# defaults from config and apiserver should be adequate  
> \# Add your own!  
> KUBE\_SCHEDULER\_ARGS=""

4）slave节点配置kubelet

> sudo vi /etc/kubernetes/kubelet

/etc/kubernetes/kubelet内容：

> \#\#\#  
> \# kubernetes kubelet (minion) config  
> \# The address for the info server to serve on (set to 0.0.0.0 or “” for all  
> interfaces)  
> KUBELET\_ADDRESS="–address=0.0.0.0"  
> \# The port for the info server to serve on  
> \# KUBELET\_PORT="–port=10250"  
> \# You may leave this blank to use the actual hostname  
> KUBELET\_HOSTNAME="–hostname-override=192.168.245.228"  
> \# location of the api-server  
> \# KUBELET\_API\_SERVER="–api-servers=http://192.168.245.227:8080"  
> \# pod infrastructure container  
> KUBELET\_POD\_INFRA\_CONTAINER="–pod-infra-container-image=[registry.access.redhat.com/rhel7/pod-infrastructure:latest][registry.access.redhat.com_rhel7_pod-infrastructure_latest]"  
> \# Add your own!  
> KUBELET\_ARGS="–kubeconfig=/etc/kubernetes/kubeconfig  
> –runtime-cgroups=/systemd/system.slice  
> –kubelet-cgroups=/systemd/system.slice --cluster-dns=172.16.0.0  
> –cgroup-driver=systemd --enable-server=true  
> –enable-debugging-handlers=true"

注意：

不同slave需修改KUBELET\_HOSTNAME

这里需要说明的是kubernetesV1.9  
开始已经不支持–api-servers参数，需要指定-- kubeconfig=/etc/kubernetes/kubeconfig配置文件来指明master的地址，否则master主机执行kubectl get nodes，会提示找不到节点

5）slave节点配置kubeconfig

> sudo vi /etc/kubernetes/kubeconfig

/etc/kubernetes/kubeconfig内容：

> apiVersion: v1  
> kind: Config  
> users:  
> \- name: kubelet  
> clusters:  
> \- name: kubernetes  
> cluster:  
> server: [http://192.168.245.227:8080][http_192.168.245.227_8080]  
> contexts:  
> \- context:  
> cluster: kubernetes  
> user: kubelet  
> name: service-account-context  
> current-context: service-account-context

6）slave节点配置proxy

> sudo vi /etc/kubernetes/proxy

/etc/kubernetes/proxy内容：

> \# kubernetes proxy config  
> \# default config should be adequate  
> \# Add your own!  
> KUBE\_PROXY\_ARGS="–bind-address=192.168.245.228  
> –hostname-override=192.168.245.228"

注意：不同slave需修改KUBE\_PROXY\_ARGS中的ip

7）修改docker cgroupdriver

> sudo vi /etc/docker/daemon.json

/etc/docker/daemon.json内容：

> \{  
> “storage-driver”: “devicemapper”,  
> “storage-opts”: \[  
> “dm.thinpooldev=/dev/mapper/docker-thinpool”,  
> “dm.use\_deferred\_removal=true”,  
> “dm.use\_deferred\_deletion=true”\]  
> , “exec-opts”:\[ “native.cgroupdriver=systemd” \]  
> \}

### master节点启动服务 ###

> sudo systemctl daemon-reload

> sudo systemctl start kube-apiserver

> sudo systemctl start kube-controller-manager

> sudo systemctl start kube-scheduler

### master节点开机启动 ###

> sudo systemctl enable kube-apiserver

> sudo systemctl enable kube-controller-manager

> sudo systemctl enable kube-scheduler

### slave节点启动服务 ###

> sudo systemctl daemon-reload

> sudo systemctl start kubelet

> sudo systemctl start kube-proxy

### slave节点开机启动 ###

> sudo systemctl enable kubelet

> sudo systemctl enable kube-proxy

### 验证kubernetes集群 ###

在master节点，执行

> kubectl get nodes

在master节点，执行

> kubectl get node --show-labels=true

在master节点，执行

> kubectl get cs

## 离线安装etcd ##

### 下载地址 ###

> [http://www.rpmfind.net/linux/centos/7.5.1804/extras/x86\_64/Packages/][http_www.rpmfind.net_linux_centos_7.5.1804_extras_x86_64_Packages]

### 安装包 ###

> etcd-3.2.22-1.el7.x86\_64.rpm

### 安装etcd ###

进入到etcd安装包和依赖包所在目录执行命令：

> sudo rpm -ivh \*.rpm

### 配置etcd ###

相关配置参考etcd在线安装

## 离线安装kubernetes ##

### 下载地址 ###

> [http://cbs.centos.org/repos/virt7-kubernetes-110-release/x86\_64/os/Packages/][http_cbs.centos.org_repos_virt7-kubernetes-110-release_x86_64_os_Packages]

### 安装包 ###

> kubernetes-master-1.10.3-0.el7.x86\_64.rpm  
> kubernetes-client-1.10.3-0.el7.x86\_64.rpm  
> kubernetes-node-1.10.3-0.el7.x86\_64.rpm

### 依赖包 ###

> PyYAML-3.10-11.el7.x86\_64.rpm  
> atomic-registries-1.22.1-22.git5a342e3.el7.x86\_64.rpm  
> checkpolicy-2.5-6.el7.x86\_64.rpm  
> conntrack-tools-1.4.4-3.el7\_3.x86\_64.rpm  
> audit-libs-python-2.8.1-3.el7\_5.1.x86\_64.rpm  
> container-storage-setup-0.11.0-2.git5eaf76c.el7.noarch.rpm  
> container-selinux-2.68-1.el7.noarch.rpm  
> docker-client-1.13.1-74.git6e3bb8e.el7.centos.x86\_64.rpm  
> docker-common-1.13.1-74.git6e3bb8e.el7.centos.x86\_64.rpm  
> libnetfilter\_cthelper-1.0.0-9.el7.x86\_64.rpm  
> libnetfilter\_cttimeout-1.0.0-6.el7.x86\_64.rpm  
> libnetfilter\_queue-1.0.2-2.el7\_2.x86\_64.rpm  
> libseccomp-2.3.1-3.el7.x86\_64.rpm  
> libcgroup-0.41-15.el7.x86\_64.rpm  
> libsemanage-python-2.5-11.el7.x86\_64.rpm  
> docker-1.13.1-74.git6e3bb8e.el7.centos.x86\_64.rpm  
> libyaml-0.1.4-11.el7\_0.x86\_64.rpm  
> oci-register-machine-0-6.git2b44233.el7.x86\_64.rpm  
> oci-umount-2.3.3-3.gite3c9055.el7.x86\_64.rpm  
> oci-systemd-hook-0.1.17-2.git83283a0.el7.x86\_64.rpm  
> python-IPy-0.75-6.el7.noarch.rpm  
> policycoreutils-python-2.5-22.el7.x86\_64.rpm  
> python-backports-1.0-8.el7.x86\_64.rpm  
> python-backports-ssl\_match\_hostname-3.5.0.1-1.el7.noarch.rpm  
> python-ipaddress-1.0.16-2.el7.noarch.rpm  
> python-setuptools-0.9.8-7.el7.noarch.rpm  
> skopeo-containers-0.1.31-1.dev.gitae64ff7.el7.centos.x86\_64.rpm  
> setools-libs-3.3.8-2.el7.x86\_64.rpm  
> python-pytoml-0.1.14-1.git7dea353.el7.noarch.rpm  
> socat-1.7.3.2-2.el7.x86\_64.rpm  
> subscription-manager-rhsm-certificates-1.20.11-1.el7.centos.x86\_64.rpm  
> yajl-2.0.4-4.el7.x86\_64.rpm

### 安装kubernetes ###

进入到kubernetes安装包和依赖包所在目录执行命令：

> sudo rpm -ivh \*.rpm

### 配置kubernetes ###

相关配置参考kubernetes在线安装

[https_dl.fedoraproject.org_pub_epel_epel-release-latest-7.noarch.rpm]: https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
[http_192.168.245.227_2380]: http://192.168.245.227:2380
[http_192.168.245.227_2379]: http://192.168.245.227:2379
[http_127.0.0.1_2379]: http://127.0.0.1:2379
[http_192.168.245.228_2380]: http://192.168.245.228:2380
[http_192.168.245.228_2379]: http://192.168.245.228:2379
[http_192.168.245.229_2380]: http://192.168.245.229:2380
[http_192.168.245.229_2379]: http://192.168.245.229:2379
[registry.access.redhat.com_rhel7_pod-infrastructure_latest]: http://registry.access.redhat.com/rhel7/pod-infrastructure:latest
[http_192.168.245.227_8080]: http://192.168.245.227:8080
[http_www.rpmfind.net_linux_centos_7.5.1804_extras_x86_64_Packages]: http://www.rpmfind.net/linux/centos/7.5.1804/extras/x86_64/Packages/
[http_cbs.centos.org_repos_virt7-kubernetes-110-release_x86_64_os_Packages]: http://cbs.centos.org/repos/virt7-kubernetes-110-release/x86_64/os/Packages/