【Kubernetes】Kubernetes之Kubeadm部署

亦凉 2024-03-23 19:04 58阅读 0赞

#### Kubeadm ####

*  一、Kubeadm 部署
 *   *  1. 环境准备
     *  2. 所有节点安装 docker
     *  3. 所有节点安装 kubeadm，kubelet 和 kubectl
     *  4. 部署 K8S 集群
     *   *  4.1 配置 master01 节点
         *  4.2 配置 node 节点
 *  二、Kubeadm 高可用部署
 *   *  1. 环境准备
     *  2. 所有节点安装 docker
     *  2. 所有节点安装kubeadm，kubelet和kubectl
     *  3. 高可用组件安装、配置
     *  4. 部署 K8S 集群
 *  总结
 *   *  kubeadm 部署过程
     *  kubeadm 部署的K8S集群更新证书

--------------------

## 一、Kubeadm 部署 ##

<table> 
 <thead> 
  <tr> 
   <th>集群服务器主机名</th> 
   <th>服务器IP地址</th> 
   <th>集群服务器部署的服务</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>master（2C/4G，cpu核心数要求大于2）</td> 
   <td>192.168.145.15</td> 
   <td>docker、kubeadm、kubelet、kubectl、flannel</td> 
  </tr> 
  <tr> 
   <td>node01（2C/2G）</td> 
   <td>192.168.145.30</td> 
   <td>docker、kubeadm、kubelet、kubectl、flannel</td> 
  </tr> 
  <tr> 
   <td>node02（2C/2G）</td> 
   <td>192.168.145.45</td> 
   <td>docker、kubeadm、kubelet、kubectl、flannel</td> 
  </tr> 
 </tbody> 
</table>

1、在所有节点上安装Docker和kubeadm
    2、部署Kubernetes Master
    3、部署容器网络插件
    4、部署 Kubernetes Node，将节点加入Kubernetes集群中

### 1. 环境准备 ###

#所有节点，关闭防火墙规则，关闭selinux，关闭swap交换
    systemctl stop firewalld
    systemctl disable firewalld
    setenforce 0
    sed -i 's/enforcing/disabled/' /etc/selinux/config
    iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X
    swapoff -a								#交换分区必须要关闭
    sed -ri 's/.*swap.*/#&/' /etc/fstab		#永久关闭swap分区，&符号在sed命令中代表上次匹配的结果
    #加载 ip_vs 模块
    for i in $(ls /usr/lib/modules/$(uname -r)/kernel/net/netfilter/ipvs|grep -o "^[^.]*");do echo $i; /sbin/modinfo -F filename $i >/dev/null 2>&1 && /sbin/modprobe $i;done

![在这里插入图片描述][8300c472e93b46348531a5b9673294df.png_pic_center]

#修改主机名
    hostnamectl set-hostname master01
    hostnamectl set-hostname node01
    hostnamectl set-hostname node02
    
    #所有节点修改hosts文件
    vim /etc/hosts
    192.168.145.15 master01
    192.168.145.30 node01
    192.168.145.45 node02

![在这里插入图片描述][1d07ce1db1084f1db5e0916cf711a6b6.png_pic_center]

#调整内核参数
    cat > /etc/sysctl.d/kubernetes.conf << EOF
    #开启网桥模式，可将网桥的流量传递给iptables链
    net.bridge.bridge-nf-call-ip6tables=1
    net.bridge.bridge-nf-call-iptables=1
    #关闭ipv6协议
    net.ipv6.conf.all.disable_ipv6=1
    net.ipv4.ip_forward=1
    EOF
    
    #生效参数
    sysctl --system

![在这里插入图片描述][e630b38770274b1c91ceab004a4d2b86.png_pic_center]

### 2. 所有节点安装 docker ###

![在这里插入图片描述][19a58c31ebd84fe4b9b04d1890c42e06.png_pic_center]

cd /etc/docker
    cat > /etc/docker/daemon.json <<EOF
    {
      "registry-mirrors": ["https://00ub0bmk.mirror.aliyuncs.com"],
      "exec-opts": ["native.cgroupdriver=systemd"],
      "log-driver": "json-file",
      "log-opts": {
        "max-size": "500m", "max-file": "3"
      }
    }
    EOF
    #使用Systemd管理的Cgroup来进行资源控制与管理，因为相对Cgroupfs而言，Systemd限制CPU、内存等资源更加简单和成熟稳定。
    #日志使用json-file格式类型存储，大小为100M，保存在/var/log/containers目录下，方便ELK等日志系统收集和管理日志。

![在这里插入图片描述][a9e66218c4f54e8fbf76f45dd34ce778.png_pic_center]

systemctl daemon-reload				#重载配置文件
    systemctl start docker.service		#启动 docker 服务
    systemctl enable docker.service		#将 docker 服务设置开机自启
    
    docker info | grep "Cgroup Driver"	#查看 dokcer 的配置信息
    Cgroup Driver: systemd

![在这里插入图片描述][3c9d6feaacac4528aa6bee6c36b84aeb.png_pic_center]

### 3. 所有节点安装 kubeadm，kubelet 和 kubectl ###

#定义kubernetes源
    cat > /etc/yum.repos.d/kubernetes.repo << EOF
    [kubernetes]
    name=Kubernetes
    baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
    enabled=1
    gpgcheck=0
    repo_gpgcheck=0
    gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
    EOF
    
    yum install -y kubelet-1.20.15 kubeadm-1.20.15 kubectl-1.20.15
    
    #开机自启kubelet
    systemctl enable kubelet.service
    #K8S通过kubeadm安装出来以后都是以Pod方式存在，即底层是以容器方式运行，所以kubelet必须设置开机自启

![在这里插入图片描述][9994eecd9da64f59a6597c23df1dd673.png_pic_center]

### 4. 部署 K8S 集群 ###

#### 4.1 配置 master01 节点 ####

#查看初始化需要的镜像
    kubeadm config images list --kubernetes-version 1.20.15

![在这里插入图片描述][3f17c0aee3284b72bea9514d067b6e1d.png_pic_center]

#初始化kubeadm
    kubeadm config print init-defaults > /opt/kubeadm-config.yaml
    
    cd /opt/
    vim kubeadm-config.yaml
    ......
    11 localAPIEndpoint:
    12   advertiseAddress: 192.168.145.15		#指定master节点的IP地址
    13   bindPort: 6443
    ......
    32 imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers          #指定拉取镜像的仓库，默认是k8s.gcr.io
    33 kind: ClusterConfiguration
    34 kubernetesVersion: v1.20.15				#指定kubernetes版本号
    35 networking:
    36   dnsDomain: cluster.local
    37   podSubnet: 10.244.0.0/16				#指定pod网段，10.244.0.0/16用于匹配flannel默认网段
    38   serviceSubnet: 10.96.0.0/16			#指定service网段
    39 scheduler: {}
    #末尾再添加以下内容
    --- 
    apiVersion: kubeproxy.config.k8s.io/v1alpha1
    kind: KubeProxyConfiguration
    mode: ipvs									#把默认的kube-proxy调度方式改为ipvs模式

![在这里插入图片描述][26eec1ee3adb4e3f952e0ca8731dfd4b.png_pic_center]

#在线拉取镜像
    kubeadm config images pull --config /opt/kubeadm-config.yaml
    #查看镜像
    docker images

![在这里插入图片描述][4d96cfe4a96c49ed8f21f2b55969d477.png_pic_center]

#初始化 master
    kubeadm init --config=/opt/kubeadm-config.yaml --upload-certs | tee kubeadm-init.log
    #--upload-certs 参数可以在后续执行加入节点时自动分发证书文件
    #tee kubeadm-init.log 用以输出日志
    
    #查看 kubeadm-init 日志
    less kubeadm-init.log
    
    #kubernetes配置文件目录
    ls /etc/kubernetes/
    
    #存放ca等证书和密码的目录
    ls /etc/kubernetes/pki

![在这里插入图片描述][87df08ea46bd40d5b4e3781cb4ee2e17.png_pic_center]

#设定kubectl
    kubectl需经由API server认证及授权后方能执行相应的管理操作，kubeadm 部署的集群为其生成了一个具有管理员权限的认证配置文件 /etc/kubernetes/admin.conf，它可由 kubectl 通过默认的 “$HOME/.kube/config” 的路径进行加载。
    
    mkdir -p $HOME/.kube									#在家目录中新键 .kube 目录
    cp -i /etc/kubernetes/admin.conf $HOME/.kube/config		#将自动生成的 admin.conf 文件复制到家目录中 ./kube 目录
    chown $(id -u):$(id -g) $HOME/.kube/config				#给 config 文件授权

![在这里插入图片描述][74d1f949fb134612850f397fe2d8a7b7.png_pic_center]

#部署网络插件flannel
    #上传 flannel 镜像 flannel.tar 和网络插件 cni-plugins-linux-amd64-v0.8.6.tgz 到 /opt 目录，master节点上传 kube-flannel.yml 文件
    cd /opt/
    unzip flannel-v0.21.5.zip
    
    docker load -i flannel.tar
    docker load -i flannel-cni-plugin.tar

![在这里插入图片描述][858991bf834040c488702c0dd8abdd52.png_pic_center])

mv /opt/cni /opt/cni_bak
    mkdir -p /opt/cni/bin
    tar xf cni-plugins-linux-amd64-v0.8.6.tgz -C /opt/cni/bin

![在这里插入图片描述][c7ea914bbd9843d885909dd84a093d4b.png_pic_center]

#在 master 节点创建 flannel 资源
    kubectl apply -f kube-flannel.yml 
    
    #在master节点查看节点状态
    kubectl get nodes
    
    kubectl get pods -A
    NAMESPACE      NAME                               READY   STATUS    RESTARTS   AGE
    kube-flannel   kube-flannel-ds-xffxg              1/1     Running   0          24s
    kube-system    coredns-54d67798b7-7jjll           1/1     Running   0          9m49s
    kube-system    coredns-54d67798b7-txdlc           1/1     Running   0          9m49s
    kube-system    etcd-master01                      1/1     Running   0          9m58s
    kube-system    kube-apiserver-master01            1/1     Running   0          9m58s
    kube-system    kube-controller-manager-master01   1/1     Running   0          9m58s
    kube-system    kube-proxy-ll5bh                   1/1     Running   0          9m49s
    kube-system    kube-scheduler-master01            1/1     Running   0          9m58s

![在这里插入图片描述][7964c0974794420f86d9b9e20cf86311.png_pic_center]

#### 4.2 配置 node 节点 ####

#将 master01 节点上的 flannel 相关文件传送到所有 node 节点
    scp flannel*tar node01:/opt		#在master操作
    scp flannel*tar node02:/opt		#在master操作

![在这里插入图片描述][e674290571a647ff9464a4c5a432b43c.png_pic_center]

#将node节点根据根据kubeadm init初始化的提示信息快速的加入到K8S集群当中
    kubeadm join 192.168.145.15:6443 --token abcdef.0123456789abcdef \
    >     --discovery-token-ca-cert-hash sha256:508d252e11aadcbe12da889eb90038a6f72d8aa4922f166a1fa4f765bfe723ce

![在这里插入图片描述][1e11f584b85b45448be6022107b90430.png_pic_center]

![在这里插入图片描述][e771e053a0394cb39bd249082b5083cb.png_pic_center]

#部署网络插件flannel
    cd /opt/
    docker load -i flannel.tar
    docker load -i flannel-cni-plugin.tar

![在这里插入图片描述][8c91b23b002646119157921f70c02df4.png_pic_center]

![在这里插入图片描述][0d754fcea0f245d4a360e67a65ec1206.png_pic_center]

#在master节点查看节点状态
    kubectl get nodes
    
    kubectl get pods -A

![在这里插入图片描述][6a07df51b0d0409bb4514d76f351e4a2.png_pic_center]

## 二、Kubeadm 高可用部署 ##

#注意事项：
    master节点cpu核心数要求大于2
    ● 最新的版本不一定好，但相对于旧版本，核心功能稳定，但新增功能、接口相对不稳
    ● 学会一个版本的 高可用部署，其他版本操作都差不多
    ● 宿主机尽量升级到CentOS 7.9
    ● 内核kernel升级到 4.19+ 这种稳定的内核
    ● 部署k8s版本时，尽量找 1.xx.5 这种大于5的小版本（这种一般是比较稳定的版本）

<table> 
 <thead> 
  <tr> 
   <th>集群服务器主机名</th> 
   <th>服务器IP地址</th> 
   <th>集群服务器部署的服务</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>master01（2C/4G，cpu核心数要求大于2）</td> 
   <td>192.168.145.15</td> 
   <td>docker、kubeadm、kubelet、kubectl、flannel</td> 
  </tr> 
  <tr> 
   <td>master02（2C/4G，cpu核心数要求大于2）</td> 
   <td>192.168.145.30</td> 
   <td>docker、kubeadm、kubelet、kubectl、flannel</td> 
  </tr> 
  <tr> 
   <td>master03（2C/4G，cpu核心数要求大于2）</td> 
   <td>192.168.145.45</td> 
   <td>docker、kubeadm、kubelet、kubectl、flannel</td> 
  </tr> 
  <tr> 
   <td>node01（2C/2G）</td> 
   <td>192.168.145.60</td> 
   <td>docker、kubeadm、kubelet、kubectl、flannel</td> 
  </tr> 
  <tr> 
   <td>node02（2C/2G）</td> 
   <td>192.168.145.75</td> 
   <td>docker、kubeadm、kubelet、kubectl、flannel</td> 
  </tr> 
 </tbody> 
</table>

### 1. 环境准备 ###

![在这里插入图片描述][749bb7099aad4fc39a9f85fce15fb157.png_pic_center]

#修改主机名
    hostnamectl set-hostname master01
    hostnamectl set-hostname master02
    hostnamectl set-hostname master03
    hostnamectl set-hostname node01
    hostnamectl set-hostname node02
    
    
    #所有节点修改hosts文件
    vim /etc/hosts
    192.168.145.15 master01
    192.168.145.30 master02
    192.168.145.45 master03
    192.168.145.60 node01
    192.168.145.75 node02

![在这里插入图片描述][9078701aa135476a87252fb14b6b1ff5.png_pic_center]

#所有节点时间同步
    yum -y install ntpdate
    ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
    echo 'Asia/Shanghai' >/etc/timezone
    ntpdate time2.aliyun.com
    
    systemctl enable --now crond
    
    crontab -e
    */30 * * * * /usr/sbin/ntpdate time2.aliyun.com
    
    #所有节点实现Linux的资源限制
    vim /etc/security/limits.conf
    * soft nofile 65536
    * hard nofile 131072
    * soft nproc 65535
    * hard nproc 655350
    * soft memlock unlimited
    * hard memlock unlimited

![在这里插入图片描述][36f0b3bc20ba4a129258bd32d675625a.png_pic_center]

#调整内核参数
    cat > /etc/sysctl.d/k8s.conf <<EOF
    net.ipv4.ip_forward = 1
    net.bridge.bridge-nf-call-iptables = 1
    net.bridge.bridge-nf-call-ip6tables = 1
    fs.may_detach_mounts = 1
    vm.overcommit_memory=1
    vm.panic_on_oom=0
    fs.inotify.max_user_watches=89100
    fs.file-max=52706963
    fs.nr_open=52706963
    net.netfilter.nf_conntrack_max=2310720
    
    net.ipv4.tcp_keepalive_time = 600
    net.ipv4.tcp_keepalive_probes = 3
    net.ipv4.tcp_keepalive_intvl =15
    net.ipv4.tcp_max_tw_buckets = 36000
    net.ipv4.tcp_tw_reuse = 1
    net.ipv4.tcp_max_orphans = 327680
    net.ipv4.tcp_orphan_retries = 3
    net.ipv4.tcp_syncookies = 1
    net.ipv4.tcp_max_syn_backlog = 16384
    net.ipv4.ip_conntrack_max = 65536
    net.ipv4.tcp_max_syn_backlog = 16384
    net.ipv4.tcp_timestamps = 0
    net.core.somaxconn = 16384
    EOF
    
    #生效参数
    sysctl --system  
    
    
    #加载 ip_vs 模块
    for i in $(ls /usr/lib/modules/$(uname -r)/kernel/net/netfilter/ipvs|grep -o "^[^.]*");do echo $i; /sbin/modinfo -F filename $i >/dev/null 2>&1 && /sbin/modprobe $i;done

![在这里插入图片描述][d1f121c6d4e344a193c96d586822c377.png_pic_center]

![在这里插入图片描述][623eec42fd5e4b2585b35d1734c60124.png_pic_center]

### 2. 所有节点安装 docker ###

yum install -y yum-utils device-mapper-persistent-data lvm2 
    yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo 
    yum install -y docker-ce docker-ce-cli containerd.io
    
    cd /etc/docker
    cat > /etc/docker/daemon.json <<EOF
    {
      "registry-mirrors": ["https://00ub0bmk.mirror.aliyuncs.com"],
      "exec-opts": ["native.cgroupdriver=systemd"],
      "log-driver": "json-file",
      "log-opts": {
        "max-size": "500m", "max-file": "3"
      }
    }
    EOF
    
    systemctl daemon-reload
    systemctl restart docker.service
    systemctl enable docker.service

![在这里插入图片描述][60297a8baa4e47bbbc16474838bc58d1.png_pic_center]

### 2. 所有节点安装kubeadm，kubelet和kubectl ###

#定义kubernetes源
    cat > /etc/yum.repos.d/kubernetes.repo << EOF
    [kubernetes]
    name=Kubernetes
    baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
    enabled=1
    gpgcheck=0
    repo_gpgcheck=0
    gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
    EOF
    
    yum install -y kubelet-1.20.15 kubeadm-1.20.15 kubectl-1.20.15
    
    #配置Kubelet使用阿里云的pause镜像
    cat > /etc/sysconfig/kubelet <<EOF
    KUBELET_EXTRA_ARGS="--cgroup-driver=systemd --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64:3.2"
    EOF
    
    #开机自启kubelet
    systemctl enable --now kubelet

![在这里插入图片描述][69c3a9185cb64ea6985f6cde78784ca0.png_pic_center]

### 3. 高可用组件安装、配置 ###

#所有 master 节点部署 Haproxy
    yum -y install haproxy keepalived
    
    cat > /etc/haproxy/haproxy.cfg << EOF
    global
        log         127.0.0.1 local0 info
        log         127.0.0.1 local1 warning
        chroot      /var/lib/haproxy
        pidfile     /var/run/haproxy.pid
        maxconn     4000
        user        haproxy
        group       haproxy
        daemon
        stats socket /var/lib/haproxy/stats
    
    defaults
        mode                    tcp
        log                     global
        option                  tcplog
        option                  dontlognull
        option                  redispatch
        retries                 3
        timeout queue           1m
        timeout connect         10s
        timeout client          1m
        timeout server          1m
        timeout check           10s
        maxconn                 3000
    
    frontend monitor-in
        bind *:33305
        mode http
        option httplog
        monitor-uri /monitor
    
    frontend k8s-master
        bind *:16443
        mode tcp
        option tcplog
        default_backend k8s-master
    
    backend k8s-master
        mode tcp
        option tcplog
        option tcp-check
        balance roundrobin
        server k8s-master1 192.168.145.15:6443  check inter 10000 fall 2 rise 2 weight 1
        server k8s-master2 192.168.145.30:6443  check inter 10000 fall 2 rise 2 weight 1
        server k8s-master3 192.168.145.45:6443  check inter 10000 fall 2 rise 2 weight 1
    EOF

![在这里插入图片描述][90f68b06c0ef417fb46869f4626f398c.png_pic_center]

#所有 master 节点部署 keepalived
    cd /etc/keepalived/
    vim keepalived.conf
    ! Configuration File for keepalived
    global_defs {
        router_id LVS_HA1			#路由标识符，每个节点配置不同
    }
    
    vrrp_script chk_haproxy {
        script "/etc/keepalived/check_haproxy.sh"
        interval 2
        weight 2
    }
    
    vrrp_instance VI_1 {
        state MASTER				#本机实例状态，MASTER/BACKUP，备机配置文件中设置BACKUP
        interface ens33
        virtual_router_id 51
        priority 100				#本机初始权重，备机设置小于主机的值
        advert_int 1
        virtual_ipaddress {
            192.168.145.100          #设置VIP地址
        }
        track_script {
            chk_haproxy
        }
    }

![在这里插入图片描述][14a36a7c9a0049da9421ba7d0a8a3b7c.png_pic_center]

![在这里插入图片描述][a31f64382a0744cc9cf3a7c055630518.png_pic_center]

![在这里插入图片描述][7d3c62d616ea41e7b6edfc3948141d30.png_pic_center]

![在这里插入图片描述][8d472b21d9884ecca4644d91a7d64b5b.png_pic_center]

vim check_haproxy.sh
    #!/bin/bash
    if ! killall -0 haproxy; then
        systemctl stop keepalived
    fi
    
    
    systemctl enable --now haproxy
    systemctl enable --now keepalived

![在这里插入图片描述][1b633c84eb574a66901e270a0e0c9c45.png_pic_center]

### 4. 部署 K8S 集群 ###

#在 master01 节点上设置集群初始化配置文件
    kubeadm config print init-defaults > /opt/kubeadm-config.yaml
    
    cd /opt/
    vim kubeadm-config.yaml
    ......
    11 localAPIEndpoint:
    12   advertiseAddress: 192.168.145.15		#指定当前master节点的IP地址
    13   bindPort: 6443
    
    21 apiServer:
    22   certSANs:								#在apiServer属性下面添加一个certsSANs的列表，添加所有master节点的IP地址和集群VIP地址
    23   - 192.168.145.100
    24   - 192.168.145.15
    25   - 192.168.145.30
    26   - 192.168.145.45
    
    30 clusterName: kubernetes
    31 controlPlaneEndpoint: "192.168.145.100:6443"		#指定集群VIP地址
    32 controllerManager: {}
    
    38 imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers			#指定镜像下载地址
    39 kind: ClusterConfiguration
    40 kubernetesVersion: v1.20.15				#指定kubernetes版本号
    41 networking:
    42   dnsDomain: cluster.local
    43   podSubnet: "10.244.0.0/16"				#指定pod网段，10.244.0.0/16用于匹配flannel默认网段
    44   serviceSubnet: 10.96.0.0/16			#指定service网段
    45 scheduler: {}
    #末尾再添加以下内容
    --- 
    apiVersion: kubeproxy.config.k8s.io/v1alpha1
    kind: KubeProxyConfiguration
    mode: ipvs									#把默认的kube-proxy调度方式改为ipvs模式
    
    
    #更新集群初始化配置文件
    kubeadm config migrate --old-config kubeadm-config.yaml --new-config new.yaml

![在这里插入图片描述][22a00cfa8deb478fac0e8cb66c4969e6.png_pic_center]

![在这里插入图片描述][45b030b796b8465a836e5026efa4d356.png_pic_center]

#所有节点拉取镜像
    #拷贝yaml配置文件给其他主机，通过配置文件进行拉取镜像
    for i in master02 master03 node01 node02; do scp /opt/new.yaml $i:/opt/; done
    
    kubeadm config images pull --config /opt/new.yaml

![在这里插入图片描述][aae487a3ee12418bb80fb4130c7fd3ad.png_pic_center]

#master01 节点进行初始化
    kubeadm init --config new.yaml --upload-certs | tee kubeadm-init.log
    Your Kubernetes control-plane has initialized successfully!
    
    To start using your cluster, you need to run the following as a regular user:
    
      mkdir -p $HOME/.kube
      sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
      sudo chown $(id -u):$(id -g) $HOME/.kube/config
    
    Alternatively, if you are the root user, you can run:
    
      export KUBECONFIG=/etc/kubernetes/admin.conf
    
    You should now deploy a pod network to the cluster.
    Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
      https://kubernetes.io/docs/concepts/cluster-administration/addons/
    
    You can now join any number of the control-plane node running the following command on each as root:
    #master节点加入使用的命令，记录！
      kubeadm join 192.168.145.100:6443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:63df6e626b225f6b60112bbc592d0ff69c1ae2e7869c9b41f142e4776df71d53 \
        --control-plane --certificate-key 0aad75a1c7f178755e4776e474e12616ab88086c93984ba311a465df1999d193
    
    Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
    As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
    "kubeadm init phase upload-certs --upload-certs" to reload certs afterward.
    
    Then you can join any number of worker nodes by running the following on each as root:
    #node节点加入使用的命令。记录！
    kubeadm join 192.168.145.100:6443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:63df6e626b225f6b60112bbc592d0ff69c1ae2e7869c9b41f142e4776df71d53

![在这里插入图片描述][180c49e51a2645f59258240d3f9d2907.png_pic_center]

#将其他节点加入集群中
    
    #将mater节点加入集群中
    kubeadm join 192.168.145.100:6443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:63df6e626b225f6b60112bbc592d0ff69c1ae2e7869c9b41f142e4776df71d53 \
        --control-plane --certificate-key 0aad75a1c7f178755e4776e474e12616ab88086c93984ba311a465df1999d193
    
    #将node节点加入集群中
    kubeadm join 192.168.145.100:6443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:63df6e626b225f6b60112bbc592d0ff69c1ae2e7869c9b41f142e4776df71d53

![在这里插入图片描述][564b90d8c90948fc8f294179930e78c2.png_pic_center]

#若初始化失败，进行的操作
    kubeadm reset -f
    ipvsadm --clear 
    rm -rf ~/.kube
    再次进行初始化

#master01 节点进行环境配置
    #配置 kubectl
    mkdir -p $HOME/.kube
    cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    chown $(id -u):$(id -g) $HOME/.kube/config
    
    #修改controller-manager和scheduler配置文件
    vim /etc/kubernetes/manifests/kube-scheduler.yaml 
    vim /etc/kubernetes/manifests/kube-controller-manager.yaml
    ......
        #- --port=0					#搜索port=0，把这一行注释掉
    
    systemctl restart kubelet

![在这里插入图片描述][851ac72c85ec45e4b822d66d6df4c61c.png_pic_center]

#部署网络插件flannel
    所有节点上传 flannel 镜像 flannel.tar 和网络插件 cni-plugins-linux-amd64-v0.8.6.tgz 到 /opt 目录，master节点上传 kube-flannel.yml 文件
    cd /opt
    unzip flannel-v0.21.5.zip
    docker load -i flannel.tar
    docker load -i flannel-cni-plugin.tar
    
    mv /opt/cni /opt/cni_bak
    mkdir -p /opt/cni/bin
    tar xf cni-plugins-linux-amd64-v1.3.0.tgz /opt/cni/bin
    
    kubectl apply -f kube-flannel.yml

![在这里插入图片描述][387197b184014efebcf9f7343400311b.png_pic_center]

#其他节点操作
    #配置 kubectl
    mkdir -p $HOME/.kube
    cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    chown $(id -u):$(id -g) $HOME/.kube/config
    
    kubectl apply -f kube-flannel.yml

![在这里插入图片描述][ee35d7d9ac4e4becb0d9447152686d3a.png_pic_center]

![在这里插入图片描述][9b61353dfd8943bc8f109870c532f363.png_pic_center]

#在 master01 查看集群信息
    kubectl get nodes
    
    kubectl get pod -n kube-system

![在这里插入图片描述][1f5c54a0ae1e40fd98dcad1e6c3e6747.png_pic_center]

## 总结 ##

### kubeadm 部署过程 ###

1）所有节点进行初始化,安装docker引擎和kubeadm kubelet kubectl
    2）生成集群初始化配置文件并进行修改
    3）使用kubeadm init根据初始化配置文件生成K8S的master控制管理节点
    4）安装CNI网络插件（flannel、calico等）
    5）在其他节点使用kubeadm join将节点以node

### kubeadm 部署的K8S集群更新证书 ###

1）备份老证书和kubeconfig配置文件
    mkdir /etc/kubernetes.bak
    cp -r /etc/kubernetes/pki/ /etc/kubernetes.bak
    cp /etc/kubernetes/*.conf /etc/kubernetes.bak
    
    2）重新生成证书
    kubeadm alpha certs renew all --config=kubeadm.yaml
    
    3）重新生成kubeconfig配置文件
    kubeadm init phase kubeconfig all --config kubeadm.yaml
    
    4）重启kubelet和其他K8S组件的Pod容器
    systemctl restart kubelet
    
    mv /etc/kubernetes/manifests  /tmp
    mv /tmp/*.yaml  /etc/kubernetes/manifests

[8300c472e93b46348531a5b9673294df.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/615cf56f853e490ca00912aa99122441.png
[1d07ce1db1084f1db5e0916cf711a6b6.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/04aaca6a4f2b45edbb3cf03526aafb78.png
[e630b38770274b1c91ceab004a4d2b86.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/e2fc5851714d4a4c9f30e8cea6080218.png
[19a58c31ebd84fe4b9b04d1890c42e06.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/92e95510b5d94d2ab10ad1165bbbee01.png
[a9e66218c4f54e8fbf76f45dd34ce778.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/64a7396464134f5db797f916b897fc57.png
[3c9d6feaacac4528aa6bee6c36b84aeb.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/e4d10ac1fd924b1aa6288bbb26801d3c.png
[9994eecd9da64f59a6597c23df1dd673.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/7285a36c57cb43c084ee7470908dd950.png
[3f17c0aee3284b72bea9514d067b6e1d.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/7101f7fc22764b319fbad2d16c9defe4.png
[26eec1ee3adb4e3f952e0ca8731dfd4b.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/ae7a8ff0bae843e29a91a9e568537e95.png
[4d96cfe4a96c49ed8f21f2b55969d477.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/e521861279cb4f32932815e5f7134550.png
[87df08ea46bd40d5b4e3781cb4ee2e17.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/ed08f51e4d4444508ec7f8551091c184.png
[74d1f949fb134612850f397fe2d8a7b7.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/857ca24aa6a14c93a8f81aafccc7ab16.png
[858991bf834040c488702c0dd8abdd52.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/2297c6a97a95448a9a2cf6faed147756.png
[c7ea914bbd9843d885909dd84a093d4b.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/ace8450b13bc437a891bf77f4c2b602c.png
[7964c0974794420f86d9b9e20cf86311.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/ce59b24f1fba4384866fb63ff7ea9e03.png
[e674290571a647ff9464a4c5a432b43c.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/2081752124274e37a33209071ba49314.png
[1e11f584b85b45448be6022107b90430.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/ceb9def0fe3446c2a3131c61707d14f4.png
[e771e053a0394cb39bd249082b5083cb.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/786fe0dd3ac2476eb195ac8870d58e3d.png
[8c91b23b002646119157921f70c02df4.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/e1b5e1c64d5849f687bbc7e15dd2223c.png
[0d754fcea0f245d4a360e67a65ec1206.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/5be141e8b81c489bb7f4162f6cf3dd89.png
[6a07df51b0d0409bb4514d76f351e4a2.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/0039085e1b5a4809bf5c4a698b3c7951.png
[749bb7099aad4fc39a9f85fce15fb157.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/e2fd927d6f3a4cf3bb783c903ef2fe76.png
[9078701aa135476a87252fb14b6b1ff5.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/d1abde53e4f74f26aefca6cefee8b550.png
[36f0b3bc20ba4a129258bd32d675625a.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/45c20bf1148a4312a898617963e58295.png
[d1f121c6d4e344a193c96d586822c377.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/48aca87427f14f6aa6c7c35a241dcb63.png
[623eec42fd5e4b2585b35d1734c60124.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/390228f7068c41189a9b4fd5a8ddcff6.png
[60297a8baa4e47bbbc16474838bc58d1.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/4a314ebc38a84a2c964350186b34f1bb.png
[69c3a9185cb64ea6985f6cde78784ca0.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/79143ec0eac24166a6525141a2532664.png
[90f68b06c0ef417fb46869f4626f398c.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/65ed75fa892d489f9d1322fcce914fbb.png
[14a36a7c9a0049da9421ba7d0a8a3b7c.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/c4a698ab03e943e1aa7d8eb2969c5c0d.png
[a31f64382a0744cc9cf3a7c055630518.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/ed1dd8a1cd444a759f206b91623c9826.png
[7d3c62d616ea41e7b6edfc3948141d30.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/d18423e022ff417aaefaa98b2a6479d5.png
[8d472b21d9884ecca4644d91a7d64b5b.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/c396244da1e441fcb0ce0bd387ed0596.png
[1b633c84eb574a66901e270a0e0c9c45.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/2cb4896ea10b49ad89d8544c4b482afa.png
[22a00cfa8deb478fac0e8cb66c4969e6.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/1ae8e20bc3fd4226825cb58d8e927892.png
[45b030b796b8465a836e5026efa4d356.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/ee7edce67a5a4ddcbcc5cf91c6847ff3.png
[aae487a3ee12418bb80fb4130c7fd3ad.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/da8fa601f1734a58bb92c15cc2c4c732.png
[180c49e51a2645f59258240d3f9d2907.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/ed2dbe0fa5b3409bbf6636e24a793474.png
[564b90d8c90948fc8f294179930e78c2.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/a79d51df76c24eedab857e0e175698fe.png
[851ac72c85ec45e4b822d66d6df4c61c.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/0e2dbb5cbbfc495aa9f98a354d231bc2.png
[387197b184014efebcf9f7343400311b.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/79401cd69431434d96b0f6ed6c4da670.png
[ee35d7d9ac4e4becb0d9447152686d3a.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/58436ae8d2cc42d59e5963608f2d18a9.png
[9b61353dfd8943bc8f109870c532f363.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/d16227883e8e408ab9ea5c68b2ce7096.png
[1f5c54a0ae1e40fd98dcad1e6c3e6747.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/03/14/56e7115c6899463f9f688a7d8fac9e08.png