升级Kubernetes版本至1.18

电玩女神 2022-12-16 06:16 222阅读 0赞

# #

继上篇我的博文升级Postman：[https://blog.csdn.net/dong19891210/article/details/106458776][https_blog.csdn.net_dong19891210_article_details_106458776]，这次计划升级Kubernetes，要跟上时代，看这个明显落伍了![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70][]

发现和官网的相差十个多等级

我就一一记录下升级步骤

1.  更改yum源

切换到

cd  /etc/yum.repos.d

然后删除里面的默认yum配置，不放心的话可以备份，然后执行下载yum源配置

wget http://mirrors.aliyun.com/repo/Centos-7.repo
    wget http://mirrors.163.com/.help/CentOS7-Base-163.repo
    #dokcer yum源
    wget  https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

最后记得执行以下命令，让新yum源生效：

yum clean all && yum makecache

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 1][]

2.    升级docker

先看下老版本的docker

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 2][]

请先卸载老版本的docker，要不然会出现

![20200602020556823.png][]

执行以下命令

yum list installed | grep docker

看装的docker列表

![20200602020737811.png][]

删除它们

yum remove -y docker*.x86_64

此时再次执行

yum list installed | grep docker

![20200602020933464.png][]

然后执行docker安装：

yum -y install docker-ce

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 3][]

下载进行中。。。。。。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 4][]

安装完毕，然后启动docker

systemctl start docker

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 5][]

最后设置docker开机自启动功能

systemctl enable docker

![20200602022044613.png][]

很熟悉，spring生态里有很多@Enable开头的注解，也是表示启用某功能的意思

至此，docker也已升级完毕！！！

顺便修改下机器主机名

hostnamectl set-hostname  master

![20200602022517140.png][]

再次打开一个窗口就能看的主机名已被改了

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 6][]

3  .升级Kubernetes

前提跟docker类似，记得卸载老版本的kubernetes

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 7][]

配置阿里yum源

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
    [kubernetes]
    name=Kubernetes
    baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
    enabled=1
    gpgcheck=1
    repo_gpgcheck=1
    gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
    EOF

然后再次生成yum信息·

yum clean all && yum makecache

然后执行命令

yum list kubelet --showduplicates | sort -r

查看哪个版本的kubernetes可用

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 8][]

有最新的，说明此yum源不错，很齐全

然后执行最新安装命令

yum install -y kubelet-1.18.3-0 kubeadm-1.18.3-0 kubectl-1.18.3-0

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 9][]

安装完毕![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 10][]

设置为开机自启动

systemctl enable kubelet && systemctl start kubelet

执行以下命令列出所需镜像，可以手动下载，也可以不用手动下载，master初始化时会自动下载。

kubeadm --kubernetes-version=v1.18.3 config images list

![20200602030414218.png][]

初始化master节点，执行以下命令

kubeadm init --apiserver-advertise-address=192.168.8.200 \
     --image-repository registry.aliyuncs.com/google_containers \
     --kubernetes-version v1.18.3 \
     --service-cidr=10.1.0.0/16 \
     --pod-network-cidr=10.244.0.0/16

出现以下问题![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 11][]

那就关掉swap：

swapoff -a

由于之前该机器已装过etcd

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 12][]

目录/var/lib/etcd/既然不空，那就强制删除

![20200602031959644.png][]

然后新建拉去镜像脚本images.sh,键入以下内容

# 下面的镜像应该去除"k8s.gcr.io/"的前缀
    images=(  
        kube-apiserver:v1.18.3
        kube-controller-manager:v1.18.3
        kube-scheduler:v1.18.3
        kube-proxy:v1.18.3
        pause:3.2
        etcd:3.4.3-0
        coredns:1.6.7
    )
    
    for imageName in ${images[@]} ; do
        docker pull mirrorgcrio/$imageName
        docker tag mirrorgcrio/$imageName k8s.gcr.io/$imageName
        docker rmi mirrorgcrio/$imageName
    done

然后执行

sh images.sh

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 13][]

慢慢拉镜像

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 14][]

再次执行初始化工作：

kubeadm init --apiserver-advertise-address=192.168.8.200 \
    --control-plane-endpoint="192.168.8.200:6443" \
     --kubernetes-version v1.18.3 \
     --service-cidr=10.1.0.0/16 \
     --pod-network-cidr=10.244.0.0/16 \
     --ignore-preflight-errors all

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 15][]

然后按照提示设置普通账户权限

mkdir -p $HOME/.kube
    cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    chown $(id -u):$(id -g) $HOME/.kube/config

接着部署网络插件flannel

创建网络配置文件kube-flannel.yml

---
    apiVersion: policy/v1beta1
    kind: PodSecurityPolicy
    metadata:
      name: psp.flannel.unprivileged
      annotations:
        seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default
        seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default
        apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
        apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default
    spec:
      privileged: false
      volumes:
        - configMap
        - secret
        - emptyDir
        - hostPath
      allowedHostPaths:
        - pathPrefix: "/etc/cni/net.d"
        - pathPrefix: "/etc/kube-flannel"
        - pathPrefix: "/run/flannel"
      readOnlyRootFilesystem: false
      # Users and groups
      runAsUser:
        rule: RunAsAny
      supplementalGroups:
        rule: RunAsAny
      fsGroup:
        rule: RunAsAny
      # Privilege Escalation
      allowPrivilegeEscalation: false
      defaultAllowPrivilegeEscalation: false
      # Capabilities
      allowedCapabilities: ['NET_ADMIN']
      defaultAddCapabilities: []
      requiredDropCapabilities: []
      # Host namespaces
      hostPID: false
      hostIPC: false
      hostNetwork: true
      hostPorts:
      - min: 0
        max: 65535
      # SELinux
      seLinux:
        # SELinux is unused in CaaSP
        rule: 'RunAsAny'
    ---
    kind: ClusterRole
    apiVersion: rbac.authorization.k8s.io/v1beta1
    metadata:
      name: flannel
    rules:
      - apiGroups: ['extensions']
        resources: ['podsecuritypolicies']
        verbs: ['use']
        resourceNames: ['psp.flannel.unprivileged']
      - apiGroups:
          - ""
        resources:
          - pods
        verbs:
          - get
      - apiGroups:
          - ""
        resources:
          - nodes
        verbs:
          - list
          - watch
      - apiGroups:
          - ""
        resources:
          - nodes/status
        verbs:
          - patch
    ---
    kind: ClusterRoleBinding
    apiVersion: rbac.authorization.k8s.io/v1beta1
    metadata:
      name: flannel
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: flannel
    subjects:
    - kind: ServiceAccount
      name: flannel
      namespace: kube-system
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: flannel
      namespace: kube-system
    ---
    kind: ConfigMap
    apiVersion: v1
    metadata:
      name: kube-flannel-cfg
      namespace: kube-system
      labels:
        tier: node
        app: flannel
    data:
      cni-conf.json: |
        {
          "name": "cbr0",
          "cniVersion": "0.3.1",
          "plugins": [
            {
              "type": "flannel",
              "delegate": {
                "hairpinMode": true,
                "isDefaultGateway": true
              }
            },
            {
              "type": "portmap",
              "capabilities": {
                "portMappings": true
              }
            }
          ]
        }
      net-conf.json: |
        {
          "Network": "10.244.0.0/16",
          "Backend": {
            "Type": "vxlan"
          }
        }
    ---
    apiVersion: apps/v1
    kind: DaemonSet
    metadata:
      name: kube-flannel-ds-amd64
      namespace: kube-system
      labels:
        tier: node
        app: flannel
    spec:
      selector:
        matchLabels:
          app: flannel
      template:
        metadata:
          labels:
            tier: node
            app: flannel
        spec:
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                  - matchExpressions:
                      - key: kubernetes.io/os
                        operator: In
                        values:
                          - linux
                      - key: kubernetes.io/arch
                        operator: In
                        values:
                          - amd64
          hostNetwork: true
          tolerations:
          - operator: Exists
            effect: NoSchedule
          serviceAccountName: flannel
          initContainers:
          - name: install-cni
            image: quay.io/coreos/flannel:v0.12.0-amd64
            command:
            - cp
            args:
            - -f
            - /etc/kube-flannel/cni-conf.json
            - /etc/cni/net.d/10-flannel.conflist
            volumeMounts:
            - name: cni
              mountPath: /etc/cni/net.d
            - name: flannel-cfg
              mountPath: /etc/kube-flannel/
          containers:
          - name: kube-flannel
            image: quay.io/coreos/flannel:v0.12.0-amd64
            command:
            - /opt/bin/flanneld
            args:
            - --ip-masq
            - --kube-subnet-mgr
            resources:
              requests:
                cpu: "100m"
                memory: "50Mi"
              limits:
                cpu: "100m"
                memory: "50Mi"
            securityContext:
              privileged: false
              capabilities:
                add: ["NET_ADMIN"]
            env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            volumeMounts:
            - name: run
              mountPath: /run/flannel
            - name: flannel-cfg
              mountPath: /etc/kube-flannel/
          volumes:
            - name: run
              hostPath:
                path: /run/flannel
            - name: cni
              hostPath:
                path: /etc/cni/net.d
            - name: flannel-cfg
              configMap:
                name: kube-flannel-cfg
    ---
    apiVersion: apps/v1
    kind: DaemonSet
    metadata:
      name: kube-flannel-ds-arm64
      namespace: kube-system
      labels:
        tier: node
        app: flannel
    spec:
      selector:
        matchLabels:
          app: flannel
      template:
        metadata:
          labels:
            tier: node
            app: flannel
        spec:
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                  - matchExpressions:
                      - key: kubernetes.io/os
                        operator: In
                        values:
                          - linux
                      - key: kubernetes.io/arch
                        operator: In
                        values:
                          - arm64
          hostNetwork: true
          tolerations:
          - operator: Exists
            effect: NoSchedule
          serviceAccountName: flannel
          initContainers:
          - name: install-cni
            image: quay.io/coreos/flannel:v0.12.0-arm64
            command:
            - cp
            args:
            - -f
            - /etc/kube-flannel/cni-conf.json
            - /etc/cni/net.d/10-flannel.conflist
            volumeMounts:
            - name: cni
              mountPath: /etc/cni/net.d
            - name: flannel-cfg
              mountPath: /etc/kube-flannel/
          containers:
          - name: kube-flannel
            image: quay.io/coreos/flannel:v0.12.0-arm64
            command:
            - /opt/bin/flanneld
            args:
            - --ip-masq
            - --kube-subnet-mgr
            resources:
              requests:
                cpu: "100m"
                memory: "50Mi"
              limits:
                cpu: "100m"
                memory: "50Mi"
            securityContext:
              privileged: false
              capabilities:
                 add: ["NET_ADMIN"]
            env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            volumeMounts:
            - name: run
              mountPath: /run/flannel
            - name: flannel-cfg
              mountPath: /etc/kube-flannel/
          volumes:
            - name: run
              hostPath:
                path: /run/flannel
            - name: cni
              hostPath:
                path: /etc/cni/net.d
            - name: flannel-cfg
              configMap:
                name: kube-flannel-cfg
    ---
    apiVersion: apps/v1
    kind: DaemonSet
    metadata:
      name: kube-flannel-ds-arm
      namespace: kube-system
      labels:
        tier: node
        app: flannel
    spec:
      selector:
        matchLabels:
          app: flannel
      template:
        metadata:
          labels:
            tier: node
            app: flannel
        spec:
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                  - matchExpressions:
                      - key: kubernetes.io/os
                        operator: In
                        values:
                          - linux
                      - key: kubernetes.io/arch
                        operator: In
                        values:
                          - arm
          hostNetwork: true
          tolerations:
          - operator: Exists
            effect: NoSchedule
          serviceAccountName: flannel
          initContainers:
          - name: install-cni
            image: quay.io/coreos/flannel:v0.12.0-arm
            command:
            - cp
            args:
            - -f
            - /etc/kube-flannel/cni-conf.json
            - /etc/cni/net.d/10-flannel.conflist
            volumeMounts:
            - name: cni
              mountPath: /etc/cni/net.d
            - name: flannel-cfg
              mountPath: /etc/kube-flannel/
          containers:
          - name: kube-flannel
            image: quay.io/coreos/flannel:v0.12.0-arm
            command:
            - /opt/bin/flanneld
            args:
            - --ip-masq
            - --kube-subnet-mgr
            resources:
              requests:
                cpu: "100m"
                memory: "50Mi"
              limits:
                cpu: "100m"
                memory: "50Mi"
            securityContext:
              privileged: false
              capabilities:
                 add: ["NET_ADMIN"]
            env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            volumeMounts:
            - name: run
              mountPath: /run/flannel
            - name: flannel-cfg
              mountPath: /etc/kube-flannel/
          volumes:
            - name: run
              hostPath:
                path: /run/flannel
            - name: cni
              hostPath:
                path: /etc/cni/net.d
            - name: flannel-cfg
              configMap:
                name: kube-flannel-cfg
    ---
    apiVersion: apps/v1
    kind: DaemonSet
    metadata:
      name: kube-flannel-ds-ppc64le
      namespace: kube-system
      labels:
        tier: node
        app: flannel
    spec:
      selector:
        matchLabels:
          app: flannel
      template:
        metadata:
          labels:
            tier: node
            app: flannel
        spec:
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                  - matchExpressions:
                      - key: kubernetes.io/os
                        operator: In
                        values:
                          - linux
                      - key: kubernetes.io/arch
                        operator: In
                        values:
                          - ppc64le
          hostNetwork: true
          tolerations:
          - operator: Exists
            effect: NoSchedule
          serviceAccountName: flannel
          initContainers:
          - name: install-cni
            image: quay.io/coreos/flannel:v0.12.0-ppc64le
            command:
            - cp
            args:
            - -f
            - /etc/kube-flannel/cni-conf.json
            - /etc/cni/net.d/10-flannel.conflist
            volumeMounts:
            - name: cni
              mountPath: /etc/cni/net.d
            - name: flannel-cfg
              mountPath: /etc/kube-flannel/
          containers:
          - name: kube-flannel
            image: quay.io/coreos/flannel:v0.12.0-ppc64le
            command:
            - /opt/bin/flanneld
            args:
            - --ip-masq
            - --kube-subnet-mgr
            resources:
              requests:
                cpu: "100m"
                memory: "50Mi"
              limits:
                cpu: "100m"
                memory: "50Mi"
            securityContext:
              privileged: false
              capabilities:
                 add: ["NET_ADMIN"]
            env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            volumeMounts:
            - name: run
              mountPath: /run/flannel
            - name: flannel-cfg
              mountPath: /etc/kube-flannel/
          volumes:
            - name: run
              hostPath:
                path: /run/flannel
            - name: cni
              hostPath:
                path: /etc/cni/net.d
            - name: flannel-cfg
              configMap:
                name: kube-flannel-cfg
    ---
    apiVersion: apps/v1
    kind: DaemonSet
    metadata:
      name: kube-flannel-ds-s390x
      namespace: kube-system
      labels:
        tier: node
        app: flannel
    spec:
      selector:
        matchLabels:
          app: flannel
      template:
        metadata:
          labels:
            tier: node
            app: flannel
        spec:
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                  - matchExpressions:
                      - key: kubernetes.io/os
                        operator: In
                        values:
                          - linux
                      - key: kubernetes.io/arch
                        operator: In
                        values:
                          - s390x
          hostNetwork: true
          tolerations:
          - operator: Exists
            effect: NoSchedule
          serviceAccountName: flannel
          initContainers:
          - name: install-cni
            image: quay.io/coreos/flannel:v0.12.0-s390x
            command:
            - cp
            args:
            - -f
            - /etc/kube-flannel/cni-conf.json
            - /etc/cni/net.d/10-flannel.conflist
            volumeMounts:
            - name: cni
              mountPath: /etc/cni/net.d
            - name: flannel-cfg
              mountPath: /etc/kube-flannel/
          containers:
          - name: kube-flannel
            image: quay.io/coreos/flannel:v0.12.0-s390x
            command:
            - /opt/bin/flanneld
            args:
            - --ip-masq
            - --kube-subnet-mgr
            resources:
              requests:
                cpu: "100m"
                memory: "50Mi"
              limits:
                cpu: "100m"
                memory: "50Mi"
            securityContext:
              privileged: false
              capabilities:
                 add: ["NET_ADMIN"]
            env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            volumeMounts:
            - name: run
              mountPath: /run/flannel
            - name: flannel-cfg
              mountPath: /etc/kube-flannel/
          volumes:
            - name: run
              hostPath:
                path: /run/flannel
            - name: cni
              hostPath:
                path: /etc/cni/net.d
            - name: flannel-cfg
              configMap:
                name: kube-flannel-cfg

执行命令安装

kubectl apply -f kube-flannel.yml

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 16][]

终于升级到了最新版！！！

建议裸机安装，不要像我一样用装过docker和kubernetes旧版本的机器，这老外天天改动，有出入，就像spring新版和老版本并不完全兼容，追人家的永远都受制于人！！！！！！

小结：不建议此种方式安装升级，还是以前源码式安装比较牢靠。

参考:

## 0.  升级Kubernetes 1.18前，你不得不知的9件事 ##

[http://dockone.io/article/9952][http_dockone.io_article_9952]

## 1.  Kubernetes 1.18: Fit & Finish ##

[https://kubernetes.io/blog/2020/03/25/kubernetes-1-18-release-announcement/][https_kubernetes.io_blog_2020_03_25_kubernetes-1-18-release-announcement]

2.   Kubernetes 版本及版本倾斜支持策略  [https://kubernetes.io/zh/docs/setup/release/][https_kubernetes.io_zh_docs_setup_release]

3.  [使用kubeadm在Centos8上部署kubernetes1.18][kubeadm_Centos8_kubernetes1.18]  [https://www.kubernetes.org.cn/7189.html][kubeadm_Centos8_kubernetes1.18]

4.  kubernetes/yum/  [https://mirrors.aliyun.com/kubernetes/yum/][https_mirrors.aliyun.com_kubernetes_yum]

5.   kubeadm安装kubenetes1.15.4集群   [http://www.shangdixinxi.com/detail-1301548.html][http_www.shangdixinxi.com_detail-1301548.html]

6    Linux下minikube启动失败(It seems like the kubelet isn't running or healthy)

[http://www.luyixian.cn/news\_show\_73520.aspx][http_www.luyixian.cn_news_show_73520.aspx]

7. Kubernetes使用中发现的问题和错误  [https://yq.aliyun.com/articles/679198][https_yq.aliyun.com_articles_679198]

8. 备忘录  [https://blog-static.cnblogs.com/files/dongguangming/%E5%9C%A8centos7%E4%B8%8A%E5%AE%89%E8%A3%85%E5%92%8C%E9%85%8D%E7%BD%AEKubernetes,docker%E9%9B%86%E7%BE%A4%E7%AE%A1%E7%90%86.js][https_blog-static.cnblogs.com_files_dongguangming_E5_9C_A8centos7_E4_B8_8A_E5_AE_89_E8_A3_85_E5_92_8C_E9_85_8D_E7_BD_AEKubernetes_docker_E9_9B_86_E7_BE_A4_E7_AE_A1_E7_90_86.js]

9.   how to install kubernetes on centos7 or ubuntu18-04 （要 fan qiang） [https://www.itzgeek.com/how-tos/linux/centos-how-tos/how-to-install-kubernetes-on-centos-7-ubuntu-18-04-16-04-debian-9.html][https_www.itzgeek.com_how-tos_linux_centos-how-tos_how-to-install-kubernetes-on-centos-7-ubuntu-18-04-16-04-debian-9.html]

10.   setting up kubernetes cluster  single-cp ormultiple-cp（也要fan qiang）  [  https://medium.com/the-tech-inducers/setting-up-kubernetes-cluster-single-cp-or-multiple-cp-a7e8d95b3c0c][https_medium.com_the-tech-inducers_setting-up-kubernetes-cluster-single-cp-or-multiple-cp-a7e8d95b3c0c]

11. ubuntu18.04 kubeadm 安装kubernetes v1.18.3 [https://www.taodudu.cc/news/show-58109.html][https_www.taodudu.cc_news_show-58109.html]

[https_blog.csdn.net_dong19891210_article_details_106458776]: https://blog.csdn.net/dong19891210/article/details/106458776
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70]: /images/20221123/a6baae7a7336412fb105170fb42f4512.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 1]: /images/20221123/39e1fea8a6a54dd189dac254403cdfbc.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 2]: /images/20221123/5156698d878b410b8919a91d28b7497a.png
[20200602020556823.png]: /images/20221123/b8080e6c7adb4ddd980c6511aa35679a.png
[20200602020737811.png]: /images/20221123/f0744431cd4d481099bfdd1dd8d3fa86.png
[20200602020933464.png]: /images/20221123/a64dc5c3ce254b0ea4ae707d1bf4a291.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 3]: /images/20221123/84d043ba15e7491f8007a05200389c4a.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 4]: /images/20221123/fdc05813815240ed90ec7d63d392f227.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 5]: /images/20221123/da243c8958914de19d7a221bd27f837e.png
[20200602022044613.png]: /images/20221123/95eae41afb104b38be5073c3aa5caf4e.png
[20200602022517140.png]: /images/20221123/233cf1495e3b428dac36e52f75cbd573.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 6]: /images/20221123/69d091ec5fcf4f15a613d09b1523c559.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 7]: /images/20221123/aa2dfd2adcb8494fbbead138c9436701.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 8]: /images/20221123/2903940cf5c14138a9c5d59263a0e254.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 9]: /images/20221123/2702b9626ff442089f6f6c46bec6af6a.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 10]: /images/20221123/1d13cd08d1af4eef957aca3a4c2f7fbf.png
[20200602030414218.png]: /images/20221123/399f6e82dd2b4abdacbae30d305a0a6d.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 11]: /images/20221123/257d0e3d0ac04958ac6811e5604e1c52.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 12]: /images/20221123/8ab42b300f214fe4a94878a64edf74e8.png
[20200602031959644.png]: /images/20221123/abfdcc4f68d04a4596f36bf6f4427abc.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 13]: /images/20221123/9ace3d9d7e8c46fb843a6145afc3969f.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 14]: /images/20221123/089188251e7340c784e461465c8cce40.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 15]: /images/20221123/7290159556e748ef8e097cdc38febd0b.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RvbmcxOTg5MTIxMA_size_16_color_FFFFFF_t_70 16]: /images/20221123/0169a2b1f7a64ee68889cd4e36cda289.png
[http_dockone.io_article_9952]: http://dockone.io/article/9952
[https_kubernetes.io_blog_2020_03_25_kubernetes-1-18-release-announcement]: https://kubernetes.io/blog/2020/03/25/kubernetes-1-18-release-announcement/
[https_kubernetes.io_zh_docs_setup_release]: https://kubernetes.io/zh/docs/setup/release/
[kubeadm_Centos8_kubernetes1.18]: https://www.kubernetes.org.cn/7189.html
[https_mirrors.aliyun.com_kubernetes_yum]: https://mirrors.aliyun.com/kubernetes/yum/
[http_www.shangdixinxi.com_detail-1301548.html]: http://www.shangdixinxi.com/detail-1301548.html
[http_www.luyixian.cn_news_show_73520.aspx]: http://www.luyixian.cn/news_show_73520.aspx
[https_yq.aliyun.com_articles_679198]: https://yq.aliyun.com/articles/679198
[https_blog-static.cnblogs.com_files_dongguangming_E5_9C_A8centos7_E4_B8_8A_E5_AE_89_E8_A3_85_E5_92_8C_E9_85_8D_E7_BD_AEKubernetes_docker_E9_9B_86_E7_BE_A4_E7_AE_A1_E7_90_86.js]: https://blog-static.cnblogs.com/files/dongguangming/%E5%9C%A8centos7%E4%B8%8A%E5%AE%89%E8%A3%85%E5%92%8C%E9%85%8D%E7%BD%AEKubernetes,docker%E9%9B%86%E7%BE%A4%E7%AE%A1%E7%90%86.js
[https_www.itzgeek.com_how-tos_linux_centos-how-tos_how-to-install-kubernetes-on-centos-7-ubuntu-18-04-16-04-debian-9.html]: https://www.itzgeek.com/how-tos/linux/centos-how-tos/how-to-install-kubernetes-on-centos-7-ubuntu-18-04-16-04-debian-9.html
[https_medium.com_the-tech-inducers_setting-up-kubernetes-cluster-single-cp-or-multiple-cp-a7e8d95b3c0c]: https://medium.com/the-tech-inducers/setting-up-kubernetes-cluster-single-cp-or-multiple-cp-a7e8d95b3c0c
[https_www.taodudu.cc_news_show-58109.html]: https://www.taodudu.cc/news/show-58109.html