Kubenetes 入门安装1m1s安装

旧城等待， 2023-05-22 08:49 1阅读 0赞

### 文章目录 ###

*   *  准备环境
     *  安装master & slave
     *   *  配置环境
         *   *  添加镜像
             *  关闭交换分区
             *  关闭selinux
         *  1. 安装软件
         *  2. 启动
         *  3. 启动Master
         *  4. 按照日志的提示操作
         *  5. 安装网络插件
         *  6. 安装node 假如集群(192.168.121.102)
         *  7. 检查集群
     *  运维监控平台
     *   *  1. 安装
         *  2. 创建用户
         *  3. 创建 token
         *  4. 网络代理
         *  5. 配置一下nodePort
         *  6. 访问
     *  问题
     *   *  解决kubeadm init /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1
         *  failed to check server version: Get https://192.168.117.132:6443/version: x509: certificate has expired or is not yet valid
         *  tip

## 准备环境 ##

[https://segmentfault.com/a/1190000008743806][https_segmentfault.com_a_1190000008743806]  
[https://www.cnblogs.com/eastonliu/p/11637929.html][https_www.cnblogs.com_eastonliu_p_11637929.html]

<table> 
 <thead> 
  <tr> 
   <th>机器</th> 
   <th></th> 
   <th></th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>192.168.121.101</td> 
   <td>master</td> 
   <td></td> 
  </tr> 
  <tr> 
   <td>192.168.121.102</td> 
   <td>slave</td> 
   <td></td> 
  </tr> 
  <tr> 
   <td></td> 
   <td></td> 
   <td></td> 
  </tr> 
 </tbody> 
</table>

## 安装master & slave ##

### 配置环境 ###

#### 添加镜像 ####

vi /etc/yum.repos.d/kubernetes.repo
       
       [kubernetes]
       name=Kubernetes
       baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
       enabled=1
       gpgcheck=1
       repo_gpgcheck=1
       gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg

#### 关闭交换分区 ####

echo 'vm.swappiness=10' >>/etc/sysctl.conf

#### 关闭selinux ####

# vi /etc/selinux/config 
    
    # This file controls the state of SELinux on the system.
    # SELINUX= can take one of these three values:
    #     enforcing - SELinux security policy is enforced.
    #     permissive - SELinux prints warnings instead of enforcing.
    #     disabled - No SELinux policy is loaded.
    SELINUX=disabled
    # SELINUXTYPE= can take one of three two values:
    #     targeted - Targeted processes are protected,
    #     minimum - Modification of targeted policy. Only selected processes are protected. 
    #     mls - Multi Level Security protection.
    SELINUXTYPE=targeted

默认 docker 已经装好了

### 1. 安装软件 ###

sudo yum install -y --nogpgcheck kubelet kubeadm kubectl kubernetes-cni

### 2. 启动 ###

sudo systemctl enable docker && systemctl start docker
    sudo systemctl enable kubelet

### 3. 启动Master ###

## 安装镜像
    docker pull mirrorgooglecontainers/kube-apiserver-amd64:v1.16.0
    docker pull mirrorgooglecontainers/kube-controller-manager-amd64:v1.16.0
    docker pull mirrorgooglecontainers/etcd-amd64:3.3.15-0
    docker pull mirrorgooglecontainers/kube-scheduler-amd64:v1.16.0
    docker pull mirrorgooglecontainers/kube-proxy-amd64:v1.16.0
    docker pull mirrorgooglecontainers/pause:3.1
    docker pull coredns/coredns:1.6.2
    
    docker tag mirrorgooglecontainers/kube-apiserver-amd64:v1.16.0 k8s.gcr.io/kube-apiserver:v1.16.0
    docker tag mirrorgooglecontainers/kube-controller-manager-amd64:v1.16.0 k8s.gcr.io/kube-controller-manager:v1.16.0
    docker tag mirrorgooglecontainers/kube-scheduler-amd64:v1.16.0 k8s.gcr.io/kube-scheduler:v1.16.0
    docker tag mirrorgooglecontainers/kube-proxy-amd64:v1.16.0 k8s.gcr.io/kube-proxy:v1.16.0
    docker tag mirrorgooglecontainers/etcd-amd64:3.3.15-0 k8s.gcr.io/etcd:3.3.15-0
    docker tag mirrorgooglecontainers/pause:3.1 k8s.gcr.io/pause:3.1
    docker tag coredns/coredns:1.6.2 k8s.gcr.io/coredns:1.6.2
    
    
    # kubeadm init --kubernetes-version=1.16.0
    
    如果没有成功 可以使用 kubeadm reset 命令重置

[root@flinkhadoop opt]# kubeadm init --kubernetes-version=1.16.0
    [init] Using Kubernetes version: v1.16.0
    [preflight] Running pre-flight checks
    [preflight] Pulling images required for setting up a Kubernetes cluster
    [preflight] This might take a minute or two, depending on the speed of your internet connection
    [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
    [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
    [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
    [kubelet-start] Activating the kubelet service
    [certs] Using certificateDir folder "/etc/kubernetes/pki"
    [certs] Generating "ca" certificate and key
    [certs] Generating "apiserver" certificate and key
    [certs] apiserver serving cert is signed for DNS names [flinkhadoop kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.121.101]
    [certs] Generating "apiserver-kubelet-client" certificate and key
    [certs] Generating "front-proxy-ca" certificate and key
    [certs] Generating "front-proxy-client" certificate and key
    [certs] Generating "etcd/ca" certificate and key
    [certs] Generating "etcd/server" certificate and key
    [certs] etcd/server serving cert is signed for DNS names [flinkhadoop localhost] and IPs [192.168.121.101 127.0.0.1 ::1]
    [certs] Generating "etcd/peer" certificate and key
    [certs] etcd/peer serving cert is signed for DNS names [flinkhadoop localhost] and IPs [192.168.121.101 127.0.0.1 ::1]
    [certs] Generating "etcd/healthcheck-client" certificate and key
    [certs] Generating "apiserver-etcd-client" certificate and key
    [certs] Generating "sa" key and public key
    [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
    [kubeconfig] Writing "admin.conf" kubeconfig file
    [kubeconfig] Writing "kubelet.conf" kubeconfig file
    [kubeconfig] Writing "controller-manager.conf" kubeconfig file
    [kubeconfig] Writing "scheduler.conf" kubeconfig file
    [control-plane] Using manifest folder "/etc/kubernetes/manifests"
    [control-plane] Creating static Pod manifest for "kube-apiserver"
    [control-plane] Creating static Pod manifest for "kube-controller-manager"
    [control-plane] Creating static Pod manifest for "kube-scheduler"
    [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
    [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
    [apiclient] All control plane components are healthy after 17.002116 seconds
    [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
    [kubelet] Creating a ConfigMap "kubelet-config-1.16" in namespace kube-system with the configuration for the kubelets in the cluster
    [upload-certs] Skipping phase. Please see --upload-certs
    [mark-control-plane] Marking the node flinkhadoop as control-plane by adding the label "node-role.kubernetes.io/master=''"
    [mark-control-plane] Marking the node flinkhadoop as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
    [bootstrap-token] Using token: abehtg.twc1ljq5c8cdk5s4
    [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
    [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
    [bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
    [bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
    [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
    [addons] Applied essential addon: CoreDNS
    [addons] Applied essential addon: kube-proxy
    
    Your Kubernetes control-plane has initialized successfully!
    
    To start using your cluster, you need to run the following as a regular user:
    
      mkdir -p $HOME/.kube
      sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
      sudo chown $(id -u):$(id -g) $HOME/.kube/config
    
    You should now deploy a pod network to the cluster.
    Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
      https://kubernetes.io/docs/concepts/cluster-administration/addons/
    
    Then you can join any number of worker nodes by running the following on each as root:
    
    kubeadm join 192.168.121.101:6443 --token abehtg.twc1ljq5c8cdk5s4 \
        --discovery-token-ca-cert-hash sha256:99304ef8f9c585ce3fc0c403f60f581716bc128e1248830a24b3dec950f60e77

### 4. 按照日志的提示操作 ###

mkdir -p $HOME/.kube
      sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
      sudo chown $(id -u):$(id -g) $HOME/.kube/config

### 5. 安装网络插件 ###

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

### 6. 安装node 假如集群(192.168.121.102) ###

kubeadm join 192.168.121.101:6443 --token 8b0m1x.6ll8pj0hil33ddl8 \
     --discovery-token-ca-cert-hash sha256:4c17f2e1d0e1d635cfdde04def7c1bb9be3673150d5f0a89b861e6cb53bd9bb3

[root@kubeslave1 ~]# kubeadm join --token hwvkc4.h5r41h4c1db04vnl --discovery-token-ca-cert-hash sha256:4c17f2e1d0e1d635cfdde04def7c1bb9be3673150d5f0a89b861e6cb53bd9bb3 --v=1 192.168.121.101:6443
    I1019 17:31:49.136191   28257 join.go:363] [preflight] found NodeName empty; using OS hostname as NodeName
    I1019 17:31:49.136250   28257 initconfiguration.go:102] detected and using CRI socket: /var/run/dockershim.sock
    [preflight] Running pre-flight checks
    I1019 17:31:49.136338   28257 preflight.go:90] [preflight] Running general checks
    I1019 17:31:49.136388   28257 checks.go:250] validating the existence and emptiness of directory /etc/kubernetes/manifests
    I1019 17:31:49.136411   28257 checks.go:287] validating the existence of file /etc/kubernetes/kubelet.conf
    I1019 17:31:49.136416   28257 checks.go:287] validating the existence of file /etc/kubernetes/bootstrap-kubelet.conf
    I1019 17:31:49.136425   28257 checks.go:103] validating the container runtime
    I1019 17:31:49.161004   28257 checks.go:129] validating if the service is enabled and active
    I1019 17:31:49.200273   28257 checks.go:336] validating the contents of file /proc/sys/net/bridge/bridge-nf-call-iptables
    I1019 17:31:49.200306   28257 checks.go:336] validating the contents of file /proc/sys/net/ipv4/ip_forward
    I1019 17:31:49.200326   28257 checks.go:650] validating whether swap is enabled or not
    I1019 17:31:49.200341   28257 checks.go:377] validating the presence of executable ip
    I1019 17:31:49.200361   28257 checks.go:377] validating the presence of executable iptables
    I1019 17:31:49.200372   28257 checks.go:377] validating the presence of executable mount
    I1019 17:31:49.200383   28257 checks.go:377] validating the presence of executable nsenter
    I1019 17:31:49.200394   28257 checks.go:377] validating the presence of executable ebtables
    I1019 17:31:49.200405   28257 checks.go:377] validating the presence of executable ethtool
    I1019 17:31:49.200415   28257 checks.go:377] validating the presence of executable socat
    I1019 17:31:49.200425   28257 checks.go:377] validating the presence of executable tc
    I1019 17:31:49.200434   28257 checks.go:377] validating the presence of executable touch
    I1019 17:31:49.200447   28257 checks.go:521] running all checks
    I1019 17:31:49.234568   28257 checks.go:407] checking whether the given node name is reachable using net.LookupHost
    I1019 17:31:49.234754   28257 checks.go:619] validating kubelet version
    I1019 17:31:49.300011   28257 checks.go:129] validating if the service is enabled and active
    I1019 17:31:49.308346   28257 checks.go:202] validating availability of port 10250
    I1019 17:31:49.308465   28257 checks.go:287] validating the existence of file /etc/kubernetes/pki/ca.crt
    I1019 17:31:49.308474   28257 checks.go:433] validating if the connectivity type is via proxy or direct
    I1019 17:31:49.308500   28257 join.go:433] [preflight] Discovering cluster-info
    I1019 17:31:49.308780   28257 token.go:199] [discovery] Trying to connect to API Server "192.168.121.101:6443"
    I1019 17:31:49.309192   28257 token.go:74] [discovery] Created cluster-info discovery client, requesting info from "https://192.168.121.101:6443"
    I1019 17:31:49.324262   28257 token.go:140] [discovery] Requesting info from "https://192.168.121.101:6443" again to validate TLS against the pinned public key
    I1019 17:31:49.333399   28257 token.go:163] [discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "192.168.121.101:6443"
    I1019 17:31:49.333430   28257 token.go:205] [discovery] Successfully established connection with API Server "192.168.121.101:6443"
    I1019 17:31:49.333460   28257 discovery.go:51] [discovery] Using provided TLSBootstrapToken as authentication credentials for the join process
    I1019 17:31:49.333487   28257 join.go:447] [preflight] Fetching init configuration
    I1019 17:31:49.333494   28257 join.go:485] [preflight] Retrieving KubeConfig objects
    [preflight] Reading configuration from the cluster...
    [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
    I1019 17:31:49.349070   28257 preflight.go:101] [preflight] Running configuration dependant checks
    I1019 17:31:49.349100   28257 controlplaneprepare.go:211] [download-certs] Skipping certs download
    I1019 17:31:49.349112   28257 kubelet.go:107] [kubelet-start] writing bootstrap kubelet config file at /etc/kubernetes/bootstrap-kubelet.conf
    I1019 17:31:49.349873   28257 kubelet.go:115] [kubelet-start] writing CA certificate at /etc/kubernetes/pki/ca.crt
    I1019 17:31:49.350369   28257 kubelet.go:133] [kubelet-start] Stopping the kubelet
    [kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.16" ConfigMap in the kube-system namespace
    [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
    [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
    I1019 17:31:49.402315   28257 kubelet.go:150] [kubelet-start] Starting the kubelet
    [kubelet-start] Activating the kubelet service
    [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
    I1019 17:31:49.953514   28257 kubelet.go:168] [kubelet-start] preserving the crisocket information for the node
    I1019 17:31:49.953535   28257 patchnode.go:30] [patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "kubeslave1" as an annotation
    
    This node has joined the cluster:
    * Certificate signing request was sent to apiserver and a response was received.
    * The Kubelet was informed of the new secure connection details.
    
    Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

> 如有问题，转到问题2

### 7. 检查集群 ###

[root@flinkhadoop /]# kubectl get pods --all-namespaces
    NAMESPACE     NAME                                READY   STATUS    RESTARTS   AGE
    kube-system   coredns-5644d7b6d9-2qmfx            1/1     Running   0          5m24s
    kube-system   coredns-5644d7b6d9-lfqh9            1/1     Running   0          5m24s
    kube-system   etcd-kubeslave                      1/1     Running   0          4m21s
    kube-system   kube-apiserver-kubeslave            1/1     Running   0          4m30s
    kube-system   kube-controller-manager-kubeslave   1/1     Running   0          4m26s
    kube-system   kube-proxy-rs4xr                    1/1     Running   0          5m24s
    kube-system   kube-scheduler-kubeslave            1/1     Running   0          4m34s
    kube-system   weave-net-mnvcv                     2/2     Running   0          2m13s

## 运维监控平台 ##

### 1. 安装 ###

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta4/aio/deploy/recommended.yaml

### 2. 创建用户 ###

We are creating Service Account with name `admin-user` in namespace `kubernetes-dashboard` first.

apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: admin
      namespace: kubernetes-dashboard
      
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      name: admin1
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: cluster-admin
    subjects:
    - kind: ServiceAccount
      name: admin1
      namespace: kubernetes-dashboard

### 3. 创建 token ###

kubectl -n kubernetes-dashboard describe secret $(kubectl -n kubernetes-dashboard get secret | grep admin | awk '{print $1}')

### 4. 网络代理 ###

kubectl proxy --address 192.168.121.101 --accept-hosts='.*'

### 5. 配置一下nodePort ###

因为是http代理的https，所以一直访问不了，需要配置一下  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMwNzYwNDQ_size_16_color_FFFFFF_t_70]

### 6. 访问 ###

https://192.168.121.101:30001/\#/service?namespace=default

![\[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-RwwiRsTn-1587047353317)(4083A3E3093F42848DB3946DECB2F33A)\]][img-RwwiRsTn-1587047353317_4083A3E3093F42848DB3946DECB2F33A]

## 问题 ##

### 解决kubeadm init /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1 ###

[https://yq.aliyun.com/articles/701252][https_yq.aliyun.com_articles_701252]

[root@flinkhadoop opt]# echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
    [root@flinkhadoop opt]# echo 1 > /proc/sys/net/bridge/bridge-nf-call-ip6tables

### failed to check server version: Get https://192.168.117.132:6443/version: x509: certificate has expired or is not yet valid ###

时间不同步导致的问题  
https://blog.csdn.net/zhongyuemengxiang/article/details/79121932  
安装一下ntpd服务

![\[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-7Tp52gZ4-1587047353318)(6462621060CD4C86B74FCE6CC4C42655)\]][img-7Tp52gZ4-1587047353318_6462621060CD4C86B74FCE6CC4C42655]

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMwNzYwNDQ_size_16_color_FFFFFF_t_70 1]

同步一下时间  
ntpdate -u 192.168.121.101

--------------------

### tip ###

1.  journalctl -xeu kubelet 查看日志
2.  配置日志级别如

kubeadm join phase preflight  --token 8b0m1x.6ll8pj0hil33ddl8     --discovery-token-ca-cert-hash sha256:4c17f2e1d0e1d635cfdde04def7c1bb9be3673150d5f0a89b861e6cb53bd9bb3 192.168.121.101:6443 --v=1

[https_segmentfault.com_a_1190000008743806]: https://segmentfault.com/a/1190000008743806
[https_www.cnblogs.com_eastonliu_p_11637929.html]: https://www.cnblogs.com/eastonliu/p/11637929.html
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMwNzYwNDQ_size_16_color_FFFFFF_t_70]: /images/20230521/33369bd0b8d2432e8649d649da920160.png
[img-RwwiRsTn-1587047353317_4083A3E3093F42848DB3946DECB2F33A]: /images/20230521/054492bf6a1c428eb4dfcf61ae418312.png
[https_yq.aliyun.com_articles_701252]: https://yq.aliyun.com/articles/701252
[img-7Tp52gZ4-1587047353318_6462621060CD4C86B74FCE6CC4C42655]: /images/20230521/ff2a81a30d9349fd9970ce6e20fbf581.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMwNzYwNDQ_size_16_color_FFFFFF_t_70 1]: /images/20230521/c756c149e2134fa8925ea15d4aaa88aa.png