k8s笔记5-多master集群配置

爱被打了一巴掌 2022-03-21 04:46 472阅读 0赞

生产环境中多应用多master集群架构  
准备以下配置：

<table> 
 <thead> 
  <tr> 
   <th>配置</th> 
   <th>IP</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>master01</td> 
   <td>10.0.3.171</td> 
  </tr> 
  <tr> 
   <td>master02</td> 
   <td>10.0.3.172</td> 
  </tr> 
  <tr> 
   <td>node01</td> 
   <td>10.0.3.104</td> 
  </tr> 
  <tr> 
   <td>node02</td> 
   <td>10.0.3.150</td> 
  </tr> 
  <tr> 
   <td>LB(master)</td> 
   <td>10.0.3.173</td> 
  </tr> 
  <tr> 
   <td>LB(backup)</td> 
   <td>10.0.3.165</td> 
  </tr> 
  <tr> 
   <td>虚拟地址(VIP)</td> 
   <td>10.0.3.160</td> 
  </tr> 
 </tbody> 
</table>

*LB使用nginx+keepalived组合方式。*  
在master01中，将相应的配置内容copy到master02中：

scp -r /opt/kubernetes/ root@10.0.3.172:/opt/
    scp /usr/lib/systemd/system/{ kube-apiserver,kube-scheduler,kube-controller-manager}.service root@10.0.3.172:/usr/lib/systemd/system

修改/opt/kubernetes/cfg/kube-apiserver的地址。  
直接重启

systemctl daemon-reload
    systemctl restart kube-apiserver
    systemctl restart kube-scheduler
    systemctl restart kube-controller-manager

安装nginx，见[官网][Link 1]，Installing nginx—>packages—>CentOS，按照步骤来即可。  
在负载均衡器LB-master中。

vi /etc/nginx/nginx.conf

events {
        worker_connections  1024;
    }
    #-------------------添加内容
    stream {
        log_format main "$remote_addr $upstream_addr $time_local $status";
        access_log /var/log/nginx/k8s-access.log main;
        upstream k8s-apiserver {
            server 10.0.3.171:6443;
            server 10.0.3.172:6443;
        }
        server {
            listen 10.0.3.173:6443;
            proxy_pass k8s-apiserver;
        }
    }
    #-----------------------
    http {

systemctl restart nginx

![在这里插入图片描述][20190124165222528.png]

netstat -antp |grep 8080

![在这里插入图片描述][20190124165310289.png]

tail /var/log/nginx/k8s-access.log -f

重新启动k8s的进程，会在该log中看到分配信息。  
VRRP——虚拟路由冗余协议  
安装keepalived，并修改配置文件

yum install keepalived -y
    vi /etc/keepalived/keepalived.conf

cat /etc/keepalived/keepalived.conf
    ! Configuration File for keepalived
    
    global_defs {
       # 接收邮件地址
       notification_email {
         acassen@firewall.loc
         failover@firewall.loc
         sysadmin@firewall.loc
       }
       # 邮件发送地址
       notification_email_from Alexandre.Cassen@firewall.loc
       smtp_server 127.0.0.1
       smtp_connect_timeout 30
       router_id NGINX_MASTER
    }
    
    # 通过一个vrrp协议检查本机nginx是否正常，若不正常，将关闭keepalived
    # 实现vip的漂移
    vrrp_script check_nginx {
        script "/etc/keepalived/check_nginx.sh" #生成一个.sh文件，将地址写入
    }
    
    vrrp_instance VI_1 {
        state MASTER
        # interface ens32  # 网卡接口
        interface eth0  # 网卡接口
        # VRRP 路由 ID实例，每个实例是唯一的，两个节点设置一样，以指明各个节点同属一VRRP组
        virtual_router_id 51 
        priority 100    # 优先级，备服务器设置 90
        advert_int 1    # 指定VRRP 心跳包通告间隔时间即组播信息发送间隔，默认1秒，两个节点需一致
        # 设置验证信息，两个节点一致
        authentication {
            auth_type PASS
            auth_pass 1111
        }
        virtual_ipaddress {
            10.0.3.160/24    #虚拟地址
        }
        track_script {
            check_nginx
        }
    }

vrrp协议内容

# vi /etc/keepalived/check_nginx.sh
    
    count=$(ps -ef |grep nginx |egrep -cv "grep|$$")
    
    if [ "$count" -eq 0 ];then
        /etc/init.d/keepalived stop
    fi

通常在终端中输入

count=$(ps -ef |grep nginx |egrep -cv "grep|$$")

正常情况下不会为0。  
![在这里插入图片描述][20190130191243629.png]

通过ifconfig或ip a查看网卡接口，在vrrp\_instance中修改![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTA4MDE5OTQ_size_16_color_FFFFFF_t_70]  
重启keepalived

systemctl restart keepalived
    ps -ef |grep keepalived

启动成功  
![在这里插入图片描述][20190130192126564.png]  
ip a查看是否绑定了虚拟ip

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTA4MDE5OTQ_size_16_color_FFFFFF_t_70 1]  
keepalived备机的keepalived.conf的配置，不同之处如下：

state BACKUP
    priority 90
    #其它配置跟keepalived主机相同

##### 杀死keepalived进程后，可以实现vip的偏移，但是原机器的vip无法自动删除 #####

**解决方案**：关闭keepalived时用命令  
service keepalived stop 或 kill -15 pid（注：只删除第一个进程号）  
**存在问题**：  
非正常关闭keepalived。 禁止使用kill -9 或killall杀死keepalived。

[Link 1]: https://nginx.org/en/docs/
[20190124165222528.png]: /images/20220321/d14c5026e2cf45339198feca8341fbae.png
[20190124165310289.png]: /images/20220321/2c36f76436c542df9c59bce53d5bd801.png
[20190130191243629.png]: /images/20220321/7245eb800e4346bfa5bf8fb2b694cade.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTA4MDE5OTQ_size_16_color_FFFFFF_t_70]: /images/20220321/efc29201533b4a6ca0061665f7165573.png
[20190130192126564.png]: /images/20220321/8d5232f5dafd475285d63ddb051ded87.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTA4MDE5OTQ_size_16_color_FFFFFF_t_70 1]: /images/20220321/73a8235e331941a092ff7888325e00c9.png