k8s，盘他！k8s的安全机制与RBAC使用方法

小咪咪 2023-02-11 03:13 23阅读 0赞

### 文章目录 ###

*  前言
 *  一：k8s安全机制
 *   *   *  1.1：kubernetes安全框架
         *  1.2：第一关：Authentication认证
         *  1.3：第二关：Authorization授权
         *   *   *  1.3.1：RBAC使用测试
         *  1.4：第三关：准入控制Admission Control
         *  如有疑问可评论区交流！

# 前言 #

# 一：k8s安全机制 #

安全框架、

传输安全、认证，授权，准入控制

使用rbac授权

### 1.1：kubernetes安全框架 ###

![mark][]

*  安全框架的流程

> 流程：kubectl先请求api资源，然后是过三关，第一关是认证（Authentication），第二关是授权（Authorization），第三关是准入控制（Admission Control），只有通过这三关才可能会被k8s创建资源。
> 
> K8S安全控制框架主要由下面3个阶段进行控制，每一个阶段都支持插件方式，通过API Server配置来启用插件。
> 
> 普通用户若要安全访问集群API Server，往往需要证书、Token或者用户名+密码；Pod访问，需要ServiceAccount

*  apiserver使用的是token认证
    
        [root@master ~]# ps aux |grep apiserver
        root      12636  3.0 23.6 420460 235620 ?       Ssl  5月17 156:09 /opt/kubernetes/bin/kube-apiserver --logtostderr=true --v=4 --etcd-servers=https://192.168.233.131:2379,https://192.168.233.132:2379,https://192.168.233.133:2379 --bind-address=192.168.233.131 --secure-port=6443 --advertise-address=192.168.233.131 --allow-privileged=true --service-cluster-ip-range=10.0.0.0/24 --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction --authorization-mode=RBAC,Node --kubelet-https=true --enable-bootstrap-token-auth --token-auth-file=/opt/kubernetes/cfg/token.csv --service-node-port-range=30000-50000 --tls-cert-file=/opt/kubernetes/ssl/server.pem --tls-private-key-file=/opt/kubernetes/ssl/server-key.pem --client-ca-file=/opt/kubernetes/ssl/ca.pem --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem --etcd-cafile=/opt/etcd/ssl/ca.pem --etcd-certfile=/opt/etcd/ssl/server.pem --etcd-keyfile=/opt/etcd/ssl/server-key.pem
        root      22849  0.0  0.0 112728   976 pts/0    S+   10:20   0:00 grep --color=auto apiserver
        '//其中能够查询到token认证等信息'
 *  查看ServiceAccount，可以通过ServiceAccount在pod中去访问apiserver
    
        [root@master ~]# kubectl get sa
        NAME      SECRETS   AGE
        default   1         21d
        '//Service Account它并不是给kubernetes集群的用户使用的，而是给pod里面的进程使用的，它为pod提供必要的身份认证。'
 *  传输安全方面：8080用于内部通讯，6443是提供给外部访问的端口
    
        [root@master ~]# netstat -ntap |grep 8080 |grep LISTEN	'//默认8080监听本地（是通过master及其他组件连接使用）'
        tcp        0      0 127.0.0.1:8080          0.0.0.0:*               LISTEN      12636/kube-apiserve 
        [root@master ~]# netstat -ntap |grep 6443 |grep LISTEN	'//对外提供服务端口是6443'
        tcp        0      0 192.168.233.131:6443    0.0.0.0:*               LISTEN      12636/kube-apiserve

### 1.2：第一关：Authentication认证 ###

> 三种客户端身份认证：
> 
> 1、HTTPS 证书认证：基于CA证书签名的数字证书认证
> 
> 2、HTTP Token认证：通过一个Token来识别用户（生产环境中使用广泛）
> 
> 3、HTTP Base认证：用户名+密码的方式认证

*  1、HTTPS证书认证
    
        [root@master ~]# cat k8s/k8s-cert/k8s-cert.sh cat > ca-config.json <<EOF
        { 
          "signing": { 
            "default": { 
              "expiry": "87600h"
            }, "profiles": { 
              "kubernetes": { 
                 "expiry": "87600h",
                 "usages": [
                    "signing",
                    "key encipherment",
                    "server auth",
                    "client auth"
                ]
              }
            }
          }
        }
        EOF cat > ca-csr.json <<EOF
        { 
            "CN": "kubernetes", "key": { 
                "algo": "rsa",
                "size": 2048
            }, "names": [
                { 
                    "C": "CN",
                    "L": "Beijing",
                    "ST": "Beijing",
              	    "O": "k8s",
                    "OU": "System"
                }
            ]
        }
        EOF cfssl gencert -initca ca-csr.json | cfssljson -bare ca - #----------------------- cat > server-csr.json <<EOF
        { 
            "CN": "kubernetes", "hosts": [ "10.0.0.1", "127.0.0.1", "192.168.233.131", '//此处直接指定了负载均衡和master的节点' "192.168.233.130", "192.168.233.100", "192.168.233.128", "192.168.233.129", "kubernetes", "kubernetes.default", "kubernetes.default.svc", "kubernetes.default.svc.cluster", "kubernetes.default.svc.cluster.local" ], "key": { 
                "algo": "rsa",
                "size": 2048
            }, "names": [
                { 
                    "C": "CN",
                    "L": "BeiJing",
                    "ST": "BeiJing",
                    "O": "k8s",
                    "OU": "System"
                }
            ]
        }
        EOF cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson -bare server #----------------------- cat > admin-csr.json <<EOF
        { 
          "CN": "admin", "hosts": [], "key": { 
            "algo": "rsa",
            "size": 2048
          }, "names": [
            { 
              "C": "CN",
              "L": "BeiJing",
              "ST": "BeiJing",
              "O": "system:masters",
              "OU": "System"
            }
          ]
        }
        EOF cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin #----------------------- cat > kube-proxy-csr.json <<EOF
        { 
          "CN": "system:kube-proxy", "hosts": [], "key": { 
            "algo": "rsa",
            "size": 2048
          }, "names": [
            { 
              "C": "CN",
              "L": "BeiJing",
              "ST": "BeiJing",
              "O": "k8s",
              "OU": "System"
            }
          ]
        }
        EOF
        
        cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy
 *  2、httpd的token认证
    
        [root@master ~]# cat /opt/kubernetes/cfg/token.csv 
        7ea8f86b157225fd4b9273765e88a3ca,kubelet-bootstrap,10001,"system:kubelet-bootstrap"

### 1.3：第二关：Authorization授权 ###

*  RBAC（Role-Based Access Control，基于角色的访问控制）：负责完成授权（Authorization）工作，允许通过Kubernetes API动态配置策略。
 *  使用RBAC授权
 *  ![mark][mark 1]
    
    > 角色：
    > 
    > 1、Role：授权特定命名空间的访问权限
    > 
    > 2、ClusterRole：授权所有命名空间的访问权限
    > 
    > 角色绑定
    > 
    > 1、RoleBinding：将角色绑定到主体（即subject）
    > 
    > 2、ClusterRoleBinding：将集群角色绑定到主体
    > 
    > 主体（subject）
    > 
    > 1、User：用户
    > 
    > 2、Group：用户组
    > 
    > 3、ServiceAccount：服务账号

##### 1.3.1：RBAC使用测试 #####

*  1、创建名称空间dabao
    
        [root@master ~]# kubectl create ns dabao
        namespace/dabao created
        [root@master ~]# kubectl get ns
        NAME          STATUS   AGE
        dabao         Active   7s
        default       Active   21d
        kube-public   Active   21d
        kube-system   Active   21d
 *  2、创建测试pod
    
        [root@master ~]# kubectl run nginx01  --image=nginx -n dabao
        kubectl run --generator=deployment/apps.v1beta1 is DEPRECATED and will be removed in a future version. Use kubectl create instead.
        deployment.apps/nginx-test created
        [root@master ~]# kubectl get pod -n dabao
        NAME                         READY   STATUS    RESTARTS   AGE
        nginx01-77c44977fd-p4xbh     1/1     Running   0          15m
 *  3、扩容成3个副本，使用scale命令
    
        [root@master ~]# kubectl scale deploy/nginx01 --replicas=3 -n dabao
        deployment.extensions/nginx01 scaled
        [root@master ~]# kubectl get pod -n dabao
        NAME                         READY   STATUS    RESTARTS   AGE
        nginx01-77c44977fd-98jc4     1/1     Running   0          31s
        nginx01-77c44977fd-nt424     1/1     Running   0          31s
        nginx01-77c44977fd-p4xbh     1/1     Running   0          17m
 *  3、创建角色
    
        [root@master ~]# vim rbac-role.yaml
        apiVersion: rbac.authorization.k8s.io/v1
        kind: Role
        metadata:
          namespace: dabao
          name: pod-reader
        rules:
        - apiGroups: [""] # "" indicates the core API group
          resources: ["pods"]	'//创建角色只有pod资源的操作权限'
          verbs: ["get", "watch", "list"]	'//只有这些操作可以使用'
        [root@master ~]# kubectl apply -f rbac-role.yaml 
        role.rbac.authorization.k8s.io/pod-reader created
        [root@master ~]# kubectl get role -n dabao
        NAME         AGE
        pod-reader   8s
        [root@master ~]#
 *  4、角色绑定
    
        [root@master ~]# vim rbac-rolebinding.yaml
        apiVersion: rbac.authorization.k8s.io/v1
        kind: RoleBinding
        metadata:
          name: read-pods
          namespace: dabao
        subjects:
        - kind: User
          name: zhangsan
          apiGroup: rbac.authorization.k8s.io
        roleRef:
          kind: Role   
          name: pod-reader
          apiGroup: rbac.authorization.k8s.io
        [root@master ~]# kubectl apply -f rbac-rolebinding.yaml 
        rolebinding.rbac.authorization.k8s.io/read-pods created
        [root@master ~]# kubectl get role,rolebinding -n dabao
        NAME                                        AGE
        role.rbac.authorization.k8s.io/pod-reader   2m14s
        
        NAME                                              AGE
        rolebinding.rbac.authorization.k8s.io/read-pods   13s
 *  5、创建两个rbac的文件
    
        [root@master ~]# mkdir zhangsan [root@master ~]# cd zhangsan/ [root@master zhangsan]# vim rbac.yaml kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: namespace: default name: pod-reader rules: - apiGroups: [""] resources: ["pods"] verbs: ["get", "watch", "list"] --- kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: read-pods namespace: default subjects: - kind: User name: jane apiGroup: rbac.authorization.k8s.io roleRef: kind: Role name: pod-reader apiGroup: rbac.authorization.k8s.io apiVersion: v1 kind: ServiceAccount metadata: name: pod-reader namespace: default --- kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: sa-read-pods namespace: default subjects: - kind: ServiceAccount name: pod-reader roleRef: kind: Role name: pod-reader apiGroup: rbac.authorization.k8s.io [root@master zhangsan]# vim rbac-user.sh cat > zhangsan-csr.json <<EOF
        { 
          "CN": "zhangsan", "hosts": [], "key": { 
            "algo": "rsa",
            "size": 2048
          }, "names": [
            { 
              "C": "CN",
              "L": "BeiJing",
              "ST": "BeiJing"
            }
          ]
        }
        EOF
        
        cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes zhangsan-csr.json | cfssljson -bare zhangsan 
        
        kubectl config set-cluster kubernetes \
          --certificate-authority=ca.pem \
          --embed-certs=true \
          --server=https://192.168.233.100:6443 \	'//修改为负载均衡VIP地址'
          --kubeconfig=zhangsan-kubeconfig
          
        kubectl config set-credentials zhangsan \
          --client-key=zhangsan-key.pem \
          --client-certificate=zhangsan.pem \
          --embed-certs=true \
          --kubeconfig=zhangsan-kubeconfig
        
        kubectl config set-context default \
          --cluster=kubernetes \
          --user=zhangsan \
          --kubeconfig=zhangsan-kubeconfig
        
        kubectl config use-context default --kubeconfig=zhangsan-kubeconfig
 *  6、拷贝证书到张三目录并安装格式化工具
    
        [root@master zhangsan]# cp /root/k8s/k8s-cert/ca* ./
        [root@master zhangsan]# ls
        ca-config.json  ca.csr  ca-csr.json  ca-key.pem  ca.pem  rbac-user.sh  rbac.yaml
        [root@master zhangsan]# yum install dos2unix -y
        [root@master zhangsan]# dos2unix rbac-user.sh 	'//执行格式化'
        dos2unix: converting file rbac-user.sh to Unix format ...
        [root@master zhangsan]# bash rbac-user.sh 	'//运行脚本'
 *  7、使用zhangsan-kubeconfig访问资源测试
    
        [root@master zhangsan]# cat zhangsan-kubeconfig
        [root@master zhangsan]# kubectl --kubeconfig=zhangsan-kubeconfig get pod -n dabao
        NAME                       READY   STATUS    RESTARTS   AGE
        nginx01-77c44977fd-98jc4   1/1     Running   0          23m
        nginx01-77c44977fd-nt424   1/1     Running   0          23m
        nginx01-77c44977fd-p4xbh   1/1     Running   0          40m
        '//但是访问不到除了pod的其他资源'
        [root@master zhangsan]# kubectl --kubeconfig=zhangsan-kubeconfig get svc -n dabao	'//service无法访问'
        Error from server (Forbidden): services is forbidden: User "zhangsan" cannot list resource "services" in API group "" in the namespace "dabao"
        [root@master zhangsan]# kubectl --kubeconfig=zhangsan-kubeconfig get svc	'//默认名称空间也无法访问'
        Error from server (Forbidden): services is forbidden: User "zhangsan" cannot list resource "services" in API group "" in the namespace "default"
 *  UI访问控制
    
        [root@master zhangsan]# kubectl get svc -n kube-system
        NAME                   TYPE       CLUSTER-IP   EXTERNAL-IP   PORT(S)         AGE
        kubernetes-dashboard   NodePort   10.0.0.139   <none>        443:30005/TCP   13d
        [root@master zhangsan]# kubectl get pod -n kube-system -o wide
        NAME                                    READY   STATUS    RESTARTS   AGE   IP            NODE              NOMINATED NODE
        kubernetes-dashboard-7dffbccd68-58qms   1/1     Running   5          13d   172.17.78.3   192.168.233.132   <none>
        
        '//接下来访问之前搭建的UI界面'
    
    ![mark][mark 2]
 *  查看令牌
    
        [root@master zhangsan]# kubectl get secret -n kube-system	'//这是管理员的令牌'
        NAME                               TYPE                                  DATA   AGE
        dashboard-admin-token-zwktc        kubernetes.io/service-account-token   3      13d
        default-token-cnnbv                kubernetes.io/service-account-token   3      21d
        kubernetes-dashboard-certs         Opaque                                11     13d
        kubernetes-dashboard-key-holder    Opaque                                2      13d
        kubernetes-dashboard-token-qgppd   kubernetes.io/service-account-token   3      13d
        [root@master zhangsan]# vim sa.yaml	'//编辑认证yaml文件'
        apiVersion: v1
        kind: ServiceAccount
        metadata:
          name: pod-reader
          namespace: dabao
        
        ---
        
        kind: RoleBinding
        apiVersion: rbac.authorization.k8s.io/v1
        metadata:
          name: sa-read-pods
          namespace: dabao
        subjects:
        - kind: ServiceAccount
          name: pod-reader
        roleRef:
          kind: Role
          name: pod-reader
          apiGroup: rbac.authorization.k8s.io
        [root@master zhangsan]# kubectl apply -f sa.yaml 
        serviceaccount/pod-reader created
        rolebinding.rbac.authorization.k8s.io/sa-read-pods created
        [root@master zhangsan]# kubectl get sa -n dabao
        NAME         SECRETS   AGE
        default      1         60m
        pod-reader   1         11s
        [root@master zhangsan]# kubectl get secret -n dabao
        NAME                     TYPE                                  DATA   AGE
        default-token-5lkxq      kubernetes.io/service-account-token   3      61m
        pod-reader-token-l49zb   kubernetes.io/service-account-token   3      76s
        [root@master zhangsan]# kubectl describe secret pod-reader-token-l49zb -n dabao
        Name:         pod-reader-token-l49zb
        Namespace:    dabao
        Labels:       <none>
        Annotations:  kubernetes.io/service-account.name: pod-reader
                      kubernetes.io/service-account.uid: b262ec71-9b16-11ea-8c4f-000c294b2dd3
        
        Type:  kubernetes.io/service-account-token
        
        Data
        ====
        ca.crt:     1359 bytes
        namespace:  5 bytes
        token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkYWJhbyIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJwb2QtcmVhZGVyLXRva2VuLWw0OXpiIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6InBvZC1yZWFkZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJiMjYyZWM3MS05YjE2LTExZWEtOGM0Zi0wMDBjMjk0YjJkZDMiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6ZGFiYW86cG9kLXJlYWRlciJ9.vUdzOLAxq4GE6sRGmS4RkK6_s-NeF7mAeNnuZf9zkacYXZSxxWR6yz7h2-xpSPFJJIvZ1IAmP_G0XO4pazlS_DkLn_cNzLwAT7moOp8CxalATySLY-KtXxbRslwpcyLfyxaZ-PSEiI3fKt5f66C0eL3aoFYIM-xukVQNx_UJE2vsmu93WTuUb8XQjMVGUkQW-p7Mw_2f2wCyGbk_Y__LzXv3dRj0df7EoANHPiadYbntO-_k04LNSfrA1cHRZKBgIddO5tF8olw6rTs99IkSWfMUoF4-qCBDROOBf2h8tSMgz3jrOlhAbvq8kuvR4zjBtwUfI4l0c2IS00HZFW3K3g
        [root@master zhangsan]#
 *  7、使用令牌登陆UI界面
    
    ![mark][mark 3]

### 1.4：第三关：准入控制Admission Control ###

> Adminssion Control实际上是一个准入控制器插件列表，发送到API Server的请求都需要经过这个列表中的每个准入控制器 插件的检查，检查不通过，则拒绝请求。

*  查看进程信息
    
        [root@master zhangsan]# ps aux |grep apiserver
        root      12636  3.0 20.9 420460 208584 ?       Ssl  5月17 160:57 /opt/kubernetes/bin/kube-apiserver --logtostderr=true --v=4 --etcd-servers=https://192.168.233.131:2379,https://192.168.233.132:2379,https://192.168.233.133:2379 --bind-address=192.168.233.131 --secure-port=6443 --advertise-address=192.168.233.131 --allow-privileged=true --service-cluster-ip-range=10.0.0.0/24 --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction '//此行是准入控制的插件信息'--authorization-mode=RBAC,Node --kubelet-https=true --enable-bootstrap-token-auth --token-auth-file=/opt/kubernetes/cfg/token.csv --service-node-port-range=30000-50000 --tls-cert-file=/opt/kubernetes/ssl/server.pem --tls-private-key-file=/opt/kubernetes/ssl/server-key.pem --client-ca-file=/opt/kubernetes/ssl/ca.pem --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem --etcd-cafile=/opt/etcd/ssl/ca.pem --etcd-certfile=/opt/etcd/ssl/server.pem --etcd-keyfile=/opt/etcd/ssl/server-key.pem
        root      23615  0.0  0.0 112728   972 pts/0    S+   12:18   0:00 grep --color=auto apiserver

> NamespaceLifecycle： 命令空间回收
> 
> LimitRanger：配额管理
> 
> ServiceAccount： 每个pod中导入方便访问apiserver
> 
> ResourceQuota： 基于命名空间的高级配额管理
> 
> NodeRestriction： Node加入到k8s群集中以最小权限运行
> 
> 官网推荐的插件：
> 
> 1.11版本以上推荐使用的插件：
> 
> –enable-admission-plugins= \\ NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds, ResourceQuota

### 如有疑问可评论区交流！ ###

[mark]: https://imgconvert.csdnimg.cn/aHR0cDovL3Rhbmd6aGVuZy1waWNnby5vc3MtY24taGFuZ3pob3UuYWxpeXVuY3MuY29tL2ltZy8yMDIwMDUxNS8xNzEyNTMxMTcucG5n?x-oss-process=image/format,png
[mark 1]: https://imgconvert.csdnimg.cn/aHR0cDovL3Rhbmd6aGVuZy1waWNnby5vc3MtY24taGFuZ3pob3UuYWxpeXVuY3MuY29tL2ltZy8yMDIwMDUyMS8xMDQ5NTAwMDAucG5n?x-oss-process=image/format,png
[mark 2]: https://imgconvert.csdnimg.cn/aHR0cDovL3Rhbmd6aGVuZy1waWNnby5vc3MtY24taGFuZ3pob3UuYWxpeXVuY3MuY29tL2ltZy8yMDIwMDUyMS8xMTQ5NDI1MzkucG5n?x-oss-process=image/format,png
[mark 3]: https://imgconvert.csdnimg.cn/aHR0cDovL3Rhbmd6aGVuZy1waWNnby5vc3MtY24taGFuZ3pob3UuYWxpeXVuY3MuY29tL2ltZy8yMDIwMDUyMS8xMTU4MjM0MDcucG5n?x-oss-process=image/format,png