k8s集群外访问集群内部服务的几种方式

￡神魔★判官ぃ 2022-10-09 06:19 600阅读 0赞

## 前言 ##

目前k8s+docker算是运维必修了，docker能让我们的环境一键迁移，k8s能自动编排,还能保证服务高可用，两者结合自然是无敌，那我们要不要使用k8s呢？这要根据自身情况，任何技术的使用，都需要根据具体情况来定，就像你要杀鸡，却非要选牛刀，牛刀是很锋利，但还不够你折腾的

本文重点介绍k8s集群外如何访问集群内的服务，主要方式有：hostPort或hostNetwork、NodePort、Ingress

## 一、hostPort或hostNetwork ##

hostPort和hostNetwork 放在首位是因为大家很容易忽略它们，它们也可让集群外访问集群内应用，用法：

*  **hostNetwork**

apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: nginx
      name: nginx-deployment
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec: 
          nodeSelector:      # node节点选择器
            role: master     # node节点标签(Label)
          hostNetwork: true  # 使用node节点网络
          containers:
          - image: nginx
            imagePullPolicy: IfNotPresent
            name: nginx
            ports:
            - containerPort: 8080

重点在和containers平级的**hostNetwork: true**,表示pod使用宿主机网络，配合nodeSelector，把pod实例化在固定节点，如上，我给mater节点加上标签role: master,通过nodeSelector,nginx就会实例化在master节点，这样就可以通过master节点的ip和8080端口访问这个nginx了

*  **hostPort**

apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: nginx
      name: nginx-deployment
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec: 
          nodeSelector:      # node节点选择器
            role: master     # node节点标签(Label)
          containers:
          - image: nginx
            imagePullPolicy: IfNotPresent
            name: nginx
            ports:
            - containerPort: 8080
              hostPort: 80                #重点

和**hostNetwork**相比多了映射能力，可以把容器端口映射为node节点不同端口，**hostPort,**当然也需要**nodeSelector**来固定节点，不然每次创建，节点不同，ip也会改变

访问方式：nodeSelector所选节点ip:hostPort, 如上：role=Master标签节点Ip:80

## 二、NodePort ##

NodePort是最常见的提供集群外访问的方式之一，该方式使用Service提供集群外访问

apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: nginx
      name: nginx-deployment
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec: 
          containers:
          - image: nginx
            imagePullPolicy: IfNotPresent
            name: nginx
            ports:
            - containerPort: 8080
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: nginx-pod-service
      labels:
        app: nginx                  #自身标签
    spec:
      type: NodePort                # 类型ExternalName, ClusterIP, NodePort, and LoadBalancer
      ports:
      - port: 8080                 # service在k8s集群内服务端口
        targetPort: 8080           # 关联pod对外开放端口
        nodePort: 30088            # 集群外访问端口，端口范围【30000-32767】
      selector:
        app: nginx                 # pod标签

访问方式：集群内任意节点ip加nodePort所配端口号，如上：集群内任一节点ip:30088，即可访问nginx

## 三、Ingress ##

Ingress基本生产必备，可以简单理解为部署了一个nginx服务，该服务使用hostNetwork或hostPort方式提供集群外访问，再根据配置的路由规则，路由的集群内部各个service

*  **Ingress安装**

Ingress有各种实现版本，这里安装的为ingress-nginx,其yaml文件如下：ingress.yaml

apiVersion: v1
    kind: Namespace
    metadata:
      name: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
     
    ---
     
    kind: ConfigMap
    apiVersion: v1
    metadata:
      name: nginx-configuration
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
     
    ---
    kind: ConfigMap
    apiVersion: v1
    metadata:
      name: tcp-services
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
     
    ---
    kind: ConfigMap
    apiVersion: v1
    metadata:
      name: udp-services
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
     
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: nginx-ingress-serviceaccount
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
     
    ---
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: ClusterRole
    metadata:
      name: nginx-ingress-clusterrole
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    rules:
      - apiGroups:
          - ""
        resources:
          - configmaps
          - endpoints
          - nodes
          - pods
          - secrets
        verbs:
          - list
          - watch
      - apiGroups:
          - ""
        resources:
          - nodes
        verbs:
          - get
      - apiGroups:
          - ""
        resources:
          - services
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - ""
        resources:
          - events
        verbs:
          - create
          - patch
      - apiGroups:
          - "extensions"
          - "networking.k8s.io"
        resources:
          - ingresses
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - "extensions"
          - "networking.k8s.io"
        resources:
          - ingresses/status
        verbs:
          - update
     
    ---
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: Role
    metadata:
      name: nginx-ingress-role
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    rules:
      - apiGroups:
          - ""
        resources:
          - configmaps
          - pods
          - secrets
          - namespaces
        verbs:
          - get
      - apiGroups:
          - ""
        resources:
          - configmaps
        resourceNames:
          # Defaults to "<election-id>-<ingress-class>"
          # Here: "<ingress-controller-leader>-<nginx>"
          # This has to be adapted if you change either parameter
          # when launching the nginx-ingress-controller.
          - "ingress-controller-leader-nginx"
        verbs:
          - get
          - update
      - apiGroups:
          - ""
        resources:
          - configmaps
        verbs:
          - create
      - apiGroups:
          - ""
        resources:
          - endpoints
        verbs:
          - get
     
    ---
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: RoleBinding
    metadata:
      name: nginx-ingress-role-nisa-binding
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: Role
      name: nginx-ingress-role
    subjects:
      - kind: ServiceAccount
        name: nginx-ingress-serviceaccount
        namespace: ingress-nginx
     
    ---
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: ClusterRoleBinding
    metadata:
      name: nginx-ingress-clusterrole-nisa-binding
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: nginx-ingress-clusterrole
    subjects:
      - kind: ServiceAccount
        name: nginx-ingress-serviceaccount
        namespace: ingress-nginx
     
    ---
     
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: nginx-ingress-controller
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    spec:
      replicas: 1
      selector:
        matchLabels:
          app.kubernetes.io/name: ingress-nginx
          app.kubernetes.io/part-of: ingress-nginx
      template:
        metadata:
          labels:
            app.kubernetes.io/name: ingress-nginx
            app.kubernetes.io/part-of: ingress-nginx
          annotations:
            prometheus.io/port: "10254"
            prometheus.io/scrape: "true"
        spec:
          # wait up to five minutes for the drain of connections
          terminationGracePeriodSeconds: 300
          serviceAccountName: nginx-ingress-serviceaccount
          nodeSelector:                  #注意修改自己nodeSelector
            role: master
          hostNetwork: true
          containers:
            - name: nginx-ingress-controller
              image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0
              args:
                - /nginx-ingress-controller
                - --configmap=$(POD_NAMESPACE)/nginx-configuration
                - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
                - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
                - --publish-service=$(POD_NAMESPACE)/ingress-nginx
                - --annotations-prefix=nginx.ingress.kubernetes.io
              securityContext:
                allowPrivilegeEscalation: true
                capabilities:
                  drop:
                    - ALL
                  add:
                    - NET_BIND_SERVICE
                # www-data -> 101
                runAsUser: 101
              env:
                - name: POD_NAME
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.name
                - name: POD_NAMESPACE
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.namespace
              ports:
                - name: http
                  containerPort: 80
                  protocol: TCP
                - name: https
                  containerPort: 443
                  protocol: TCP
              livenessProbe:
                failureThreshold: 3
                httpGet:
                  path: /healthz
                  port: 10254
                  scheme: HTTP
                initialDelaySeconds: 10
                periodSeconds: 10
                successThreshold: 1
                timeoutSeconds: 10
              readinessProbe:
                failureThreshold: 3
                httpGet:
                  path: /healthz
                  port: 10254
                  scheme: HTTP
                periodSeconds: 10
                successThreshold: 1
                timeoutSeconds: 10
              lifecycle:
                preStop:
                  exec:
                    command:
                      - /wait-shutdown
     
    ---
     
    apiVersion: v1
    kind: LimitRange
    metadata:
      name: ingress-nginx
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    spec:
      limits:
      - min:
          memory: 90Mi
          cpu: 100m
        type: Container

kubectl apply -f ingress.yaml

该yaml会创建ingress-nginx命名空间，待对应pod进入running状态，ingress服务就安装完成了

![20200603161350490.png][]

*  **配置ingress路由规则**

apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: nginx
      name: nginx-deployment
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec: 
          containers:
          - image: nginx
            imagePullPolicy: IfNotPresent
            name: nginx
            ports:
            - containerPort: 8080
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: nginx-pod-service
      labels:
        app: nginx                  #自身标签
    spec:
      type: NodePort                # 类型ExternalName, ClusterIP, NodePort, and LoadBalancer
      ports:
      - port: 8080                 # service在k8s集群内服务端口
        targetPort: 8080           # 关联pod对外开放端口
      selector:
        app: nginx                 # pod标签
    ---
    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
      name: nginx-ingress
      labels:
        app.kubernetes.io/version: v1
      annotations:
        nginx.ingress.kubernetes.io/rewrite-target: /
    spec:
      rules:
      - host: nginx.test.com
        http:
          paths:
          - path: /
            backend:
              serviceName: nginx-pod-service     # 上面nginx Service配置的name
              servicePort: 8080                  #  Service配置的port

集群外配置好host

ingress服务部署节点的ip   nginx.test.com

通过域名进行访问，且仅能通过域名

## 总结各方式利弊 ##

**hostPort**和**hostNetwork**直接使用节点网络，部署时节点需固定，访问ip也固定(也可以用host)，端口为正常端口

**nodeport**方式部署时不要求固定节点，可通过集群内任一ip进行访问，就是端口为30000以上，很多时候由于公司安全策略导致不能访问

**ingress**需要额外安装ingress模块，配置路由规则，且仅能通过所配置域名访问，配置好域名后，可以直接对外提供服务，和传统的nginx作用类似

可根据各自情况灵活使用

[20200603161350490.png]: /images/20221005/26714f45851b42db8c46af42c522bce3.png