深度解析Istio系列之Istio-proxy初始化篇

深碍√TFBOYSˉ_ 2022-03-02 09:24 742阅读 0赞

注：以下讲述的按理环境场景是基于Kubernetes环境基础上部署的Istio环境。

涉及到Envoy概念介绍请参考[深度解析Istio系列之流量控制篇][Istio]。本文重点针对Envoy初始化场景进行拆解。

Istio-proxy（Envoy）作为Istio数据平面的重要组件，基于sidecar方式与业务应用混合部署到同一pod，为应用提供代理服务。Pilot作为控制平面组件，基于元数据的抽象层，屏蔽底层具体容器环境（Kubernetes或者docker），同时为Envoy的策略执行提供有效的数据支撑。**那么Envoy如何获取Pilot所在地址的信息呢？**

下面通过官网BookInfo案例中Productpage服务的配置文件来说明。

### Pod容器说明 ###

执行kubectl get deploy productpage-v1 -o yaml，获取productpage部署模板，如下所示：

apiVersion: extensions/v1beta1
    
    kind: Deployment
    
    metadata:
    
      annotations:
    
        deployment.kubernetes.io/revision: "2"
    
        kubectl.kubernetes.io/last-applied-configuration: |
    
    {"apiVersion":"extensions/v1beta1","kind":"Deployment","metadata":{"annotations":{},"creationTimestamp":null,"name":"productpage-v1","namespace":"default"},"spec":{"replicas":1,"strategy":{},"template":{"metadata":{"annotations":{"sidecar.istio.io/status":"{\"version\":\"50128f63e7b050c58e1cdce95b577358054109ad2aff4bc4995158c06924a43b\",\"initContainers\":[\"istio-init\"],\"containers\":[\"istio-proxy\"],\"volumes\":[\"istio-envoy\",\"istio-certs\"],\"imagePullSecrets\":null}"},"creationTimestamp":null,"labels":{"app":"productpage","version":"v1"}},"spec":{"containers":[{"image":"istio/examples-bookinfo-productpage-v1:1.8.0","imagePullPolicy":"IfNotPresent","name":"productpage","ports":[{"containerPort":9080}],"resources":{}},{"args":["proxy","sidecar","--configPath","/etc/istio/proxy","--binaryPath","/usr/local/bin/envoy","--serviceCluster","productpage","--drainDuration","45s","--parentShutdownDuration","1m0s","--discoveryAddress","istio-pilot.istio-system:15007","--discoveryRefreshDelay","1s","--zipkinAddress","zipkin.istio-system:9411","--connectTimeout","10s","--proxyAdminPort","15000","--controlPlaneAuthPolicy","NONE"],"env":[{"name":"POD_NAME","valueFrom":{"fieldRef":{"fieldPath":"metadata.name"}}},{"name":"POD_NAMESPACE","valueFrom":{"fieldRef":{"fieldPath":"metadata.namespace"}}},{"name":"INSTANCE_IP","valueFrom":{"fieldRef":{"fieldPath":"status.podIP"}}},{"name":"ISTIO_META_POD_NAME","valueFrom":{"fieldRef":{"fieldPath":"metadata.name"}}},{"name":"ISTIO_META_INTERCEPTION_MODE","value":"REDIRECT"},{"name":"ISTIO_METAJSON_LABELS","value":"{\"app\":\"productpage\",\"version\":\"v1\"}\n"}],"image":"docker.io/istio/proxyv2:1.0.5","imagePullPolicy":"IfNotPresent","name":"istio-proxy","ports":[{"containerPort":15090,"name":"http-envoy-prom","protocol":"TCP"}],"resources":{"requests":{"cpu":"10m"}},"securityContext":{"readOnlyRootFilesystem":true,"runAsUser":1337},"volumeMounts":[{"mountPath":"/etc/istio/proxy","name":"istio-envoy"},{"mountPath":"/etc/certs/","name":"istio-certs","readOnly":true}]}],"initContainers":[{"args":["-p","15001","-u","1337","-m","REDIRECT","-i","*","-x","","-b","9080","-d",""],"image":"docker.io/istio/proxy_init:1.0.5","imagePullPolicy":"IfNotPresent","name":"istio-init","resources":{},"securityContext":{"capabilities":{"add":["NET_ADMIN"]},"privileged":true}}],"volumes":[{"emptyDir":{"medium":"Memory"},"name":"istio-envoy"},{"name":"istio-certs","secret":{"optional":true,"secretName":"istio.default"}}]}}},"status":{}}
    
      creationTimestamp: 2018-12-18T01:51:18Z
    
      generation: 2
    
      labels:
    
        app: productpage
    
        version: v1
    
      name: productpage-v1
    
      namespace: default
    
      resourceVersion: "17402700"
    
      selfLink: /apis/extensions/v1beta1/namespaces/default/deployments/productpage-v1
    
      uid: 695ba22c-0267-11e9-8475-0050569c62d0
    
    spec:
    
      progressDeadlineSeconds: 600
    
      replicas: 1
    
      revisionHistoryLimit: 10
    
      selector:
    
        matchLabels:
    
          app: productpage
    
          version: v1
    
      strategy:
    
        rollingUpdate:
    
          maxSurge: 1
    
          maxUnavailable: 1
    
        type: RollingUpdate
    
      template:
    
        metadata:
    
          annotations:
    
            sidecar.istio.io/status: '{"version":"50128f63e7b050c58e1cdce95b577358054109ad2aff4bc4995158c06924a43b","initContainers":["istio-init"],"containers":["istio-proxy"],"volumes":["istio-envoy","istio-certs"],"imagePullSecrets":null}'
    
          creationTimestamp: null
    
          labels:
    
            app: productpage
    
            version: v1
    
        spec:
    
          containers:
    
          - image: istio/examples-bookinfo-productpage-v1:1.8.0
    
            imagePullPolicy: IfNotPresent
    
            name: productpage
    
            ports:
    
            - containerPort: 9080
    
              protocol: TCP
    
            resources: {}
    
            terminationMessagePath: /dev/termination-log
    
            terminationMessagePolicy: File
    
          - args:
    
            - proxy
    
            - sidecar
    
            - --configPath
    
            - /etc/istio/proxy
    
            - --binaryPath
    
            - /usr/local/bin/envoy
    
            - --serviceCluster
    
            - productpage
    
            - --drainDuration
    
            - 45s
    
            - --parentShutdownDuration
    
            - 1m0s
    
            - --discoveryAddress
    
            - istio-pilot.istio-system:15007
    
            - --discoveryRefreshDelay
    
            - 1s
    
            - --zipkinAddress
    
            - zipkin.istio-system:9411
    
            - --connectTimeout
    
            - 10s
    
            - --proxyAdminPort
    
            - "15000"
    
            - --controlPlaneAuthPolicy
    
            - NONE
    
            env:
    
            - name: POD_NAME
    
              valueFrom:
    
                fieldRef:
    
                  apiVersion: v1
    
                  fieldPath: metadata.name
    
            - name: POD_NAMESPACE
    
              valueFrom:
    
                fieldRef:
    
                  apiVersion: v1
    
                  fieldPath: metadata.namespace
    
            - name: INSTANCE_IP
    
              valueFrom:
    
                fieldRef:
    
                  apiVersion: v1
    
                  fieldPath: status.podIP
    
            - name: ISTIO_META_POD_NAME
    
              valueFrom:
    
                fieldRef:
    
                  apiVersion: v1
    
                  fieldPath: metadata.name
    
            - name: ISTIO_META_INTERCEPTION_MODE
    
              value: REDIRECT
    
            - name: ISTIO_METAJSON_LABELS
    
              value: |
    
                {"app":"productpage","version":"v1"}
    
            image: docker.io/istio/proxyv2:1.0.5
    
            imagePullPolicy: IfNotPresent
    
            name: istio-proxy
    
            ports:
    
            - containerPort: 15090
    
              name: http-envoy-prom
    
              protocol: TCP
    
            resources:
    
              requests:
    
                cpu: 10m
    
            securityContext:
    
              readOnlyRootFilesystem: true
    
              runAsUser: 1337
    
            terminationMessagePath: /dev/termination-log
    
            terminationMessagePolicy: File
    
            volumeMounts:
    
            - mountPath: /etc/istio/proxy
    
              name: istio-envoy
    
            - mountPath: /etc/certs/
    
              name: istio-certs
    
              readOnly: true
    
          dnsPolicy: ClusterFirst
    
          initContainers:
    
          - args:
    
            - -p
    
            - "15001"
    
            - -u
    
            - "1337"
    
            - -m
    
            - REDIRECT
    
            - -i
    
            - '*'
    
            - -x
    
            - ""
    
            - -b
    
            - "9080"
    
            - -d
    
            - ""
    
            image: docker.io/istio/proxy_init:1.0.5
    
            imagePullPolicy: IfNotPresent
    
            name: istio-init
    
            resources: {}
    
            securityContext:
    
              capabilities:
    
                add:
    
                - NET_ADMIN
    
              privileged: true
    
            terminationMessagePath: /dev/termination-log
    
            terminationMessagePolicy: File
    
          restartPolicy: Always
    
          schedulerName: default-scheduler
    
          securityContext: {}
    
          terminationGracePeriodSeconds: 30
    
          volumes:
    
          - emptyDir:
    
              medium: Memory
    
            name: istio-envoy
    
          - name: istio-certs
    
            secret:
    
              defaultMode: 420
    
              optional: true
    
              secretName: istio.default
    
    status:
    
      availableReplicas: 1
    
      conditions:
    
      - lastTransitionTime: 2018-12-18T01:51:19Z
    
        lastUpdateTime: 2018-12-18T01:51:19Z
    
        message: Deployment has minimum availability.
    
        reason: MinimumReplicasAvailable
    
        status: "True"
    
        type: Available
    
      - lastTransitionTime: 2018-12-18T01:51:18Z
    
        lastUpdateTime: 2018-12-18T03:48:52Z
    
        message: ReplicaSet "productpage-v1-7b96bbf89f" has successfully progressed.
    
        reason: NewReplicaSetAvailable
    
        status: "True"
    
        type: Progressing
    
      observedGeneration: 2
    
      readyReplicas: 1
    
      replicas: 1
    
      updatedReplicas: 1

模板中可见，除业务应用外，还包括istio-init，istio-proxy两个容器。istio-init作为Init Container类型，在应用容器启动之前启动，用来执行初始化任务。Istio-init主要初始化iptables规则，用于服务间调用时的请求拦截及转发。Istio-proxy是代理容器（即Envoy），容器中包含两个进程，分别是Pilot-agent进程和Envoy进程。前者负责生成Envoy启动所需的配置文件，启动Envoy进程。后者在服务调用发生时负责具体的策略执行。

### Envoy配置信息详解 ###

Envoy启动后，执行kubectl exec -it productpage-v1-7b96bbf89f-pj28l -c istio-proxy /bin/sh，进入istio-proxy容器内部，在/etc/istio/proxy目录下存在envoy-rev0.json配置文件，如图2所示：

![20190320103031274.png][]

该文件是Pilot是Pilot-agent进程根据服务启动参数以及Kubernetes Server API的信息生成，配置文件框架如图3所示：

![20190320103040339.png][]

Node包含的信息如图4所示：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NxdWlycmVsYW5pbWFsMDkyMg_size_16_color_FFFFFF_t_70][]

这里重点说明cluster属性，该属性值为productpage，说明当前的istio-proxy服务为productpage应用的代理服务，iptables将调用productpage服务请求拦截后，直接转发到该istio-proxy进行处理。

stats\_config包含的信息如图5所示：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NxdWlycmVsYW5pbWFsMDkyMg_size_16_color_FFFFFF_t_70 1][]

static\_config重点设定部分属性的命名规则。

admin包含的信息如图6所示：

![20190320103129709.png][]

admin类似Envoy内部的一个管理器，暴露出的15000端口只能在容器内部被访问。

通过curl http://127.0.0.1:15000/help访问能获取其他接口信息，如下图7所示：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NxdWlycmVsYW5pbWFsMDkyMg_size_16_color_FFFFFF_t_70 2][]

dynamic\_resources包含的信息如图8所示：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NxdWlycmVsYW5pbWFsMDkyMg_size_16_color_FFFFFF_t_70 3][]

dynamic\_resources中记录pilot调取XDS接口采用ADS聚合发现服务，这种方式能更好的保证从pilot上获取到的数据的一致性。

static\_resources包含的部分信息如图9所示：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NxdWlycmVsYW5pbWFsMDkyMg_size_16_color_FFFFFF_t_70 4][]

static\_resources name属性值为xds-grpc，与上述dynamic\_resources模块的信息里cluster\_name的值一致。同时xds-grpc对象信息中包括pilot组件的服务地址“istio-pilot.istio-system:15010”，即为istio-proxy从pilot获取动态信息的服务地址。Istio-proxy（Envoy）通过该地址调取XDS接口，获取服务信息。除此之外，static\_resources中提供zipkin服务的地址，zipkin是用于服务分布式跟踪。

综上可知Envoy启动时，pilot-agent进程生成Envoy启动所需的配置文件，Envoy基于配置文件中携带的pilot地址信息动态获取服务信息，并且基于pilot提供的XDS接口，动态获取服务的listener，cluster，endpoint和route信息。

BoCloud博云技术社区官方交流群已成立，邀请热爱前沿技术的你共同探讨交流。进群方式：添加管理员微信（ID : ruffylining 或 waret87）拉你进群。（发送好友申请时请注明姓名-公司）

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NxdWlycmVsYW5pbWFsMDkyMg_size_16_color_FFFFFF_t_70 5][]

[Istio]: http://mp.weixin.qq.com/s?__biz=Mzg3OTA4NDgzOQ==&mid=2247483716&idx=1&sn=132e22c81fe32825358a8babdef0746a&chksm=cf08950cf87f1c1af48e4876ea2be0eb4ebe970aef7441c9082eebb71542cb09d8daad7c9d2c&scene=21#wechat_redirect
[20190320103031274.png]: /images/20220302/04621878f83c42e59d5d398c57d6727d.png
[20190320103040339.png]: /images/20220302/741f536fc22d4488b8997d698fd89507.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NxdWlycmVsYW5pbWFsMDkyMg_size_16_color_FFFFFF_t_70]: /images/20220302/e648689b050c494fabc6602cb6e21ee5.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NxdWlycmVsYW5pbWFsMDkyMg_size_16_color_FFFFFF_t_70 1]: /images/20220302/17569b750c53453889ec01d20cf9e203.png
[20190320103129709.png]: /images/20220302/6aba8f65c9f44e0a9084724408c5a4de.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NxdWlycmVsYW5pbWFsMDkyMg_size_16_color_FFFFFF_t_70 2]: /images/20220302/48fdf2bc3fd742908471a25e24cc80dc.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NxdWlycmVsYW5pbWFsMDkyMg_size_16_color_FFFFFF_t_70 3]: /images/20220302/30f3813e28014b85a31cbe12d458148b.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NxdWlycmVsYW5pbWFsMDkyMg_size_16_color_FFFFFF_t_70 4]: /images/20220302/3fd7d71f11fc4a8e9a10e02f71e04e78.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NxdWlycmVsYW5pbWFsMDkyMg_size_16_color_FFFFFF_t_70 5]: https://img-blog.csdnimg.cn/20190320103308870.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NxdWlycmVsYW5pbWFsMDkyMg==,size_16,color_FFFFFF,t_70