从HelloWorld看Knative Serving代码实现

柔情只为你懂 2022-01-14 05:11 168阅读 0赞

[2019独角兽企业重金招聘Python工程师标准>>> ][2019_Python_] ![hot3.png][]

## 概念先知 ##

官方给出的这几个资源的关系图还是比较清晰的：  
![eb4afff7b0e4976878028c63eb80d335752.jpg][]  
1.Service: 自动管理工作负载整个生命周期。负责创建route,configuration以及每个service更新的revision。通过Service可以指定路由流量使用最新的revision，还是固定的revision。  
2.Route：负责映射网络端点到一个或多个revision。可以通过多种方式管理流量。包括灰度流量和重命名路由。  
3.Configuration:负责保持deployment的期望状态，提供了代码和配置之间清晰的分离，并遵循应用开发的12要素。修改一次Configuration产生一个revision。  
4.Revision：Revision资源是对工作负载进行的每个修改的代码和配置的时间点快照。Revision是不可变对象，可以长期保留。

## 看一个简单的示例 ##

我们开始运行官方hello-world示例，看看会发生什么事情：

apiVersion: serving.knative.dev/v1alpha1
    kind: Service
    metadata:
      name: helloworld-go
      namespace: default
    spec:
      runLatest: // RunLatest defines a simple Service. It will automatically configure a route that keeps the latest ready revision from the supplied configuration running.
        configuration:
          revisionTemplate:
            spec:
              container:
                image: registry.cn-shanghai.aliyuncs.com/larus/helloworld-go
                env:
                - name: TARGET
                  value: "Go Sample v1"

查看 knative-ingressgateway：

kubectl get svc knative-ingressgateway -n istio-system

![21debdf2db18528fe306d610a244c577d6a.jpg][]

查看服务访问：DOMAIN

kubectl get ksvc helloworld-go  --output=custom-columns=NAME:.metadata.name,DOMAIN:.status.domain

![fa22bb7bbd9b626700210cd4f660955e719.jpg][]  
这里直接使用cluster ip即可访问

curl -H "Host: helloworld-go.default.example.com" http://10.96.199.35

![bd0595ac98d84656baec56a34594c09b857.jpg][]

目前看一下服务是部署ok的。那我们看一下k8s里面创建了哪些资源：![26e35f7eb2a5092c98600b74695d6ed3de1.jpg][]

我们可以发现通过Serving,在k8s中创建了2个service和1个deployment：  
![f73db2235d3b43e3d690046a6c8c99d666d.jpg][]  
那么究竟Serving中做了哪些处理，接下来我们分析一下Serving源代码

## 源代码分析 ##

### Main ###

先看一下各个组件的控制器启动代码，这个比较好找，在/cmd/controller/main.go中。  
依次启动configuration、revision、route、labeler、service和clusteringress控制器。

...
    controllers := []*controller.Impl{
            configuration.NewController(
                opt,
                configurationInformer,
                revisionInformer,
            ),
            revision.NewController(
                opt,
                revisionInformer,
                kpaInformer,
                imageInformer,
                deploymentInformer,
                coreServiceInformer,
                endpointsInformer,
                configMapInformer,
                buildInformerFactory,
            ),
            route.NewController(
                opt,
                routeInformer,
                configurationInformer,
                revisionInformer,
                coreServiceInformer,
                clusterIngressInformer,
            ),
            labeler.NewRouteToConfigurationController(
                opt,
                routeInformer,
                configurationInformer,
                revisionInformer,
            ),
            service.NewController(
                opt,
                serviceInformer,
                configurationInformer,
                routeInformer,
            ),
            clusteringress.NewController(
                opt,
                clusterIngressInformer,
                virtualServiceInformer,
            ),
        }
    ...

### Service ###

首先我们要从Service来看，因为我们一开始的输入就是Service资源。在/pkg/reconciler/v1alpha1/service/service.go。  
比较简单，就是根据Service创建Configuration和Route资源

func (c *Reconciler) reconcile(ctx context.Context, service *v1alpha1.Service) error {
        ...
        configName := resourcenames.Configuration(service)
        config, err := c.configurationLister.Configurations(service.Namespace).Get(configName)
        if errors.IsNotFound(err) {
            config, err = c.createConfiguration(service)
        ...
        routeName := resourcenames.Route(service)
        route, err := c.routeLister.Routes(service.Namespace).Get(routeName)
        if errors.IsNotFound(err) {
            route, err = c.createRoute(service)
        ...
    }

### Route ###

/pkg/reconciler/v1alpha1/route/route.go  
看一下Route中reconcile做了哪些处理：  
1.判断是否有Ready的Revision可进行traffic  
2.设置目标流量的Revision（runLatest:使用最新的版本;pinned：固定版本，不过已弃用;release:通过允许在两个修订版之间拆分流量，逐步扩大到新修订版，用于替换pinned。manual：手动模式，目前来看并未实现）  
3.创建ClusterIngress:Route不直接依赖于VirtualService\[[https://istio.io/docs/reference/config/istio.networking.v1alpha3/\#VirtualService\]][https_istio.io_docs_reference_config_istio.networking.v1alpha3_VirtualService] ,而是依赖一个中间资源ClusterIngress，它可以针对不同的网络平台进行不同的协调。目前实现是基于istio网络平台。  
4.创建k8s service:这个Service主要为Istio路由提供域名访问。

func (c *Reconciler) reconcile(ctx context.Context, r *v1alpha1.Route) error {
        ....
        // 基于是否有Ready的Revision
        traffic, err := c.configureTraffic(ctx, r)
        if traffic == nil || err != nil {
            // Traffic targets aren't ready, no need to configure child resources.
            return err
        }
    
        logger.Info("Updating targeted revisions.")
        // In all cases we will add annotations to the referred targets.  This is so that when they become
        // routable we can know (through a listener) and attempt traffic configuration again.
        if err := c.reconcileTargetRevisions(ctx, traffic, r); err != nil {
            return err
        }
    
        // Update the information that makes us Addressable.
        r.Status.Domain = routeDomain(ctx, r)
        r.Status.DeprecatedDomainInternal = resourcenames.K8sServiceFullname(r)
        r.Status.Address = &duckv1alpha1.Addressable{
            Hostname: resourcenames.K8sServiceFullname(r),
        }
    
        // Add the finalizer before creating the ClusterIngress so that we can be sure it gets cleaned up.
        if err := c.ensureFinalizer(r); err != nil {
            return err
        }
    
        logger.Info("Creating ClusterIngress.")
        desired := resources.MakeClusterIngress(r, traffic, ingressClassForRoute(ctx, r))
        clusterIngress, err := c.reconcileClusterIngress(ctx, r, desired)
        if err != nil {
            return err
        }
        r.Status.PropagateClusterIngressStatus(clusterIngress.Status)
    
        logger.Info("Creating/Updating placeholder k8s services")
        if err := c.reconcilePlaceholderService(ctx, r, clusterIngress); err != nil {
            return err
        }
    
        r.Status.ObservedGeneration = r.Generation
        logger.Info("Route successfully synced")
        return nil
    }

看一下helloworld-go生成的Route资源文件：

apiVersion: serving.knative.dev/v1alpha1
    kind: Route
    metadata:
      name: helloworld-go
      namespace: default
    ...
    spec:
      generation: 1
      traffic:
      - configurationName: helloworld-go 
        percent: 100
    status:
    ...
      domain: helloworld-go.default.example.com
      domainInternal: helloworld-go.default.svc.cluster.local
      traffic:
      - percent: 100 # 所有的流量通过这个revision
        revisionName: helloworld-go-00001 # 使用helloworld-go-00001 revision

这里可以看到通过helloworld-go配置, 找到了已经ready的helloworld-go-00001(Revision)。

### Configuration ###

/pkg/reconciler/v1alpha1/configuration/configuration.go  
1.获取当前Configuration对应的Revision, 若不存在则创建。  
2.为Configuration设置最新的Revision  
3.根据Revision是否readiness，设置Configuration的状态LatestReadyRevisionName

func (c *Reconciler) reconcile(ctx context.Context, config *v1alpha1.Configuration) error {
        ...
        // First, fetch the revision that should exist for the current generation.
        lcr, err := c.latestCreatedRevision(config)
        if errors.IsNotFound(err) {
            lcr, err = c.createRevision(ctx, config)
        ...    
        revName := lcr.Name
        // Second, set this to be the latest revision that we have created.
        config.Status.SetLatestCreatedRevisionName(revName)
        config.Status.ObservedGeneration = config.Generation
    
        // Last, determine whether we should set LatestReadyRevisionName to our
        // LatestCreatedRevision based on its readiness.
        rc := lcr.Status.GetCondition(v1alpha1.RevisionConditionReady)
        switch {
        case rc == nil || rc.Status == corev1.ConditionUnknown:
            logger.Infof("Revision %q of configuration %q is not ready", revName, config.Name)
    
        case rc.Status == corev1.ConditionTrue:
            logger.Infof("Revision %q of configuration %q is ready", revName, config.Name)
    
            created, ready := config.Status.LatestCreatedRevisionName, config.Status.LatestReadyRevisionName
            if ready == "" {
                // Surface an event for the first revision becoming ready.
                c.Recorder.Event(config, corev1.EventTypeNormal, "ConfigurationReady",
                    "Configuration becomes ready")
            }
            // Update the LatestReadyRevisionName and surface an event for the transition.
            config.Status.SetLatestReadyRevisionName(lcr.Name)
            if created != ready {
                c.Recorder.Eventf(config, corev1.EventTypeNormal, "LatestReadyUpdate",
                    "LatestReadyRevisionName updated to %q", lcr.Name)
            }
    ...
    }

看一下helloworld-go生成的Configuration资源文件：

apiVersion: serving.knative.dev/v1alpha1
    kind: Configuration
    metadata:
      name: helloworld-go
      namespace: default
      ...
    spec:
      generation: 1
      revisionTemplate:
        metadata:
          creationTimestamp: null
        spec:
          container:
            env:
            - name: TARGET
              value: Go Sample v1
            image: registry.cn-shanghai.aliyuncs.com/larus/helloworld-go
            name: ""
            resources: {}
          timeoutSeconds: 300
    status:
      ...
      latestCreatedRevisionName: helloworld-go-00001
      latestReadyRevisionName: helloworld-go-00001
      observedGeneration: 1

我们可以发现LatestReadyRevisionName设置了helloworld-go-00001(Revision)。

### Revision ###

/pkg/reconciler/v1alpha1/revision/revision.go  
1.获取build进度  
2.设置镜像摘要  
3.创建deployment  
4.创建k8s service：根据Revision构建服务访问Service  
5.创建fluentd configmap  
6.创建KPA  
感觉这段代码写的很优雅，函数执行过程写的很清晰，值得借鉴。另外我们也可以发现，目前knative只支持deployment的工作负载

func (c *Reconciler) reconcile(ctx context.Context, rev *v1alpha1.Revision) error {
        ...
        if err := c.reconcileBuild(ctx, rev); err != nil {
            return err
        }
    
        bc := rev.Status.GetCondition(v1alpha1.RevisionConditionBuildSucceeded)
        if bc == nil || bc.Status == corev1.ConditionTrue {
            // There is no build, or the build completed successfully.
    
            phases := []struct {
                name string
                f    func(context.Context, *v1alpha1.Revision) error
            }{
        {
                name: "image digest",
                f:    c.reconcileDigest,
            }, {
                name: "user deployment",
                f:    c.reconcileDeployment,
            }, {
                name: "user k8s service",
                f:    c.reconcileService,
            }, {
                // Ensures our namespace has the configuration for the fluentd sidecar.
                name: "fluentd configmap",
                f:    c.reconcileFluentdConfigMap,
            }, {
                name: "KPA",
                f:    c.reconcileKPA,
            }}
    
            for _, phase := range phases {
                if err := phase.f(ctx, rev); err != nil {
                    logger.Errorf("Failed to reconcile %s: %v", phase.name, zap.Error(err))
                    return err
                }
            }
        }
    ...
    }

最后我们看一下生成的Revision资源：

apiVersion: serving.knative.dev/v1alpha1
    kind: Service
    metadata:
      name: helloworld-go
      namespace: default
      ...
    spec:
      generation: 1
      runLatest:
        configuration:
          revisionTemplate:
            spec:
              container:
                env:
                - name: TARGET
                  value: Go Sample v1
                image: registry.cn-shanghai.aliyuncs.com/larus/helloworld-go
              timeoutSeconds: 300
    status:
      address:
        hostname: helloworld-go.default.svc.cluster.local
     ...
      domain: helloworld-go.default.example.com
      domainInternal: helloworld-go.default.svc.cluster.local
      latestCreatedRevisionName: helloworld-go-00001
      latestReadyRevisionName: helloworld-go-00001
      observedGeneration: 1
      traffic:
      - percent: 100
        revisionName: helloworld-go-00001

这里我们可以看到访问域名helloworld-go.default.svc.cluster.local，以及当前revision的流量配比(100%)  
**这样我们分析完之后，现在打开Serving这个黑盒**：![fc64e62ced6d3c58e636f4a4dbaa2128182.jpg][]

## 最后 ##

这里只是基于简单的例子，分析了主要的业务流程处理代码。对于activator(如何唤醒业务容器)，autoscaler（Pod如何自动缩为0）等代码实现有兴趣的同学可以一起交流。

[原文链接][Link 1]  
本文为云栖社区原创内容，未经允许不得转载。

转载于:https://my.oschina.net/u/1464083/blog/3052268

[2019_Python_]: https://my.oschina.net/u/2663968/blog/3061697
[hot3.png]: /images/20220114/8055f1cb412e4f60bbb9eb9c848f2a7a.png
[eb4afff7b0e4976878028c63eb80d335752.jpg]: /images/20220114/397ab1cb983d406ba4b9ffe3df62dcd9.png
[21debdf2db18528fe306d610a244c577d6a.jpg]: /images/20220114/0b3ad1dd1d3c4932ba5fbc37b4530bd7.png
[fa22bb7bbd9b626700210cd4f660955e719.jpg]: /images/20220114/9c29e23168fc4cbab90741507ae2df33.png
[bd0595ac98d84656baec56a34594c09b857.jpg]: /images/20220114/09e56e1917cd4a6096e9c2d5b9b543c2.png
[26e35f7eb2a5092c98600b74695d6ed3de1.jpg]: /images/20220114/0457015971794915a09272dfae03cbb3.png
[f73db2235d3b43e3d690046a6c8c99d666d.jpg]: /images/20220114/88bd308a737e47a9bc423ee92efa23a9.png
[https_istio.io_docs_reference_config_istio.networking.v1alpha3_VirtualService]: https://istio.io/docs/reference/config/istio.networking.v1alpha3/#VirtualService%5D
[fc64e62ced6d3c58e636f4a4dbaa2128182.jpg]: /images/20220114/825292c7d54542dea989ad08ffee56e0.png
[Link 1]: https://yq.aliyun.com/articles/701985?utm_content=g_1000058763