在Kubernetes上最小化安装KubeSphere

Dear 丶 2022-09-07 09:33 252阅读 0赞

前言：本文案例：安装KubeSphere v3.1.1

首先，安装KubeSphere是有先决条件的:  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L1Jvb2tpZXhpYW9NdV9h_size_16_color_FFFFFF_t_70]

1.  kubernetes集群版本若是不满足，那只能安装其他版本的KubeSphere了，或者升级k8sjiqun版本
2.  机器可用CPU > 1核；内存 > 2G
3.  Kubernetes 集群已配置默认 StorageClass（请使用 kubectl get sc 进行确认）
4.  不用理会

这里主要说一下第三点：Kubernetes 集群已配置默认 StorageClass

在master节点输入命令：

-- 
    kubectl get sc

若是出现，则表示已经配置了，若是没有配置，请接着往下看。  
![在这里插入图片描述][c2cce4987d9c4e0eb19c4fc441088b7f.png]

1.  确认 master 节点是否有 Taint，如下看到 master 节点有 Taint。

$ kubectl describe node k8s-master | grep Taint
    Taints:             node-role.kubernetes.io/master:NoSchedule

1.  去掉 master 节点的 Taint：

$ kubectl taint nodes k8s-master node-role.kubernetes.io/master:NoSchedule-

> k8s-master：master节点主机名

1.  安装 OpenEBS

创建 OpenEBS 的 namespace，OpenEBS 相关资源将创建在这个 namespace 下：

$ kubectl create ns openebs

安装OpenEBS官网文档（v2.1X）给定两种方案：  
A. 若集群已安装了 Helm，可通过 Helm 命令来安装 OpenEBS：

helm init
    
    helm install --namespace openebs --name openebs stable/openebs --version 1.5.0

B. 除此之外 还可以通过 kubectl 命令安装：

$ kubectl apply -f https://openebs.github.io/charts/openebs-operator-1.5.0.yaml

因为我的k8s集群已经安装了kubectl，所以我们使用第二种方法。

但是在输入上面的命令却并不会安装成功，因为https://openebs.github.io/charts/openebs-operator-1.5.0.yaml这个链接已经无效了。

编辑文件：openebs-operator.yaml

vi openebs-operator.yaml

将以下内容复制进去并保存：

#
    #                             DEPRECATION NOTICE
    #    This operator file is deprecated in 2.11.0 in favour of individual operators
    #       for each storage engine and the file will be removed in version 3.0.0
    #
    # Further specific components can be deploy using there individual operator yamls
    #
    # To deploy cStor:
    # https://github.com/openebs/charts/blob/gh-pages/cstor-operator.yaml
    #
    # To deploy Jiva:
    # https://github.com/openebs/charts/blob/gh-pages/jiva-operator.yaml
    #
    # To deploy Dynamic hostpath localpv provisioner:
    # https://github.com/openebs/charts/blob/gh-pages/hostpath-operator.yaml
    #
    #
    # This manifest deploys the OpenEBS control plane components, with associated CRs & RBAC rules
    # NOTE: On GKE, deploy the openebs-operator.yaml in admin context
    
    # Create the OpenEBS namespace
    apiVersion: v1
    kind: Namespace
    metadata:
      name: openebs
    ---
    # Create Maya Service Account
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: openebs-maya-operator
      namespace: openebs
    ---
    # Define Role that allows operations on K8s pods/deployments
    kind: ClusterRole
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      name: openebs-maya-operator
    rules:
    - apiGroups: ["*"]
      resources: ["nodes", "nodes/proxy"]
      verbs: ["*"]
    - apiGroups: ["*"]
      resources: ["namespaces", "services", "pods", "pods/exec", "deployments", "deployments/finalizers", "replicationcontrollers", "replicasets", "events", "endpoints", "configmaps", "secrets", "jobs", "cronjobs"]
      verbs: ["*"]
    - apiGroups: ["*"]
      resources: ["statefulsets", "daemonsets"]
      verbs: ["*"]
    - apiGroups: ["*"]
      resources: ["resourcequotas", "limitranges"]
      verbs: ["list", "watch"]
    - apiGroups: ["*"]
      resources: ["ingresses", "horizontalpodautoscalers", "verticalpodautoscalers", "certificatesigningrequests"]
      verbs: ["list", "watch"]
    - apiGroups: ["*"]
      resources: ["storageclasses", "persistentvolumeclaims", "persistentvolumes"]
      verbs: ["*"]
    - apiGroups: ["volumesnapshot.external-storage.k8s.io"]
      resources: ["volumesnapshots", "volumesnapshotdatas"]
      verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
    - apiGroups: ["apiextensions.k8s.io"]
      resources: ["customresourcedefinitions"]
      verbs: [ "get", "list", "create", "update", "delete", "patch"]
    - apiGroups: ["openebs.io"]
      resources: [ "*"]
      verbs: ["*" ]
    - apiGroups: ["cstor.openebs.io"]
      resources: [ "*"]
      verbs: ["*" ]
    - apiGroups: ["coordination.k8s.io"]
      resources: ["leases"]
      verbs: ["get", "watch", "list", "delete", "update", "create"]
    - apiGroups: ["admissionregistration.k8s.io"]
      resources: ["validatingwebhookconfigurations", "mutatingwebhookconfigurations"]
      verbs: ["get", "create", "list", "delete", "update", "patch"]
    - nonResourceURLs: ["/metrics"]
      verbs: ["get"]
    - apiGroups: ["*"]
      resources: ["poddisruptionbudgets"]
      verbs: ["get", "list", "create", "delete", "watch"]
    ---
    # Bind the Service Account with the Role Privileges.
    # TODO: Check if default account also needs to be there
    kind: ClusterRoleBinding
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      name: openebs-maya-operator
    subjects:
    - kind: ServiceAccount
      name: openebs-maya-operator
      namespace: openebs
    roleRef:
      kind: ClusterRole
      name: openebs-maya-operator
      apiGroup: rbac.authorization.k8s.io
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: maya-apiserver
      namespace: openebs
      labels:
        name: maya-apiserver
        openebs.io/component-name: maya-apiserver
        openebs.io/version: 2.12.0
    spec:
      selector:
        matchLabels:
          name: maya-apiserver
          openebs.io/component-name: maya-apiserver
      replicas: 1
      strategy:
        type: Recreate
        rollingUpdate: null
      template:
        metadata:
          labels:
            name: maya-apiserver
            openebs.io/component-name: maya-apiserver
            openebs.io/version: 2.12.0
        spec:
          serviceAccountName: openebs-maya-operator
          containers:
          - name: maya-apiserver
            imagePullPolicy: IfNotPresent
            image: openebs/m-apiserver:2.12.0
            ports:
            - containerPort: 5656
            env:
            # OPENEBS_IO_KUBE_CONFIG enables maya api service to connect to K8s
            # based on this config. This is ignored if empty.
            # This is supported for maya api server version 0.5.2 onwards
            #- name: OPENEBS_IO_KUBE_CONFIG
            #  value: "/home/ubuntu/.kube/config"
            # OPENEBS_IO_K8S_MASTER enables maya api service to connect to K8s
            # based on this address. This is ignored if empty.
            # This is supported for maya api server version 0.5.2 onwards
            #- name: OPENEBS_IO_K8S_MASTER
            #  value: "http://172.28.128.3:8080"
            # OPENEBS_NAMESPACE provides the namespace of this deployment as an
            # environment variable
            - name: OPENEBS_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            # OPENEBS_SERVICE_ACCOUNT provides the service account of this pod as
            # environment variable
            - name: OPENEBS_SERVICE_ACCOUNT
              valueFrom:
                fieldRef:
                  fieldPath: spec.serviceAccountName
            # OPENEBS_MAYA_POD_NAME provides the name of this pod as
            # environment variable
            - name: OPENEBS_MAYA_POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            # If OPENEBS_IO_CREATE_DEFAULT_STORAGE_CONFIG is false then OpenEBS default
            # storageclass and storagepool will not be created.
            - name: OPENEBS_IO_CREATE_DEFAULT_STORAGE_CONFIG
              value: "true"
            # OPENEBS_IO_INSTALL_DEFAULT_CSTOR_SPARSE_POOL decides whether default cstor sparse pool should be
            # configured as a part of openebs installation.
            # If "true" a default cstor sparse pool will be configured, if "false" it will not be configured.
            # This value takes effect only if OPENEBS_IO_CREATE_DEFAULT_STORAGE_CONFIG
            # is set to true
            - name: OPENEBS_IO_INSTALL_DEFAULT_CSTOR_SPARSE_POOL
              value: "false"
            # OPENEBS_IO_INSTALL_CRD environment variable is used to enable/disable CRD installation
            # from Maya API server. By default the CRDs will be installed
            # - name: OPENEBS_IO_INSTALL_CRD
            #   value: "true"
            # OPENEBS_IO_BASE_DIR is used to configure base directory for openebs on host path.
            # Where OpenEBS can store required files. Default base path will be /var/openebs
            # - name: OPENEBS_IO_BASE_DIR
            #   value: "/var/openebs"
            # OPENEBS_IO_CSTOR_TARGET_DIR can be used to specify the hostpath
            # to be used for saving the shared content between the side cars
            # of cstor volume pod.
            # The default path used is /var/openebs/sparse
            #- name: OPENEBS_IO_CSTOR_TARGET_DIR
            #  value: "/var/openebs/sparse"
            # OPENEBS_IO_CSTOR_POOL_SPARSE_DIR can be used to specify the hostpath
            # to be used for saving the shared content between the side cars
            # of cstor pool pod. This ENV is also used to indicate the location
            # of the sparse devices.
            # The default path used is /var/openebs/sparse
            #- name: OPENEBS_IO_CSTOR_POOL_SPARSE_DIR
            #  value: "/var/openebs/sparse"
            # OPENEBS_IO_JIVA_POOL_DIR can be used to specify the hostpath
            # to be used for default Jiva StoragePool loaded by OpenEBS
            # The default path used is /var/openebs
            # This value takes effect only if OPENEBS_IO_CREATE_DEFAULT_STORAGE_CONFIG
            # is set to true
            #- name: OPENEBS_IO_JIVA_POOL_DIR
            #  value: "/var/openebs"
            # OPENEBS_IO_LOCALPV_HOSTPATH_DIR can be used to specify the hostpath
            # to be used for default openebs-hostpath storageclass loaded by OpenEBS
            # The default path used is /var/openebs/local
            # This value takes effect only if OPENEBS_IO_CREATE_DEFAULT_STORAGE_CONFIG
            # is set to true
            #- name: OPENEBS_IO_LOCALPV_HOSTPATH_DIR
            #  value: "/var/openebs/local"
            - name: OPENEBS_IO_JIVA_CONTROLLER_IMAGE
              value: "openebs/jiva:2.12.1"
            - name: OPENEBS_IO_JIVA_REPLICA_IMAGE
              value: "openebs/jiva:2.12.1"
            - name: OPENEBS_IO_JIVA_REPLICA_COUNT
              value: "3"
            - name: OPENEBS_IO_CSTOR_TARGET_IMAGE
              value: "openebs/cstor-istgt:2.12.0"
            - name: OPENEBS_IO_CSTOR_POOL_IMAGE
              value: "openebs/cstor-pool:2.12.0"
            - name: OPENEBS_IO_CSTOR_POOL_MGMT_IMAGE
              value: "openebs/cstor-pool-mgmt:2.12.0"
            - name: OPENEBS_IO_CSTOR_VOLUME_MGMT_IMAGE
              value: "openebs/cstor-volume-mgmt:2.12.0"
            - name: OPENEBS_IO_VOLUME_MONITOR_IMAGE
              value: "openebs/m-exporter:2.12.0"
            - name: OPENEBS_IO_CSTOR_POOL_EXPORTER_IMAGE
              value: "openebs/m-exporter:2.12.0"
            - name: OPENEBS_IO_HELPER_IMAGE
              value: "openebs/linux-utils:2.12.0"
            # OPENEBS_IO_ENABLE_ANALYTICS if set to true sends anonymous usage
            # events to Google Analytics
            - name: OPENEBS_IO_ENABLE_ANALYTICS
              value: "true"
            - name: OPENEBS_IO_INSTALLER_TYPE
              value: "openebs-operator"
            # OPENEBS_IO_ANALYTICS_PING_INTERVAL can be used to specify the duration (in hours)
            # for periodic ping events sent to Google Analytics.
            # Default is 24h.
            # Minimum is 1h. You can convert this to weekly by setting 168h
            #- name: OPENEBS_IO_ANALYTICS_PING_INTERVAL
            #  value: "24h"
            livenessProbe:
              exec:
                command:
                - sh
                - -c
                - /usr/local/bin/mayactl
                - version
              initialDelaySeconds: 30
              periodSeconds: 60
            readinessProbe:
              exec:
                command:
                - sh
                - -c
                - /usr/local/bin/mayactl
                - version
              initialDelaySeconds: 30
              periodSeconds: 60
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: maya-apiserver-service
      namespace: openebs
      labels:
        openebs.io/component-name: maya-apiserver-svc
    spec:
      ports:
      - name: api
        port: 5656
        protocol: TCP
        targetPort: 5656
      selector:
        name: maya-apiserver
      sessionAffinity: None
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: openebs-provisioner
      namespace: openebs
      labels:
        name: openebs-provisioner
        openebs.io/component-name: openebs-provisioner
        openebs.io/version: 2.12.0
    spec:
      selector:
        matchLabels:
          name: openebs-provisioner
          openebs.io/component-name: openebs-provisioner
      replicas: 1
      strategy:
        type: Recreate
        rollingUpdate: null
      template:
        metadata:
          labels:
            name: openebs-provisioner
            openebs.io/component-name: openebs-provisioner
            openebs.io/version: 2.12.0
        spec:
          serviceAccountName: openebs-maya-operator
          containers:
          - name: openebs-provisioner
            imagePullPolicy: IfNotPresent
            image: openebs/openebs-k8s-provisioner:2.12.0
            env:
            # OPENEBS_IO_K8S_MASTER enables openebs provisioner to connect to K8s
            # based on this address. This is ignored if empty.
            # This is supported for openebs provisioner version 0.5.2 onwards
            #- name: OPENEBS_IO_K8S_MASTER
            #  value: "http://10.128.0.12:8080"
            # OPENEBS_IO_KUBE_CONFIG enables openebs provisioner to connect to K8s
            # based on this config. This is ignored if empty.
            # This is supported for openebs provisioner version 0.5.2 onwards
            #- name: OPENEBS_IO_KUBE_CONFIG
            #  value: "/home/ubuntu/.kube/config"
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            - name: OPENEBS_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            # OPENEBS_MAYA_SERVICE_NAME provides the maya-apiserver K8s service name,
            # that provisioner should forward the volume create/delete requests.
            # If not present, "maya-apiserver-service" will be used for lookup.
            # This is supported for openebs provisioner version 0.5.3-RC1 onwards
            #- name: OPENEBS_MAYA_SERVICE_NAME
            #  value: "maya-apiserver-apiservice"
            # LEADER_ELECTION_ENABLED is used to enable/disable leader election. By default
            # leader election is enabled.
            #- name: LEADER_ELECTION_ENABLED
            #  value: "true"
            # OPENEBS_IO_JIVA_PATCH_NODE_AFFINITY is used to enable/disable setting node affinity
            # to the jiva replica deployments. Default is `enabled`. The valid values are 
            # `enabled` and `disabled`.
            #- name: OPENEBS_IO_JIVA_PATCH_NODE_AFFINITY
            #  value: "enabled"
             # Process name used for matching is limited to the 15 characters
             # present in the pgrep output.
             # So fullname can't be used here with pgrep (>15 chars).A regular expression
             # that matches the entire command name has to specified.
             # Anchor `^` : matches any string that starts with `openebs-provis`
             # `.*`: matches any string that has `openebs-provis` followed by zero or more char
            livenessProbe:
              exec:
                command:
                - sh
                - -c
                - test `pgrep -c "^openebs-provisi.*"` = 1
              initialDelaySeconds: 30
              periodSeconds: 60
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: openebs-snapshot-operator
      namespace: openebs
      labels:
        name: openebs-snapshot-operator
        openebs.io/component-name: openebs-snapshot-operator
        openebs.io/version: 2.12.0
    spec:
      selector:
        matchLabels:
          name: openebs-snapshot-operator
          openebs.io/component-name: openebs-snapshot-operator
      replicas: 1
      strategy:
        type: Recreate
      template:
        metadata:
          labels:
            name: openebs-snapshot-operator
            openebs.io/component-name: openebs-snapshot-operator
            openebs.io/version: 2.12.0
        spec:
          serviceAccountName: openebs-maya-operator
          containers:
            - name: snapshot-controller
              image: openebs/snapshot-controller:2.12.0
              imagePullPolicy: IfNotPresent
              env:
              - name: OPENEBS_NAMESPACE
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.namespace
             # Process name used for matching is limited to the 15 characters
             # present in the pgrep output.
             # So fullname can't be used here with pgrep (>15 chars).A regular expression
             # that matches the entire command name has to specified.
             # Anchor `^` : matches any string that starts with `snapshot-contro`
             # `.*`: matches any string that has `snapshot-contro` followed by zero or more char
              livenessProbe:
                exec:
                  command:
                  - sh
                  - -c
                  - test `pgrep -c "^snapshot-contro.*"` = 1
                initialDelaySeconds: 30
                periodSeconds: 60
            # OPENEBS_MAYA_SERVICE_NAME provides the maya-apiserver K8s service name,
            # that snapshot controller should forward the snapshot create/delete requests.
            # If not present, "maya-apiserver-service" will be used for lookup.
            # This is supported for openebs provisioner version 0.5.3-RC1 onwards
            #- name: OPENEBS_MAYA_SERVICE_NAME
            #  value: "maya-apiserver-apiservice"
            - name: snapshot-provisioner
              image: openebs/snapshot-provisioner:2.12.0
              imagePullPolicy: IfNotPresent
              env:
              - name: OPENEBS_NAMESPACE
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.namespace
              # OPENEBS_MAYA_SERVICE_NAME provides the maya-apiserver K8s service name,
              # that snapshot provisioner  should forward the clone create/delete requests.
              # If not present, "maya-apiserver-service" will be used for lookup.
              # This is supported for openebs provisioner version 0.5.3-RC1 onwards
              #- name: OPENEBS_MAYA_SERVICE_NAME
              #  value: "maya-apiserver-apiservice"
              # LEADER_ELECTION_ENABLED is used to enable/disable leader election. By default
              # leader election is enabled.
              #- name: LEADER_ELECTION_ENABLED
              #  value: "true"
             # Process name used for matching is limited to the 15 characters
             # present in the pgrep output.
             # So fullname can't be used here with pgrep (>15 chars).A regular expression
             # that matches the entire command name has to specified.
             # Anchor `^` : matches any string that starts with `snapshot-provis`
             # `.*`: matches any string that has `snapshot-provis` followed by zero or more char
              livenessProbe:
                exec:
                  command:
                  - sh
                  - -c
                  - test `pgrep -c "^snapshot-provis.*"` = 1
                initialDelaySeconds: 30
                periodSeconds: 60
    ---
    apiVersion: apiextensions.k8s.io/v1
    kind: CustomResourceDefinition
    metadata:
      annotations:
        controller-gen.kubebuilder.io/version: v0.5.0
      creationTimestamp: null
      name: blockdevices.openebs.io
    spec:
      group: openebs.io
      names:
        kind: BlockDevice
        listKind: BlockDeviceList
        plural: blockdevices
        shortNames:
          - bd
        singular: blockdevice
      scope: Namespaced
      versions:
        - additionalPrinterColumns:
            - jsonPath: .spec.nodeAttributes.nodeName
              name: NodeName
              type: string
            - jsonPath: .spec.path
              name: Path
              priority: 1
              type: string
            - jsonPath: .spec.filesystem.fsType
              name: FSType
              priority: 1
              type: string
            - jsonPath: .spec.capacity.storage
              name: Size
              type: string
            - jsonPath: .status.claimState
              name: ClaimState
              type: string
            - jsonPath: .status.state
              name: Status
              type: string
            - jsonPath: .metadata.creationTimestamp
              name: Age
              type: date
          name: v1alpha1
          schema:
            openAPIV3Schema:
              description: BlockDevice is the Schema for the blockdevices API
              properties:
                apiVersion:
                  description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
                  type: string
                kind:
                  description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
                  type: string
                metadata:
                  type: object
                spec:
                  description: DeviceSpec defines the properties and runtime status of a BlockDevice
                  properties:
                    aggregateDevice:
                      description: AggregateDevice was intended to store the hierarchical information in cases of LVM. However this is currently not implemented and may need to be re-looked into for better design. To be deprecated
                      type: string
                    capacity:
                      description: Capacity
                      properties:
                        logicalSectorSize:
                          description: LogicalSectorSize is blockdevice logical-sector size in bytes
                          format: int32
                          type: integer
                        physicalSectorSize:
                          description: PhysicalSectorSize is blockdevice physical-Sector size in bytes
                          format: int32
                          type: integer
                        storage:
                          description: Storage is the blockdevice capacity in bytes
                          format: int64
                          type: integer
                      required:
                        - storage
                      type: object
                    claimRef:
                      description: ClaimRef is the reference to the BDC which has claimed this BD
                      properties:
                        apiVersion:
                          description: API version of the referent.
                          type: string
                        fieldPath:
                          description: 'If referring to a piece of an object instead of an entire object, this string should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. For example, if the object reference is to a container within a pod, this would take on a value like: "spec.containers{name}" (where "name" refers to the name of the container that triggered the event) or if no container name is specified "spec.containers[2]" (container with index 2 in this pod). This syntax is chosen only to have some well-defined way of referencing a part of an object. TODO: this design is not final and this field is subject to change in the future.'
                          type: string
                        kind:
                          description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
                          type: string
                        name:
                          description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
                          type: string
                        namespace:
                          description: 'Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/'
                          type: string
                        resourceVersion:
                          description: 'Specific resourceVersion to which this reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency'
                          type: string
                        uid:
                          description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids'
                          type: string
                      type: object
                    details:
                      description: Details contain static attributes of BD like model,serial, and so forth
                      properties:
                        compliance:
                          description: Compliance is standards/specifications version implemented by device firmware  such as SPC-1, SPC-2, etc
                          type: string
                        deviceType:
                          description: DeviceType represents the type of device like sparse, disk, partition, lvm, crypt
                          enum:
                            - disk
                            - partition
                            - sparse
                            - loop
                            - lvm
                            - crypt
                            - dm
                            - mpath
                          type: string
                        driveType:
                          description: DriveType is the type of backing drive, HDD/SSD
                          enum:
                            - HDD
                            - SSD
                            - Unknown
                            - ""
                          type: string
                        firmwareRevision:
                          description: FirmwareRevision is the disk firmware revision
                          type: string
                        hardwareSectorSize:
                          description: HardwareSectorSize is the hardware sector size in bytes
                          format: int32
                          type: integer
                        logicalBlockSize:
                          description: LogicalBlockSize is the logical block size in bytes reported by /sys/class/block/sda/queue/logical_block_size
                          format: int32
                          type: integer
                        model:
                          description: Model is model of disk
                          type: string
                        physicalBlockSize:
                          description: PhysicalBlockSize is the physical block size in bytes reported by /sys/class/block/sda/queue/physical_block_size
                          format: int32
                          type: integer
                        serial:
                          description: Serial is serial number of disk
                          type: string
                        vendor:
                          description: Vendor is vendor of disk
                          type: string
                      type: object
                    devlinks:
                      description: DevLinks contains soft links of a block device like /dev/by-id/... /dev/by-uuid/...
                      items:
                        description: DeviceDevLink holds the mapping between type and links like by-id type or by-path type link
                        properties:
                          kind:
                            description: Kind is the type of link like by-id or by-path.
                            enum:
                              - by-id
                              - by-path
                            type: string
                          links:
                            description: Links are the soft links
                            items:
                              type: string
                            type: array
                        type: object
                      type: array
                    filesystem:
                      description: FileSystem contains mountpoint and filesystem type
                      properties:
                        fsType:
                          description: Type represents the FileSystem type of the block device
                          type: string
                        mountPoint:
                          description: MountPoint represents the mountpoint of the block device.
                          type: string
                      type: object
                    nodeAttributes:
                      description: NodeAttributes has the details of the node on which BD is attached
                      properties:
                        nodeName:
                          description: NodeName is the name of the Kubernetes node resource on which the device is attached
                          type: string
                      type: object
                    parentDevice:
                      description: "ParentDevice was intended to store the UUID of the parent Block Device as is the case for partitioned block devices. \n For example: /dev/sda is the parent for /dev/sda1 To be deprecated"
                      type: string
                    partitioned:
                      description: Partitioned represents if BlockDevice has partitions or not (Yes/No) Currently always default to No. To be deprecated
                      enum:
                        - "Yes"
                        - "No"
                      type: string
                    path:
                      description: Path contain devpath (e.g. /dev/sdb)
                      type: string
                  required:
                    - capacity
                    - devlinks
                    - nodeAttributes
                    - path
                  type: object
                status:
                  description: DeviceStatus defines the observed state of BlockDevice
                  properties:
                    claimState:
                      description: ClaimState represents the claim state of the block device
                      enum:
                        - Claimed
                        - Unclaimed
                        - Released
                      type: string
                    state:
                      description: State is the current state of the blockdevice (Active/Inactive/Unknown)
                      enum:
                        - Active
                        - Inactive
                        - Unknown
                      type: string
                  required:
                    - claimState
                    - state
                  type: object
              type: object
          served: true
          storage: true
          subresources: {}
    status:
      acceptedNames:
        kind: ""
        plural: ""
      conditions: []
      storedVersions: []
    
    ---
    apiVersion: apiextensions.k8s.io/v1
    kind: CustomResourceDefinition
    metadata:
      annotations:
        controller-gen.kubebuilder.io/version: v0.5.0
      creationTimestamp: null
      name: blockdeviceclaims.openebs.io
    spec:
      group: openebs.io
      names:
        kind: BlockDeviceClaim
        listKind: BlockDeviceClaimList
        plural: blockdeviceclaims
        shortNames:
          - bdc
        singular: blockdeviceclaim
      scope: Namespaced
      versions:
        - additionalPrinterColumns:
            - jsonPath: .spec.blockDeviceName
              name: BlockDeviceName
              type: string
            - jsonPath: .status.phase
              name: Phase
              type: string
            - jsonPath: .metadata.creationTimestamp
              name: Age
              type: date
          name: v1alpha1
          schema:
            openAPIV3Schema:
              description: BlockDeviceClaim is the Schema for the blockdeviceclaims API
              properties:
                apiVersion:
                  description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
                  type: string
                kind:
                  description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
                  type: string
                metadata:
                  type: object
                spec:
                  description: DeviceClaimSpec defines the request details for a BlockDevice
                  properties:
                    blockDeviceName:
                      description: BlockDeviceName is the reference to the block-device backing this claim
                      type: string
                    blockDeviceNodeAttributes:
                      description: BlockDeviceNodeAttributes is the attributes on the node from which a BD should be selected for this claim. It can include nodename, failure domain etc.
                      properties:
                        hostName:
                          description: HostName represents the hostname of the Kubernetes node resource where the BD should be present
                          type: string
                        nodeName:
                          description: NodeName represents the name of the Kubernetes node resource where the BD should be present
                          type: string
                      type: object
                    deviceClaimDetails:
                      description: Details of the device to be claimed
                      properties:
                        allowPartition:
                          description: AllowPartition represents whether to claim a full block device or a device that is a partition
                          type: boolean
                        blockVolumeMode:
                          description: 'BlockVolumeMode represents whether to claim a device in Block mode or Filesystem mode. These are use cases of BlockVolumeMode: 1) Not specified: VolumeMode check will not be effective 2) VolumeModeBlock: BD should not have any filesystem or mountpoint 3) VolumeModeFileSystem: BD should have a filesystem and mountpoint. If DeviceFormat is    specified then the format should match with the FSType in BD'
                          type: string
                        formatType:
                          description: Format of the device required, eg:ext4, xfs
                          type: string
                      type: object
                    deviceType:
                      description: DeviceType represents the type of drive like SSD, HDD etc.,
                      nullable: true
                      type: string
                    hostName:
                      description: Node name from where blockdevice has to be claimed. To be deprecated. Use NodeAttributes.HostName instead
                      type: string
                    resources:
                      description: Resources will help with placing claims on Capacity, IOPS
                      properties:
                        requests:
                          additionalProperties:
                            anyOf:
                              - type: integer
                              - type: string
                            pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                            x-kubernetes-int-or-string: true
                          description: 'Requests describes the minimum resources required. eg: if storage resource of 10G is requested minimum capacity of 10G should be available TODO for validating'
                          type: object
                      required:
                        - requests
                      type: object
                    selector:
                      description: Selector is used to find block devices to be considered for claiming
                      properties:
                        matchExpressions:
                          description: matchExpressions is a list of label selector requirements. The requirements are ANDed.
                          items:
                            description: A label selector requirement is a selector that contains values, a key, and an operator that relates the key and values.
                            properties:
                              key:
                                description: key is the label key that the selector applies to.
                                type: string
                              operator:
                                description: operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
                                type: string
                              values:
                                description: values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
                                items:
                                  type: string
                                type: array
                            required:
                              - key
                              - operator
                            type: object
                          type: array
                        matchLabels:
                          additionalProperties:
                            type: string
                          description: matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
                          type: object
                      type: object
                  type: object
                status:
                  description: DeviceClaimStatus defines the observed state of BlockDeviceClaim
                  properties:
                    phase:
                      description: Phase represents the current phase of the claim
                      type: string
                  required:
                    - phase
                  type: object
              type: object
          served: true
          storage: true
          subresources: {}
    status:
      acceptedNames:
        kind: ""
        plural: ""
      conditions: []
      storedVersions: []
    ---
    # This is the node-disk-manager related config.
    # It can be used to customize the disks probes and filters
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: openebs-ndm-config
      namespace: openebs
      labels:
        openebs.io/component-name: ndm-config
    data:
      # udev-probe is default or primary probe it should be enabled to run ndm
      # filterconfigs contains configs of filters. To provide a group of include
      # and exclude values add it as , separated string
      node-disk-manager.config: |
        probeconfigs:
          - key: udev-probe
            name: udev probe
            state: true
          - key: seachest-probe
            name: seachest probe
            state: false
          - key: smart-probe
            name: smart probe
            state: true
        filterconfigs:
          - key: os-disk-exclude-filter
            name: os disk exclude filter
            state: true
            exclude: "/,/etc/hosts,/boot"
          - key: vendor-filter
            name: vendor filter
            state: true
            include: ""
            exclude: "CLOUDBYT,OpenEBS"
          - key: path-filter
            name: path filter
            state: true
            include: ""
            exclude: "/dev/loop,/dev/fd0,/dev/sr0,/dev/ram,/dev/md,/dev/dm-,/dev/rbd,/dev/zd"
    ---
    apiVersion: apps/v1
    kind: DaemonSet
    metadata:
      name: openebs-ndm
      namespace: openebs
      labels:
        name: openebs-ndm
        openebs.io/component-name: ndm
        openebs.io/version: 2.12.0
    spec:
      selector:
        matchLabels:
          name: openebs-ndm
          openebs.io/component-name: ndm
      updateStrategy:
        type: RollingUpdate
      template:
        metadata:
          labels:
            name: openebs-ndm
            openebs.io/component-name: ndm
            openebs.io/version: 2.12.0
        spec:
          # By default the node-disk-manager will be run on all kubernetes nodes
          # If you would like to limit this to only some nodes, say the nodes
          # that have storage attached, you could label those node and use
          # nodeSelector.
          #
          # e.g. label the storage nodes with - "openebs.io/nodegroup"="storage-node"
          # kubectl label node <node-name> "openebs.io/nodegroup"="storage-node"
          #nodeSelector:
          #  "openebs.io/nodegroup": "storage-node"
          serviceAccountName: openebs-maya-operator
          hostNetwork: true
          # host PID is used to check status of iSCSI Service when the NDM
          # API service is enabled
          #hostPID: true
          containers:
          - name: node-disk-manager
            image: openebs/node-disk-manager:1.6.1
            args:
              - -v=4
              # The feature-gate is used to enable the new UUID algorithm.
              - --feature-gates="GPTBasedUUID"
            # Detect mount point and filesystem changes wihtout restart.
            # Uncomment the line below to enable the feature.
            # --feature-gates="MountChangeDetection"
            # The feature gate is used to start the gRPC API service. The gRPC server
            # starts at 9115 port by default. This feature is currently in Alpha state
            # - --feature-gates="APIService"
            # The feature gate is used to enable NDM, to create blockdevice resources
            # for unused partitions on the OS disk
            # - --feature-gates="UseOSDisk"
            imagePullPolicy: IfNotPresent
            securityContext:
              privileged: true
            volumeMounts:
            - name: config
              mountPath: /host/node-disk-manager.config
              subPath: node-disk-manager.config
              readOnly: true
              # make udev database available inside container
            - name: udev
              mountPath: /run/udev
            - name: procmount
              mountPath: /host/proc
              readOnly: true
            - name: devmount
              mountPath: /dev
            - name: basepath
              mountPath: /var/openebs/ndm
            - name: sparsepath
              mountPath: /var/openebs/sparse
            env:
            # namespace in which NDM is installed will be passed to NDM Daemonset
            # as environment variable
            - name: NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            # pass hostname as env variable using downward API to the NDM container
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            # specify the directory where the sparse files need to be created.
            # if not specified, then sparse files will not be created.
            - name: SPARSE_FILE_DIR
              value: "/var/openebs/sparse"
            # Size(bytes) of the sparse file to be created.
            - name: SPARSE_FILE_SIZE
              value: "10737418240"
            # Specify the number of sparse files to be created
            - name: SPARSE_FILE_COUNT
              value: "0"
            livenessProbe:
              exec:
                command:
                - pgrep
                - "ndm"
              initialDelaySeconds: 30
              periodSeconds: 60
          volumes:
          - name: config
            configMap:
              name: openebs-ndm-config
          - name: udev
            hostPath:
              path: /run/udev
              type: Directory
          # mount /proc (to access mount file of process 1 of host) inside container
          # to read mount-point of disks and partitions
          - name: procmount
            hostPath:
              path: /proc
              type: Directory
          - name: devmount
          # the /dev directory is mounted so that we have access to the devices that
          # are connected at runtime of the pod.
            hostPath:
              path: /dev
              type: Directory
          - name: basepath
            hostPath:
              path: /var/openebs/ndm
              type: DirectoryOrCreate
          - name: sparsepath
            hostPath:
              path: /var/openebs/sparse
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: openebs-ndm-operator
      namespace: openebs
      labels:
        name: openebs-ndm-operator
        openebs.io/component-name: ndm-operator
        openebs.io/version: 2.12.0
    spec:
      selector:
        matchLabels:
          name: openebs-ndm-operator
          openebs.io/component-name: ndm-operator
      replicas: 1
      strategy:
        type: Recreate
      template:
        metadata:
          labels:
            name: openebs-ndm-operator
            openebs.io/component-name: ndm-operator
            openebs.io/version: 2.12.0
        spec:
          serviceAccountName: openebs-maya-operator
          containers:
            - name: node-disk-operator
              image: openebs/node-disk-operator:1.6.1
              imagePullPolicy: IfNotPresent
              env:
                - name: WATCH_NAMESPACE
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.namespace
                - name: POD_NAME
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.name
                # the service account of the ndm-operator pod
                - name: SERVICE_ACCOUNT
                  valueFrom:
                    fieldRef:
                      fieldPath: spec.serviceAccountName
                - name: OPERATOR_NAME
                  value: "node-disk-operator"
                - name: CLEANUP_JOB_IMAGE
                  value: "openebs/linux-utils:2.12.0"
                # OPENEBS_IO_IMAGE_PULL_SECRETS environment variable is used to pass the image pull secrets
                # to the cleanup pod launched by NDM operator
                #- name: OPENEBS_IO_IMAGE_PULL_SECRETS
                #  value: ""
              livenessProbe:
                httpGet:
                  path: /healthz
                  port: 8585
                initialDelaySeconds: 15
                periodSeconds: 20
              readinessProbe:
                httpGet:
                  path: /readyz
                  port: 8585
                initialDelaySeconds: 5
                periodSeconds: 10
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: openebs-admission-server
      namespace: openebs
      labels:
        app: admission-webhook
        openebs.io/component-name: admission-webhook
        openebs.io/version: 2.12.0
    spec:
      replicas: 1
      strategy:
        type: Recreate
        rollingUpdate: null
      selector:
        matchLabels:
          app: admission-webhook
      template:
        metadata:
          labels:
            app: admission-webhook
            openebs.io/component-name: admission-webhook
            openebs.io/version: 2.12.0
        spec:
          serviceAccountName: openebs-maya-operator
          containers:
            - name: admission-webhook
              image: openebs/admission-server:2.12.0
              imagePullPolicy: IfNotPresent
              args:
                - -alsologtostderr
                - -v=2
                - 2>&1
              env:
                - name: OPENEBS_NAMESPACE
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.namespace
                - name: ADMISSION_WEBHOOK_NAME
                  value: "openebs-admission-server"
                - name: ADMISSION_WEBHOOK_FAILURE_POLICY
                  value: "Fail"
              # Process name used for matching is limited to the 15 characters
              # present in the pgrep output.
              # So fullname can't be used here with pgrep (>15 chars).A regular expression
              # Anchor `^` : matches any string that starts with `admission-serve`
              # `.*`: matche any string that has `admission-serve` followed by zero or more char
              # that matches the entire command name has to specified.
              livenessProbe:
                exec:
                  command:
                  - sh
                  - -c
                  - test `pgrep -c "^admission-serve.*"` = 1
                initialDelaySeconds: 30
                periodSeconds: 60
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: openebs-localpv-provisioner
      namespace: openebs
      labels:
        name: openebs-localpv-provisioner
        openebs.io/component-name: openebs-localpv-provisioner
        openebs.io/version: 2.12.0
    spec:
      selector:
        matchLabels:
          name: openebs-localpv-provisioner
          openebs.io/component-name: openebs-localpv-provisioner
      replicas: 1
      strategy:
        type: Recreate
      template:
        metadata:
          labels:
            name: openebs-localpv-provisioner
            openebs.io/component-name: openebs-localpv-provisioner
            openebs.io/version: 2.12.0
        spec:
          serviceAccountName: openebs-maya-operator
          containers:
          - name: openebs-provisioner-hostpath
            imagePullPolicy: IfNotPresent
            image: openebs/provisioner-localpv:2.12.0
            args:
              - "--bd-time-out=$(BDC_BD_BIND_RETRIES)"
            env:
            # OPENEBS_IO_K8S_MASTER enables openebs provisioner to connect to K8s
            # based on this address. This is ignored if empty.
            # This is supported for openebs provisioner version 0.5.2 onwards
            #- name: OPENEBS_IO_K8S_MASTER
            #  value: "http://10.128.0.12:8080"
            # OPENEBS_IO_KUBE_CONFIG enables openebs provisioner to connect to K8s
            # based on this config. This is ignored if empty.
            # This is supported for openebs provisioner version 0.5.2 onwards
            #- name: OPENEBS_IO_KUBE_CONFIG
            #  value: "/home/ubuntu/.kube/config"
            #  This sets the number of times the provisioner should try 
            #  with a polling interval of 5 seconds, to get the Blockdevice
            #  Name from a BlockDeviceClaim, before the BlockDeviceClaim
            #  is deleted. E.g. 12 * 5 seconds = 60 seconds timeout
            - name: BDC_BD_BIND_RETRIES
              value: "12"
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            - name: OPENEBS_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            # OPENEBS_SERVICE_ACCOUNT provides the service account of this pod as
            # environment variable
            - name: OPENEBS_SERVICE_ACCOUNT
              valueFrom:
                fieldRef:
                  fieldPath: spec.serviceAccountName
            - name: OPENEBS_IO_ENABLE_ANALYTICS
              value: "true"
            - name: OPENEBS_IO_INSTALLER_TYPE
              value: "openebs-operator"
            - name: OPENEBS_IO_HELPER_IMAGE
              value: "openebs/linux-utils:2.12.0"
            - name: OPENEBS_IO_BASE_PATH
              value: "/var/openebs/local"
            # LEADER_ELECTION_ENABLED is used to enable/disable leader election. By default
            # leader election is enabled.
            #- name: LEADER_ELECTION_ENABLED
            #  value: "true"
            # OPENEBS_IO_IMAGE_PULL_SECRETS environment variable is used to pass the image pull secrets
            # to the helper pod launched by local-pv hostpath provisioner
            #- name: OPENEBS_IO_IMAGE_PULL_SECRETS
            #  value: ""
            # Process name used for matching is limited to the 15 characters
            # present in the pgrep output.
            # So fullname can't be used here with pgrep (>15 chars).A regular expression
            # that matches the entire command name has to specified.
            # Anchor `^` : matches any string that starts with `provisioner-loc`
            # `.*`: matches any string that has `provisioner-loc` followed by zero or more char
            livenessProbe:
              exec:
                command:
                - sh
                - -c
                - test `pgrep -c "^provisioner-loc.*"` = 1
              initialDelaySeconds: 30
              periodSeconds: 60
    ---

应用新建的文件：

kubectl apply -f openebs-operator.yaml

查看OpenEBS的创建情况：

kubectl get pods --all-namespaces

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L1Jvb2tpZXhpYW9NdV9h_size_16_color_FFFFFF_t_70 1]  
看到状态为Running即可，若不是Running的话，等待几分钟即可…

> 注意：这里极有可能会pull镜像失败，可能有些镜像需要访问国外的网站【所以需要FQ】

1.  安装 OpenEBS 后将自动创建 4 个 StorageClass，查看创建的 StorageClass：

kubectl get sc

出现以下内容方为正确：

NAME                        PROVISIONER                                                AGE
    openebs-device              openebs.io/local                                           10h
    openebs-hostpath            openebs.io/local                                           10h
    openebs-jiva-default        openebs.io/provisioner-iscsi                               10h
    openebs-snapshot-promoter   volumesnapshot.external-storage.k8s.io/snapshot-promoter   10h

1.  如下将 openebs-hostpath设置为 默认的 StorageClass：

kubectl patch storageclass openebs-hostpath -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

出现以下内容方为正确：

storageclass.storage.k8s.io/openebs-hostpath patched

提示：由于在文档开头手动去掉了 master 节点的 Taint，我们可以在安装完 OpenEBS 和 KubeSphere 后，可以将 master 节点 Taint 加上，避免业务相关的工作负载调度到 master 节点抢占 master 资源：  
执行命令：

kubectl taint nodes k8s-master node-role.kubernetes.io/master=:NoSchedule

1.  安装kubesphere  
    执行以下命令以开始安装：

编辑文件第一个文件：kubesphere-installer.yaml

vi kubesphere-installer.yaml

输入以下内容：

---
    apiVersion: apiextensions.k8s.io/v1beta1
    kind: CustomResourceDefinition
    metadata:
      name: clusterconfigurations.installer.kubesphere.io
    spec:
      group: installer.kubesphere.io
      versions:
      - name: v1alpha1
        served: true
        storage: true
      scope: Namespaced
      names:
        plural: clusterconfigurations
        singular: clusterconfiguration
        kind: ClusterConfiguration
        shortNames:
        - cc
    
    ---
    apiVersion: v1
    kind: Namespace
    metadata:
      name: kubesphere-system
    
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: ks-installer
      namespace: kubesphere-system
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
      name: ks-installer
    rules:
    - apiGroups:
      - ""
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - apps
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - extensions
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - batch
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - rbac.authorization.k8s.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - apiregistration.k8s.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - apiextensions.k8s.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - tenant.kubesphere.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - certificates.k8s.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - devops.kubesphere.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - monitoring.coreos.com
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - logging.kubesphere.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - jaegertracing.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - storage.k8s.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - admissionregistration.k8s.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - policy
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - autoscaling
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - networking.istio.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - config.istio.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - iam.kubesphere.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - notification.kubesphere.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - auditing.kubesphere.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - events.kubesphere.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - core.kubefed.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - installer.kubesphere.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - storage.kubesphere.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - security.istio.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - monitoring.kiali.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - kiali.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - networking.k8s.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - kubeedge.kubesphere.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - types.kubefed.io
      resources:
      - '*'
      verbs:
      - '*'
    
    ---
    kind: ClusterRoleBinding
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      name: ks-installer
    subjects:
    - kind: ServiceAccount
      name: ks-installer
      namespace: kubesphere-system
    roleRef:
      kind: ClusterRole
      name: ks-installer
      apiGroup: rbac.authorization.k8s.io
    
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: ks-installer
      namespace: kubesphere-system
      labels:
        app: ks-install
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: ks-install
      template:
        metadata:
          labels:
            app: ks-install
        spec:
          serviceAccountName: ks-installer
          containers:
          - name: installer
            image: kubesphere/ks-installer:v3.1.1
            imagePullPolicy: "Always"
            resources:
              limits:
                cpu: "1"
                memory: 1Gi
              requests:
                cpu: 20m
                memory: 100Mi
            volumeMounts:
            - mountPath: /etc/localtime
              name: host-time
          volumes:
          - hostPath:
              path: /etc/localtime
              type: ""
            name: host-time

编辑第二个文件：cluster-configuration.yaml

vi cluster-configuration.yaml

输入以下内容：

---
    apiVersion: installer.kubesphere.io/v1alpha1
    kind: ClusterConfiguration
    metadata:
      name: ks-installer
      namespace: kubesphere-system
      labels:
        version: v3.1.1
    spec:
      persistence:
        storageClass: ""        # If there is no default StorageClass in your cluster, you need to specify an existing StorageClass here.
      authentication:
        jwtSecret: ""           # Keep the jwtSecret consistent with the Host Cluster. Retrieve the jwtSecret by executing "kubectl -n kubesphere-system get cm kubesphere-config -o yaml | grep -v "apiVersion" | grep jwtSecret" on the Host Cluster.
      local_registry: ""        # Add your private registry address if it is needed.
      etcd:
        monitoring: false       # Enable or disable etcd monitoring dashboard installation. You have to create a Secret for etcd before you enable it.
        endpointIps: localhost  # etcd cluster EndpointIps. It can be a bunch of IPs here.
        port: 2379              # etcd port.
        tlsEnable: true
      common:
        redis:
          enabled: false
        openldap:
          enabled: false
        minioVolumeSize: 20Gi # Minio PVC size.
        openldapVolumeSize: 2Gi   # openldap PVC size.
        redisVolumSize: 2Gi # Redis PVC size.
        monitoring:
          # type: external   # Whether to specify the external prometheus stack, and need to modify the endpoint at the next line.
          endpoint: http://prometheus-operated.kubesphere-monitoring-system.svc:9090 # Prometheus endpoint to get metrics data.
        es:   # Storage backend for logging, events and auditing.
          # elasticsearchMasterReplicas: 1   # The total number of master nodes. Even numbers are not allowed.
          # elasticsearchDataReplicas: 1     # The total number of data nodes.
          elasticsearchMasterVolumeSize: 4Gi   # The volume size of Elasticsearch master nodes.
          elasticsearchDataVolumeSize: 20Gi    # The volume size of Elasticsearch data nodes.
          logMaxAge: 7                     # Log retention time in built-in Elasticsearch. It is 7 days by default.
          elkPrefix: logstash              # The string making up index names. The index name will be formatted as ks-<elk_prefix>-log.
          basicAuth:
            enabled: false
            username: ""
            password: ""
          externalElasticsearchUrl: ""
          externalElasticsearchPort: ""
      console:
        enableMultiLogin: true  # Enable or disable simultaneous logins. It allows different users to log in with the same account at the same time.
        port: 30880
      alerting:                # (CPU: 0.1 Core, Memory: 100 MiB) It enables users to customize alerting policies to send messages to receivers in time with different time intervals and alerting levels to choose from.
        enabled: false         # Enable or disable the KubeSphere Alerting System.
        # thanosruler:
        #   replicas: 1
        #   resources: {}
      auditing:                # Provide a security-relevant chronological set of records，recording the sequence of activities happening on the platform, initiated by different tenants.
        enabled: false         # Enable or disable the KubeSphere Auditing Log System. 
      devops:                  # (CPU: 0.47 Core, Memory: 8.6 G) Provide an out-of-the-box CI/CD system based on Jenkins, and automated workflow tools including Source-to-Image & Binary-to-Image.
        enabled: false             # Enable or disable the KubeSphere DevOps System.
        jenkinsMemoryLim: 2Gi      # Jenkins memory limit.
        jenkinsMemoryReq: 1500Mi   # Jenkins memory request.
        jenkinsVolumeSize: 8Gi     # Jenkins volume size.
        jenkinsJavaOpts_Xms: 512m  # The following three fields are JVM parameters.
        jenkinsJavaOpts_Xmx: 512m
        jenkinsJavaOpts_MaxRAM: 2g
      events:                  # Provide a graphical web console for Kubernetes Events exporting, filtering and alerting in multi-tenant Kubernetes clusters.
        enabled: false         # Enable or disable the KubeSphere Events System.
        ruler:
          enabled: true
          replicas: 2
      logging:                 # (CPU: 57 m, Memory: 2.76 G) Flexible logging functions are provided for log query, collection and management in a unified console. Additional log collectors can be added, such as Elasticsearch, Kafka and Fluentd.
        enabled: false         # Enable or disable the KubeSphere Logging System.
        logsidecar:
          enabled: true
          replicas: 2
      metrics_server:                    # (CPU: 56 m, Memory: 44.35 MiB) It enables HPA (Horizontal Pod Autoscaler).
        enabled: false                   # Enable or disable metrics-server.
      monitoring:
        storageClass: ""                 # If there is an independent StorageClass you need for Prometheus, you can specify it here. The default StorageClass is used by default.
        # prometheusReplicas: 1          # Prometheus replicas are responsible for monitoring different segments of data source and providing high availability.
        prometheusMemoryRequest: 400Mi   # Prometheus request memory.
        prometheusVolumeSize: 20Gi       # Prometheus PVC size.
        # alertmanagerReplicas: 1          # AlertManager Replicas.
      multicluster:
        clusterRole: none  # host | member | none  # You can install a solo cluster, or specify it as the Host or Member Cluster.
      network:
        networkpolicy: # Network policies allow network isolation within the same cluster, which means firewalls can be set up between certain instances (Pods).
          # Make sure that the CNI network plugin used by the cluster supports NetworkPolicy. There are a number of CNI network plugins that support NetworkPolicy, including Calico, Cilium, Kube-router, Romana and Weave Net.
          enabled: false # Enable or disable network policies.
        ippool: # Use Pod IP Pools to manage the Pod network address space. Pods to be created can be assigned IP addresses from a Pod IP Pool.
          type: none # Specify "calico" for this field if Calico is used as your CNI plugin. "none" means that Pod IP Pools are disabled.
        topology: # Use Service Topology to view Service-to-Service communication based on Weave Scope.
          type: none # Specify "weave-scope" for this field to enable Service Topology. "none" means that Service Topology is disabled.
      openpitrix: # An App Store that is accessible to all platform tenants. You can use it to manage apps across their entire lifecycle.
        store:
          enabled: false # Enable or disable the KubeSphere App Store.
      servicemesh:         # (0.3 Core, 300 MiB) Provide fine-grained traffic management, observability and tracing, and visualized traffic topology.
        enabled: false     # Base component (pilot). Enable or disable KubeSphere Service Mesh (Istio-based).
      kubeedge:          # Add edge nodes to your cluster and deploy workloads on edge nodes.
        enabled: false   # Enable or disable KubeEdge.
        cloudCore:
          nodeSelector: {"node-role.kubernetes.io/worker": ""}
          tolerations: []
          cloudhubPort: "10000"
          cloudhubQuicPort: "10001"
          cloudhubHttpsPort: "10002"
          cloudstreamPort: "10003"
          tunnelPort: "10004"
          cloudHub:
            advertiseAddress: # At least a public IP address or an IP address which can be accessed by edge nodes must be provided.
              - ""            # Note that once KubeEdge is enabled, CloudCore will malfunction if the address is not provided.
            nodeLimit: "100"
          service:
            cloudhubNodePort: "30000"
            cloudhubQuicNodePort: "30001"
            cloudhubHttpsNodePort: "30002"
            cloudstreamNodePort: "30003"
            tunnelNodePort: "30004"
        edgeWatcher:
          nodeSelector: {"node-role.kubernetes.io/worker": ""}
          tolerations: []
          edgeWatcherAgent:
            nodeSelector: {"node-role.kubernetes.io/worker": ""}
            tolerations: []

应用这两个配置文件：

kubectl apply -f kubesphere-installer.yaml

kubectl apply -f cluster-configuration.yaml

> 若是嫌麻烦，可以使用以下两个命令代替，只不过这样可能导致文件获取不到，因为对网络需求比较高  
> kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.1.1/kubesphere-installer.yaml
> 
> kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.1.1/cluster-configuration.yaml

检查安装日志：

kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f

出现以下内容表示安装成功：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_Q1NETiBAeGlhb211X2E_size_31_color_FFFFFF_t_70_g_se_x_16]  
输入网址：http://192.168.12.135  
使用  
账号：admin  
密码：P@88w0rd  
登录！  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_Q1NETiBAeGlhb211X2E_size_67_color_FFFFFF_t_70_g_se_x_16]

## 安装OpenEBS踩坑记录 ##

小编在安装OpenEBS时，发现OpenEBS容器一直在急停、重启：

[root@localhost ~]# kubectl get pods --all-namespaces
    NAMESPACES		NAME                     				  READY   STATUS             RESTARTS   AGE
    openebs			openebs-ds1c1sc16s1c6s                    0/1     CrashLoopBackOff   27         3h33m
    openebs			openebs-48ew16v156db1d                    0/1     Completed          0          36s

可以看到status这一栏是：CrashLoopBackOff

1.查看此状态pod详细情况

kubectl describe pod openebs-ds1c1sc16s1c6s -n openens

> 命令解析：  
> openebs-ds1c1sc16s1c6s：pod名称  
> \-n：指定命名空间的pod  
> openens：命名空间的名称

2.查看此pod日志

kubectl logs openebs-ds1c1sc16s1c6s -n openens

可以看到错误信息：

kubectl logs openebs-admission-server-5d87f9f5d7-6w98r -n openebs                                          
    F0822 14:27:42.636833       1 main.go:88] failed to list admission deployment: Get "https://192.96.0.1:443/apis/apps/v1/namespaces/openebs/deployments?labelSelector=openebs.io%2Fcomponent-name%3Dadmission-webhook": dial tcp 192.96.0.1:443: i/o timeoutfailed to get a reference to the admission deployment object

大概意思是：连接192.96.0.1:443超时，于是我在每个节点都ping 192.96.0.1，但是可以ping通

啊啊啊啊啊，我要抓狂了…

转念一想，pod网络不通？flannel网络插件不就是保证pod、service之间的网络畅通的吗…于是我打开引用的flannel的配置文件，可以看到以下内容：

net-conf.json: |
        {
          "Network": "10.244.0.0/16",
          "Backend": {
            "Type": "vxlan"
          }
        }

在联想，最开始在构建master节点的时候，指定了service和pod的网段：  
![在这里插入图片描述][b53f08e7e78d46cc96309a916c3d3e72.png]

于是乎，我先删除之前的flannel引用配置：

kubectl delete -f kube-flannel.yml

将flannel的那一段配置改为：

net-conf.json: |
        {
          "Network": "192.244.0.0/16",
          "Backend": {
            "Type": "vxlan"
          }
        }

最后引用kube-flannel.yml配置文件：

kubectl apply -f kube-flannel.yml

然后重新查看pod的情况：

kubectl get pods --all-namespaces

可以看到就一切正常了：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L1Jvb2tpZXhpYW9NdV9h_size_16_color_FFFFFF_t_70 2]

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L1Jvb2tpZXhpYW9NdV9h_size_16_color_FFFFFF_t_70]: /images/20220829/300eaefb7da24f478ab24d03b7173c21.png
[c2cce4987d9c4e0eb19c4fc441088b7f.png]: /images/20220829/c6e54b7ee98f4c8bacd6f3cb593e9fa2.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L1Jvb2tpZXhpYW9NdV9h_size_16_color_FFFFFF_t_70 1]: /images/20220829/9d8f86d8ad6e4e6ab6514d8da978c3ec.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_Q1NETiBAeGlhb211X2E_size_31_color_FFFFFF_t_70_g_se_x_16]: /images/20220829/e0f379c2ce8a455dba53ac9c7b433d6b.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_Q1NETiBAeGlhb211X2E_size_67_color_FFFFFF_t_70_g_se_x_16]: /images/20220829/6da226c4ec4640c8a3da23975b9f9e85.png
[b53f08e7e78d46cc96309a916c3d3e72.png]: /images/20220829/497c91971d1a45b78120b60734a6d537.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L1Jvb2tpZXhpYW9NdV9h_size_16_color_FFFFFF_t_70 2]: /images/20220829/f4f72ae5201e457c9ebc1892ecb2f6b4.png