ROOK

比眉伴天荒 2023-06-15 06:58 144阅读 0赞

## rook简介 ##

Rook是一个开放源码的云本机存储协调器，提供平台、框架和对各种存储解决方案的支持，以便与云本机环境进行本机集成。

Rook将存储软件转变为自我管理、自我扩展和自我修复的存储服务。它通过自动化部署、引导、配置、供应、扩展、升级、迁移、灾难恢复、监视和资源管理来实现这一点。Rook使用底层云本地容器管理、调度和协调平台提供的设施来执行其职责。

Rook利用扩展点深入集成到云本机环境中，为调度、生命周期管理、资源管理、安全、监控和用户体验提供无缝体验。

下图说明了Ceph Rook如何与Kubernetes集成：

![format_png][]

ROOK  架构

![format_png 1][]

## 部署环境准备 ##

需要K8S环境一套，节点分配如下.每台vm至少挂一块50G的盘

\[root@k8s-master ~\]\# cat /etc/hosts  
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4  
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6  
51.0.1.213 k8s-master  
51.0.1.214 k8s-node1  
51.0.1.215 k8s-node2

软件版本:

k8s 版本:   v1.14.1

ROOK版本:  v1.0  （Kubernetes v1.10 最低K8S要求）

开始部署:

github拉去ROOK仓库

\[root@k8s-master ~\]\# git clone https://github.com/rook/rook.git

\[root@k8s-master rook\]\# git checkout -b remotes/origin/release-1.0

进入ceph部署目录

\[root@k8s-master rook\]\# cd ./cluster/examples/kubernetes/ceph/

1 [root@k8s-master ceph]# kubectl create -f common.yaml
     2 namespace/rook-ceph created
     3 customresourcedefinition.apiextensions.k8s.io/cephclusters.ceph.rook.io created
     4 customresourcedefinition.apiextensions.k8s.io/cephfilesystems.ceph.rook.io created
     5 customresourcedefinition.apiextensions.k8s.io/cephnfses.ceph.rook.io created
     6 customresourcedefinition.apiextensions.k8s.io/cephobjectstores.ceph.rook.io created
     7 customresourcedefinition.apiextensions.k8s.io/cephobjectstoreusers.ceph.rook.io created
     8 customresourcedefinition.apiextensions.k8s.io/cephblockpools.ceph.rook.io created
     9 customresourcedefinition.apiextensions.k8s.io/volumes.rook.io created
    10 clusterrole.rbac.authorization.k8s.io/rook-ceph-cluster-mgmt created
    11 clusterrole.rbac.authorization.k8s.io/rook-ceph-cluster-mgmt-rules created
    12 role.rbac.authorization.k8s.io/rook-ceph-system created
    13 clusterrole.rbac.authorization.k8s.io/rook-ceph-global created
    14 clusterrole.rbac.authorization.k8s.io/rook-ceph-global-rules created
    15 clusterrole.rbac.authorization.k8s.io/rook-ceph-mgr-cluster created
    16 clusterrole.rbac.authorization.k8s.io/rook-ceph-mgr-cluster-rules created
    17 serviceaccount/rook-ceph-system created
    18 rolebinding.rbac.authorization.k8s.io/rook-ceph-system created
    19 clusterrolebinding.rbac.authorization.k8s.io/rook-ceph-global created
    20 serviceaccount/rook-ceph-osd created
    21 serviceaccount/rook-ceph-mgr created
    22 role.rbac.authorization.k8s.io/rook-ceph-osd created
    23 clusterrole.rbac.authorization.k8s.io/rook-ceph-mgr-system created
    24 clusterrole.rbac.authorization.k8s.io/rook-ceph-mgr-system-rules created
    25 role.rbac.authorization.k8s.io/rook-ceph-mgr created
    26 rolebinding.rbac.authorization.k8s.io/rook-ceph-cluster-mgmt created
    27 rolebinding.rbac.authorization.k8s.io/rook-ceph-osd created
    28 rolebinding.rbac.authorization.k8s.io/rook-ceph-mgr created
    29 rolebinding.rbac.authorization.k8s.io/rook-ceph-mgr-system created
    30 clusterrolebinding.rbac.authorization.k8s.io/rook-ceph-mgr-cluster created

创建operator 和agent容器

1 [root@k8s-master ceph]# kubectl create -f operator.yaml
    2 deployment.apps/rook-ceph-operator created

查看先关容器是否已经启动了，部署rook-ceph-operator过程中，会触发以DaemonSet的方式在集群部署Agent和Discoverpods。operator会在集群内的每个主机创建两个pod:rook-discover,rook-ceph-agent：

\[root@k8s-master ceph\]\# kubectl get pod -n rook-ceph -o wide  
![format_png 2][]

创建ceph进群

[root@k8s-master ceph]# kubectl create -f cluster.yaml
    cephcluster.ceph.rook.io/rook-ceph created

查看容器状态

\[root@k8s-master ceph\]\# kubectl get pod -n rook-ceph -o wide

![format_png 3][]

刚创建后需要等待一段时间才可以全部创建完，osd容器还是相对比较慢

查看下deployment 信息看看集群

\[root@k8s-master ~\]\# kubectl -n rook-ceph get deployment  
![format_png 4][]

配置dashboard

在cluster.yaml文件中默认已经启用了ceph dashboard，查看dashboard的service：

\[root@k8s-master ~\]\# kubectl get service -n rook-ceph|grep dashboard

NAME                                            TYPE                 CLUSTER-IP            EXTERNAL-IP      PORT(S)             AGE  
rook-ceph-mgr-dashboard        ClusterIP            10.105.208.128              <none>                8443/TCP              25h

rook-ceph-mgr-dashboard监听的端口是8443，创建nodeport类型的service以便集群外部访问

\[root@k8s-master ceph\]\# kubectl apply -f dashboard-external-https.yaml  
service/rook-ceph-mgr-dashboard-external-https created

查看 dashboard 外网端口  
\[root@k8s-master ceph\]\# kubectl get service -n rook-ceph | grep dashboard  
rook-ceph-mgr-dashboard                              ClusterIP     10.105.208.128        <none>           8443/TCP                15m  
rook-ceph-mgr-dashboard-external-https      NodePort     10.100.241.136         <none>          8443:32299/TCP      6s

获取Dashboard的登陆账号和密码

\[root@k8s-master ~\]\# MGR\_POD=\`kubectl get pod -n rook-ceph | grep mgr | awk '\{print $1\}'\`   
\[root@k8s-master ceph\]\# kubectl -n rook-ceph logs $MGR\_POD | grep password  
debug 2019-05-16 06:36:41.934 7fc2c1822700 0 log\_channel(audit) log \[DBG\] : from='client.14398 -' entity='client.admin' cmd=\[\{"username": "admin", "prefix": "dashboard set-login-credentials", "password": "RmXvxlnOf6", "target": \["mgr", ""\], "format": "json"\}\]: dispatch

找到password字段，用户 admin，密码   RmXvxlnOf6  
打开浏览器输入任意一个Node的IP+nodeport端口，这里使用master节点 ip访问：

![format_png 5][]

部署Ceph toolbox

默认启动的Ceph集群，是开启Ceph认证的，这样你登陆Ceph组件所在的Pod里，是没法去获取集群状态，

以及执行CLI命令，这时需要部署Ceph toolbox，命令如下：

\[root@k8s-master ceph\]\# kubectl apply -f toolbox.yaml  
deployment.apps/rook-ceph-tools created  
\[root@k8s-master ceph\]\# kubectl -n    rook-ceph get pods -o wide | grep ceph-tools  
rook-ceph-tools-b8c679f95-mf5xn        1/1 Running            0          12s            51.0.1.215           k8s-node2     <none>  <none>

登录容器运行命令

\[root@k8s-master ceph\]\#  kubectl -n rook-ceph exec -it rook-ceph-tools-b8c679f95-mf5xn bash

![format_png 6][]

转载于:https://www.cnblogs.com/menkeyi/p/10882251.html

[format_png]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9pbWcyMDE4LmNuYmxvZ3MuY29tL2Jsb2cvMTAwMzA3NC8yMDE5MDUvMTAwMzA3NC0yMDE5MDUxNjE2NDIzNzAyOS0xMTAzNjQxMzc5LnBuZw?x-oss-process=image/format,png
[format_png 1]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9pbWcyMDE4LmNuYmxvZ3MuY29tL2Jsb2cvMTAwMzA3NC8yMDE5MDUvMTAwMzA3NC0yMDE5MDUxNjE2Mzc0ODQ1OS01NTY5OTIwMjQucG5n?x-oss-process=image/format,png
[format_png 2]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9pbWcyMDE4LmNuYmxvZ3MuY29tL2Jsb2cvMTAwMzA3NC8yMDE5MDUvMTAwMzA3NC0yMDE5MDUxNzE2MTkyNTY1MS0xMTAzODU1NTA3LnBuZw?x-oss-process=image/format,png
[format_png 3]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9pbWcyMDE4LmNuYmxvZ3MuY29tL2Jsb2cvMTAwMzA3NC8yMDE5MDUvMTAwMzA3NC0yMDE5MDUxNzE2MjI0MjcwMy0xODMyNjY4MDY3LnBuZw?x-oss-process=image/format,png
[format_png 4]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9pbWcyMDE4LmNuYmxvZ3MuY29tL2Jsb2cvMTAwMzA3NC8yMDE5MDUvMTAwMzA3NC0yMDE5MDUxNzE2MjgxNDE2OS0xMDk4MDQxNTQ0LnBuZw?x-oss-process=image/format,png
[format_png 5]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9pbWcyMDE4LmNuYmxvZ3MuY29tL2Jsb2cvMTAwMzA3NC8yMDE5MDUvMTAwMzA3NC0yMDE5MDUxNzE2NDAwODA5Ny0xMjM3NjU4MjU0LnBuZw?x-oss-process=image/format,png
[format_png 6]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9pbWcyMDE4LmNuYmxvZ3MuY29tL2Jsb2cvMTAwMzA3NC8yMDE5MDUvMTAwMzA3NC0yMDE5MDUxNzE2NTYwMTEzMS0xNzA5MjM2ODExLnBuZw?x-oss-process=image/format,png