Containerd容器镜像管理

悠悠 2023-10-14 14:39 18阅读 0赞

[1. 轻量级容器管理工具 Containerd][1. _ Containerd]  
[2. Containerd的两种安装方式][2. Containerd]  
[3. Containerd容器镜像管理][3. Containerd]  
[4. Containerd数据持久化和网络管理][4. Containerd]

## 1、Containerd镜像管理 ##

### 1.1 Containerd容器镜像管理命令 ###

*  docker使用`docker images`命令管理镜像
 *  单机containerd使用`ctr images`命令管理镜像,containerd本身的CLI
 *  k8s中containerd使用`crictl images`命令管理镜像,Kubernetes社区的专用CLI工具

获取命令帮助
    # ctr --help
    NAME:
       ctr -
            __
      _____/ /______
     / ___/ __/ ___/
    / /__/ /_/ /
    \___/\__/_/
    
    ......

获取命令帮助
    # ctr images
    NAME:
       ctr images - manage images
    
    USAGE:
       ctr images command [command options] [arguments...]
    
    COMMANDS:
       check                    check existing images to ensure all content is available locally
       export                   export images
       import                   import images
       list, ls                 list images known to containerd
       mount                    mount an image to a target path
       unmount                  unmount the image from the target
       pull                     pull an image from a remote
       push                     push an image to a remote
       delete, del, remove, rm  remove one or more images by reference
       tag                      tag an image
       label                    set and clear labels for an image
       convert                  convert an image
    
    OPTIONS:
       --help, -h  show help

### 1.2 查看镜像 ###

# ctr images ls
    REF TYPE DIGEST SIZE PLATFORMS LABELS

### 1.3 下载镜像 ###

containerd支持oci标准的镜像，所以可以直接使用docker官方或dockerfile构建的镜像

# 镜像名不能简写，--all-platforms：所有平台，不加的话下载当前平台架构
    # ctr images pull --all-platforms docker.io/library/nginx:alpine
    # ctr images pull docker.io/library/nginx:latest
    
    # uname -a

说明：
    这里ctr命令pull镜像时，不能直接把镜像名字写成`nginx:alpine`

查看已下载容器镜像
    # ctr images ls

<table> 
 <thead> 
  <tr> 
   <th>REF</th> 
   <th>TYPE</th> 
   <th>DIGEST</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>docker.io/library/nginx:alpine</td> 
   <td>application/vnd.docker.distribution.manifest.list.v2+json</td> 
   <td>sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3</td> 
  </tr> 
 </tbody> 
</table>

<table> 
 <thead> 
  <tr> 
   <th>SIZE</th> 
   <th>PLATFORMS</th> 
   <th>LABELS</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>9.7 MiB</td> 
   <td>linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/ppc64le,linux/s390x</td> 
   <td>-</td> 
  </tr> 
 </tbody> 
</table>

指定平台下载容器镜像
    # ctr images pull --platform linux/amd64 docker.io/library/nginx:alpine

没有镜像是无法运行容器的。

### 1.4 镜像挂载 ###

> 方便查看镜像中包含的内容。

把已下载的容器镜像挂载至当前文件系统
    # ctr images mount docker.io/library/nginx:alpine /mnt
    sha256:af2fcce448e2e4451a5f4796a9bf9cb5c9b5f88e0d6d10029cada42fb9d268ac
    /mnt
    [root@localhost ~]# ls /mnt
    bin  dev  docker-entrypoint.d  docker-entrypoint.sh  etc  home  lib  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var

卸载
    # umount /mnt

### 1.5 镜像导出 ###

把容器镜像导出
    # ctr i export --all-platforms nginx.img docker.io/library/nginx:alpine

说明
    --all-platforms,导出所有平台镜像，本版本为1.6版本，1.4版本不需要添加此选项。

查看已导出容器镜像
    # ls
    nginx.img
    
    # ls -lh
    总用量 196M
    
    -rw-r--r--  1 root root  73M 2月  18 14:48 nginx.img

### 1.6 镜像删除 ###

删除指定容器镜像
    # ctr image rm docker.io/library/nginx:alpine
    docker.io/library/nginx:alpine
    
    再次查看容器镜像
    [root@192 ~]# ctr images ls
    REF TYPE DIGEST SIZE PLATFORMS LABELS

### 1.7 镜像导入 ###

导入容器镜像
    # ctr images import nginx.img
    unpacking docker.io/library/nginx:alpine (sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3)...done

### 1.8 修改镜像tag ###

# ctr images tag docker.io/library/nginx:alpine nginx:alpine
    nginx:alpine
    
    # 建议修改tag的名称为：镜像仓库地址/目录/镜像名

说明：
    把docker.io/library/nginx:alpine 修改为 nginx:alpine

查看修改后的容器镜像
    # ctr images ls
    REF                            TYPE                                                      DIGEST                                                                  SIZE    PLATFORMS                                                                                LABELS
    docker.io/library/nginx:alpine application/vnd.docker.distribution.manifest.list.v2+json sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3 9.7 MiB linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/ppc64le,linux/s390x -
    nginx:alpine                   application/vnd.docker.distribution.manifest.list.v2+json sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3 9.7 MiB linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/ppc64le,linux/s390x -

修改后对容器镜像做检查比对
    # ctr images check
    REF                            TYPE                                                      DIGEST                                                                  STATUS         SIZE            UNPACKED
    docker.io/library/nginx:alpine application/vnd.docker.distribution.manifest.list.v2+json sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3 complete (7/7) 9.7 MiB/9.7 MiB true
    
    nginx:alpine                   application/vnd.docker.distribution.manifest.list.v2+json sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3 complete (7/7) 9.7 MiB/9.7 MiB true

## 2、Containerd容器管理 ##

### 2.1 获取命令帮助 ###

#### 2.1.1 获取ctr命令帮助 ####

[root@localhost ~]# ctr --help
    NAME:
       ctr -
            __
      _____/ /______
     / ___/ __/ ___/
    / /__/ /_/ /
    \___/\__/_/
    
    containerd CLI
    
    
    ......

#### 2.1.2 获取创建静态容器命令帮助 ####

# ctr container --help
    NAME:
       ctr containers - manage containers
    
    USAGE:
       ctr containers command [command options] [arguments...]
    
    COMMANDS:
       create                   create container
       delete, del, remove, rm  delete one or more existing containers
       info                     get info about a container
       list, ls                 list containers
       label                    set and clear labels for a container
       checkpoint               checkpoint a container
       restore                  restore a container from checkpoint
    
    OPTIONS:
       --help, -h  show help

说明：
    
    使用`ctr container create `命令创建容器后，容器并没有处于运行状态，其只是一个静态的容器。这个 container 对象只是包含了运行一个容器所需的资源及配置的数据结构，例如： namespaces、rootfs 和容器的配置都已经初始化成功了，只是用户进程(本案例为nginx)还没有启动。需要使用`ctr tasks`命令才能获取一个动态容器。

#### 2.1.3 获取动态容器命令帮助 ####

# ctr run --help
    NAME:
       ctr run - run a container
    
    ......

说明：
    使用`ctr run`命令可以创建一个静态容器并使其运行。一步到位运行容器。

### 2.2 查看容器 ###

`container`表示静态容器，可用`c`缩写代表container

# ctr container ls
    CONTAINER    IMAGE    RUNTIME

或

# ctr c ls
    CONTAINER    IMAGE    RUNTIME

以上命令无法查看出来容器的静态的还是动态的

### 2.3 查看任务 ###

task表示容器里跑的进程, 可用`t`缩写代表`task`

# ctr task ls
    TASK    PID    STATUS

或

# ctr t ls
    TASK    PID    STATUS

### 2.4 创建静态容器 ###

# ctr c create docker.io/library/nginx:alpine nginx1

# ctr container ls
    CONTAINER    IMAGE                             RUNTIME
    nginx1       docker.io/library/nginx:alpine    io.containerd.runc.v2

查看容器详细信息
    # ctr container info nginx1

### 2.5 静态容器启动为动态容器 ###

复制containerd连接runC垫片工具至系统
    # ls usr/local/bin/
    containerd  containerd-shim  containerd-shim-runc-v1  containerd-shim-runc-v2  containerd-stress  crictl  critest  ctd-decoder  ctr
    [root@localhost ~]# cp usr/local/bin/containerd-shim-runc-v2 /usr/bin/

启动task，即表时在容器中运行了进程，即为动态容器。
    # ctr task start -d nginx1
    /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
    /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/

说明：
    -d表示daemon或者后台的意思，否则会卡住终端

查看容器所在宿主机进程，是以宿主机进程的方式存在的。
    # ctr task ls
    TASK      PID     STATUS
    nginx1    3395    RUNNING

查看容器的进程(都是物理机的进程)
    # ctr task ps nginx1
    PID     INFO
    3395    -
    3434    -

物理机查看到相应的进程
    # ps -ef | grep 3395
    root       3395   3375  0 19:16 ?        00:00:00 nginx: master process nginx -g daemon off;
    101        3434   3395  0 19:16 ?        00:00:00 nginx: worker process

### 2.6 进入容器操作 ###

# ctr task exec --exec-id 1 nginx1 /bin/sh
    
    # ifconfig 查看网卡信息
    
    ...
    
    # curl 127.0.0.1 访问本地提供的web服务
    ...

说明：
    为exec进程设定一个id，可以随意输入，只要保证唯一即可，也可使用$RANDOM变量。

### 2.7 直接运行一个动态容器 ###

# ctr run -d --net-host docker.io/library/nginx:alpine nginx2
    /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
    /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/

说明：
    * -d 代表dameon，后台运行
    * --net-host 代表容器的IP就是宿主机的IP(相当于docker里的host类型网络)

查看已运行容器
    # ctr container ls
    CONTAINER    IMAGE                             RUNTIME
    nginx2       docker.io/library/nginx:alpine    io.containerd.runc.v2

查看已运行容器中运行的进程,既tasks
    # ctr tasks ls
    TASK      PID     STATUS
    nginx2    4061    RUNNING

进入容器
    # ctr task exec --exec-id 1 -t nginx2 /bin/sh

/ # ifconfig 
    ens33     Link encap:Ethernet  HWaddr 00:0C:29:B1:B6:1D
              inet addr:192.168.10.164  Bcast:192.168.10.255  Mask:255.255.255.0
              inet6 addr: fe80::2b33:40ed:9311:8812/64 Scope:Link
              inet6 addr: fe80::adf4:a8bc:a1c:a9f7/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:55360 errors:0 dropped:0 overruns:0 frame:0
              TX packets:30526 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:53511295 (51.0 MiB)  TX bytes:2735050 (2.6 MiB)
    
    lo        Link encap:Local Loopback
              inet addr:127.0.0.1  Mask:255.0.0.0
              inet6 addr: ::1/128 Scope:Host
              UP LOOPBACK RUNNING  MTU:65536  Metric:1
              RX packets:68 errors:0 dropped:0 overruns:0 frame:0
              TX packets:68 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:5916 (5.7 KiB)  TX bytes:5916 (5.7 KiB)
    
    virbr0    Link encap:Ethernet  HWaddr 52:54:00:E9:51:82
              inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
              UP BROADCAST MULTICAST  MTU:1500  Metric:1
              RX packets:0 errors:0 dropped:0 overruns:0 frame:0
              TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

为容器中运行的网站添加网站文件
    / # echo "nginx2" > /usr/share/nginx/html/index.html
    / # exit

在宿主机上访问网站
    [root@localhost ~]# curl 192.168.10.164
    nginx2

### 2.8 暂停容器 ###

查看容器状态
    # ctr tasks ls
    TASK      PID     STATUS
    nginx2    4061    RUNNING

暂停容器
    # ctr tasks pause nginx2

再次查看容器状态，看到其状态为PAUSED，表示停止。
    # ctr tasks ls
    TASK      PID     STATUS
    nginx2    4061    PAUSED

[root@localhost ~]# curl http://192.168.10.164
    在宿主机访问，发现不可以访问到网站

### 2.9 恢复容器 ###

使用resume命令恢复容器
    # ctr tasks resume nginx2

查看恢复后状态
    # ctr tasks ls
    TASK      PID     STATUS
    nginx2    4061    RUNNING

在宿主机上访问容器中提供的服务
    # curl http://192.168.10.164
    nginx2

### 2.10 停止容器 ###

# ctr tasks --help
    ......

使用kill命令停止容器中运行的进程，既为停止容器
    # ctr tasks kill nginx2

查看容器停止后状态，STATUS为STOPPED
    # ctr tasks ls
    TASK      PID     STATUS
    nginx1    3395    RUNNING
    nginx2    4061    STOPPED

### 2.11 删除容器 ###

# ctr tasks delete nginx2
    必须先删除task，再删除容器

查看静态容器，确认其还存在于系统中
    # ctr container ls
    CONTAINER    IMAGE                             RUNTIME
    nginx2       docker.io/library/nginx:alpine    io.containerd.runc.v2

删除容器
    # ctr container delete nginx2

## 3、Containerd使用私有容器镜像仓库 Harbor ##

### 3.1 Harbor准备 ###

![在这里插入图片描述][0b873293fdbd4813af75e7778f5939a9.png]

![在这里插入图片描述][42f85ceb17ab43cbb165024e69fad611.png]

### 3.2 配置Containerd使用Harbor仓库 ###

#### 3.2.1 Harbor主机名解析 ####

> 在所有安装containerd宿主机上添加此配置信息。

# vim /etc/hosts
    # cat /etc/hosts
    127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
    ::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
    192.168.10.165 harbor.kubemsb.com

说明
    * 192.168.10.165是harbor的IP
    
    * harbor.kubemsb.com建议用FQDN形式，如果用类似harbor这种短名，后面下载镜像会出问题

#### 3.2.2 修改Containerd配置文件 ####

此配置文件已提前替换过，仅修改本地容器镜像仓库地址即可。
    # vim /etc/containerd/config.toml
    # cat /etc/containerd/config.toml
    root = "/var/lib/containerd"
    state = "/run/containerd"
    oom_score = -999
    
    [grpc]
      address = "/run/containerd/containerd.sock"
      uid = 0
      gid = 0
      max_recv_message_size = 16777216
      max_send_message_size = 16777216
    
    [debug]
      address = ""
      uid = 0
      gid = 0
      level = ""
    
    [metrics]
      address = ""
      grpc_histogram = false
    
    [cgroup]
      path = ""
    
    [plugins]
      [plugins.cgroups]
        no_prometheus = false
      [plugins.cri]
        stream_server_address = "127.0.0.1"
        stream_server_port = "0"
        enable_selinux = false
        sandbox_image = "easzlab/pause-amd64:3.2"
        stats_collect_period = 10
        systemd_cgroup = false
        enable_tls_streaming = false
        max_container_log_line_size = 16384
        [plugins.cri.containerd]
          snapshotter = "overlayfs"
          no_pivot = false
          [plugins.cri.containerd.default_runtime]
            runtime_type = "io.containerd.runtime.v1.linux"
            runtime_engine = ""
            runtime_root = ""
          [plugins.cri.containerd.untrusted_workload_runtime]
            runtime_type = ""
            runtime_engine = ""
            runtime_root = ""
        [plugins.cri.cni]
          bin_dir = "/opt/kube/bin"
          conf_dir = "/etc/cni/net.d"
          conf_template = "/etc/cni/net.d/10-default.conf"
        [plugins.cri.registry]
          [plugins.cri.registry.mirrors]
            [plugins.cri.registry.mirrors."docker.io"]
              endpoint = [
                "https://docker.mirrors.ustc.edu.cn",
                "http://hub-mirror.c.163.com"
              ]
            [plugins.cri.registry.mirrors."gcr.io"]
              endpoint = [
                "https://gcr.mirrors.ustc.edu.cn"
              ]
            [plugins.cri.registry.mirrors."k8s.gcr.io"]
              endpoint = [
                "https://gcr.mirrors.ustc.edu.cn/google-containers/"
              ]
            [plugins.cri.registry.mirrors."quay.io"]
              endpoint = [
                "https://quay.mirrors.ustc.edu.cn"
              ]
            [plugins.cri.registry.mirrors."harbor.kubemsb.com"]   在此处添加,在镜像加速器下面添加这一段
              endpoint = [
                "http://harbor.kubemsb.com"
              ]
        [plugins.cri.x509_key_pair_streaming]
          tls_cert_file = ""
          tls_key_file = ""
      [plugins.diff-service]
        default = ["walking"]
      [plugins.linux]
        shim = "containerd-shim"
        runtime = "runc"
        runtime_root = ""
        no_shim = false
        shim_debug = false
      [plugins.opt]
        path = "/opt/containerd"
      [plugins.restart]
        interval = "10s"
      [plugins.scheduler]
        pause_threshold = 0.02
        deletion_threshold = 0
        mutation_threshold = 100
        schedule_delay = "0s"
        startup_delay = "100ms"

重启containerd，以便于重新加载配置文件。
    # systemctl daemon-reload
    # systemctl restart containerd

#### 3.2.3 ctr下载镜像 ####

下载容器镜像
    # ctr images pull --platform linux/amd64 docker.io/library/nginx:latest

说明:
    * --platform linux/amd64 指定系统平台，也可以使用--all-platforms指定所有平台镜像。

查看已下载容器镜像
    # ctr images ls
    REF                              TYPE                                                      DIGEST                                                                  SIZE      PLATFORMS                                                                                                                          LABELS
    
    docker.io/library/nginx:latest   application/vnd.docker.distribution.manifest.list.v2+json sha256:2834dc507516af02784808c5f48b7cbe38b8ed5d0f4837f16e78d00deb7e7767 54.1 MiB  linux/386,linux/amd64,linux/arm/v5,linux/arm/v7,linux/arm64/v8,linux/mips64le,linux/ppc64le,linux/s390x                            -

#### 3.2.4 ctr上传镜像 ####

> 上传到Harbor library公有项目

重新生成新的tag
    # ctr images tag docker.io/library/nginx:latest harbor.kubemsb.com/library/nginx:latest
    harbor.kubemsb.com/library/nginx:latest

查看已生成容器镜像
    # ctr images ls
    REF                                     TYPE                                                      DIGEST                                                                  SIZE      PLATFORMS                                                                                                                          LABELS
    docker.io/library/nginx:latest          application/vnd.docker.distribution.manifest.list.v2+json sha256:2834dc507516af02784808c5f48b7cbe38b8ed5d0f4837f16e78d00deb7e7767 54.1 MiB  linux/386,linux/amd64,linux/arm/v5,linux/arm/v7,linux/arm64/v8,linux/mips64le,linux/ppc64le,linux/s390x                            -
    harbor.kubemsb.com/library/nginx:latest application/vnd.docker.distribution.manifest.list.v2+json sha256:2834dc507516af02784808c5f48b7cbe38b8ed5d0f4837f16e78d00deb7e7767 54.1 MiB  linux/386,linux/amd64,linux/arm/v5,linux/arm/v7,linux/arm64/v8,linux/mips64le,linux/ppc64le,linux/s390x                            -

推送容器镜像至Harbor
    # ctr images push --platform linux/amd64 --plain-http -u admin:密码 harbor.kubemsb.com/library/nginx:latest

说明:
    
    * 先tag再push
    * 因为我们harbor是http协议，不是https协议，所以需要加上`--plain-http`
    * `--user admin:Harbor12345`指定harbor的用户名与密码

输出：
    manifest-sha256:0fd68ec4b64b8dbb2bef1f1a5de9d47b658afd3635dc9c45bf0cbeac46e72101: done           |++++++++++++++++++++++++++++++++++++++|
    config-sha256:dd025cdfe837e1c6395365870a491cf16bae668218edb07d85c626928a60e478:   done           |++++++++++++++++++++++++++++++++++++++|
    elapsed: 0.5 s                                                                    total:  9.3 Ki (18.1 KiB/s)

![在这里插入图片描述][585800f6c4e74dfd87016ed481c63e48.png]

下载已上传容器镜像
    # ctr images pull --plain-http harbor.kubemsb.com/library/nginx:latest

[1. _ Containerd]: https://xy5462.blog.csdn.net/article/details/132135943?spm=1001.2014.3001.5502
[2. Containerd]: https://xy5462.blog.csdn.net/article/details/132135976?spm=1001.2014.3001.5502
[3. Containerd]: https://xy5462.blog.csdn.net/article/details/132136011?spm=1001.2014.3001.5502
[4. Containerd]: https://xy5462.blog.csdn.net/article/details/132136043?spm=1001.2014.3001.5502
[0b873293fdbd4813af75e7778f5939a9.png]: https://img-blog.csdnimg.cn/0b873293fdbd4813af75e7778f5939a9.png
[42f85ceb17ab43cbb165024e69fad611.png]: https://img-blog.csdnimg.cn/42f85ceb17ab43cbb165024e69fad611.png
[585800f6c4e74dfd87016ed481c63e48.png]: https://img-blog.csdnimg.cn/585800f6c4e74dfd87016ed481c63e48.png