dlib库标注特征点以及裁剪人脸组件

一时失言乱红尘 2024-05-24 01:02 81阅读 0赞

## 1、安装dlib ##

当在anaconda下使用命令 pip install dlib 安装dlib库时，提示错误安装不成功，可能是版本不匹配。在虚拟环境下查看Python版本，高版本不一定能和当前python版本兼容。可以将dlib wheel下载到本地，再手动安装。  
dlib库：链接：[https://pan.baidu.com/s/14FKO\_wiMG85s2kgo-1aEZg][https_pan.baidu.com_s_14FKO_wiMG85s2kgo-1aEZg] 提取码：hg5m

![在这里插入图片描述][20201222171028599.png_pic_center]

## 2、标注人脸特征点 ##

# _*_ coding:utf-8 _*_
    
    import numpy as np
    import cv2
    import dlib
    from PIL import Image
    import os
    import time
    
    detector = dlib.get_frontal_face_detector()
    predictor = dlib.shape_predictor('shape_predictor_68_face_landmarks.dat')
    # dictory_name = "E:\code\celeba-1024"
    dictory_name = './imgs'
    if not os.path.exists('./eye'):
        os.mkdir('./eye')
    if not os.path.exists('./nose'):
        os.mkdir('./nose')
    if not os.path.exists('./mouse'):
        os.mkdir('./mouse')
    # 摄像头捕捉人脸图像
    def cap_face():
        cap = cv2.VideoCapture(0)
    
        while (cap.isOpened()):
            ret, frame = cap.read()
            frame = cv2.resize(frame, (800, 650))
            start = time.time()
            frame = cv2.flip(frame, 1)
            dets = detector(frame, 0)
            for k, point in enumerate(dets):
                shape = predictor(frame, point)
                landmarks = np.matrix([[p.x, p.y] for p in shape.parts()])
                for num in range(shape.num_parts):
                    cv2.circle(frame, (shape.parts()[num].x, shape.parts()[num].y), 1, (0, 255, 0), -1)
            print("time ->", time.time() - start)
            cv2.imshow('frame', frame)
            if cv2.waitKey(1) & 0xFF == ord('q'):
                break
    
        cap.release()
        cv2.destroyAllWindows()
    def landmark_face():
        img = cv2.imread(dictory_name + "\\" + "000085.jpg.jpg")
        # img = cv2.resize(img[:, :, ::-1],dsize=(256, 256))
    
        # 取灰度
        # img_gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
        # 人脸数 rects
        rects = detector(img, 0)
        for k, point in enumerate(rects):
            shape = predictor(img, point)
            landmarks = np.matrix([[p.x, p.y] for p in shape.parts()])
            for num in range(shape.num_parts):
                cv2.circle(img, (shape.parts()[num].x, shape.parts()[num].y), 3, (0, 255, 0), -1)
        cv2.imshow('image', img)
        cv2.imwrite("./imgs/landmark_face.jpg",img)
        cv2.waitKey(0)
    
    def crop_face():
        for filename in os.listdir(dictory_name):
            # cv2读取图像
            img = cv2.imread(dictory_name + "\\" + filename)
            img = cv2.resize(img[:, :, ::-1], dsize=(256, 256))
    
            # 取灰度
            # img_gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
            # 人脸数 rects
            rects = detector(img, 0)
            for k, point in enumerate(rects):
                shape = predictor(img, point)
                landmarks = np.matrix([[p.x, p.y] for p in shape.parts()])
                for num in range(shape.num_parts):
                    cv2.circle(img, (shape.parts()[num].x, shape.parts()[num].y), 1, (0, 255, 0), -1)
            # cv2.imshow('image', img)
    
            # if len(rects) == 0:
    
            # 眼睛+眉毛
            # x_eye_left_up = 63       # 3  23
            # y_eye_left_up = 95      # 18  18
            # x_eye_right_down = 186   # 92  50
            # y_eye_right_down = 134   # 43  43
            #
            # eye_box = (x_eye_left_up, y_eye_left_up, x_eye_right_down, y_eye_right_down)
            # eye_crop = Image.fromarray(img).crop(eye_box)
            # eye_crop.save("./eye3" + "/" + filename)
    
            # 鼻子
            # x_nose_left_up = 98
            # y_nose_left_up = 130
            # x_nose_right_down = 155
            # y_nose_right_down = 173
            #
            # nose_box = (x_nose_left_up, y_nose_left_up, x_nose_right_down, y_nose_right_down)
            # nose_crop = Image.fromarray(img).crop(nose_box)
            # nose_crop.save("./nose" + "/" + filename)
    
            # 嘴巴
            # x_mouse_left_up = 90
            # y_mouse_left_up = 173
            # x_mouse_right_down = 160
            # y_mouse_right_down = 210
            #
            # mouse_box = (x_mouse_left_up, y_mouse_left_up, x_mouse_right_down, y_mouse_right_down)
            # mouse_crop = Image.fromarray(img).crop(mouse_box)
            # mouse_crop.save("./mouse" + "/" + filename)
    
            # else:
            #     for i in range(len(rects)): # 超过一个人脸时
            #         landmarks = np.matrix([[p.x, p.y] for p in predictor(img, rects[i]).parts()])
            #         # print(type(landmarks))  # numpy.matrix
            #
            #         # 眼睛+眉毛
            #         x_left_up = landmarks[17, 0]
            #         y_left_up = landmarks[18, 1] - (landmarks[29, 1] - landmarks[28, 1])*2
            #         x_right_down = landmarks[26, 0]
            #         y_right_down = landmarks[28, 1] + (landmarks[29, 1] - landmarks[28, 1])
            #
            #         # 嘴巴+鼻子
            #         # x_left_up = landmarks[4, 0] + (landmarks[48, 0] - landmarks[4, 0]) / 2
            #         # y_left_up = landmarks[29, 1]
            #         # x_right_down = landmarks[54, 0] + (landmarks[12, 0] - landmarks[54, 0]) / 2
            #         # y_right_down = landmarks[57, 1] + (landmarks[8, 1] - landmarks[57, 1]) / 2
            #
            #         # 眼睛+眉毛+鼻子
            #         # x_left_up = landmarks[17, 0]
            #         # y_left_up = landmarks[18, 1] - (landmarks[29, 1] - landmarks[28, 1])*2
            #         # x_right_down = landmarks[26, 0]
            #         # y_right_down = landmarks[33, 1] + (landmarks[51, 1] - landmarks[33, 1]) / 2
            #
            #         # 眼睛+眉毛+鼻子+嘴巴
            #         # x_left_up = landmarks[17, 0]
            #         # y_left_up = landmarks[18, 1] - (landmarks[29, 1] - landmarks[28, 1]) * 2
            #         # x_right_down = landmarks[54, 0] + (landmarks[12, 0] - landmarks[54, 0])
            #         # y_right_down = landmarks[57, 1] + (landmarks[8, 1] - landmarks[57, 1]) / 2
            #
            #         box = (x_left_up, y_left_up, x_right_down, y_right_down)
            #         image_crop = Image.fromarray(img).crop(box)
            #         image_crop.save("./eye3" + "/" + filename)
    
    if __name__ == '__main__':
        # cap_face()
        # crop_face()
        landmark_face()

调用landmark\_face()，标注图像结果如下：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2JhaWR1XzMzMjU2MTc0_size_16_color_FFFFFF_t_70_pic_center]

## 3、裁剪人脸组件 ##

crop\_face()方法：采用的固定大小的尺度去裁剪人脸组件（眼睛+眉毛，鼻子，嘴巴），使用PIL保存裁剪的结果，如下图所示：  
眼睛：  
![在这里插入图片描述][20201222205506201.jpg]  
![在这里插入图片描述][20201222205506207.jpg]  
![在这里插入图片描述][20201222205506202.jpg]  
鼻子：  
![在这里插入图片描述][20201222205536329.jpg]  
![在这里插入图片描述][20201222205536325.jpg]  
![在这里插入图片描述][20201222205536333.jpg]  
嘴巴：  
![在这里插入图片描述][20201222205558644.jpg]  
![在这里插入图片描述][20201222205558645.jpg]  
![在这里插入图片描述][20201222205558629.jpg]  
PIL库中的crop方法：采用两个像素点的坐标（x,y）去裁剪组件，两个点的位置分别为矩形框的左上角和右下角。注意x的方向为向右，y为向下，图像左上角的起始坐标为（0,0）。

## 4、后记 ##

思考：借助dlib库检测人脸的特征点来辅助裁剪人脸组件，存在哪些问题？

*  当人脸有姿态和遮挡时，采用固定size的尺度裁剪，无法精准裁剪。优点是裁剪后大小一致，以便网络训练，以及进一步处理。
 *  根据标注的特征点，利用其中的几个特征点来crop人脸组件，特征点中位置要进行加减一定的像素，但是裁剪后的图像大小不一致。当输入给神经网络时，需要resize处理，这样的裁剪好处是能将人脸组件精确裁剪下来。

由于作者水平有限，文中若有不正确的地方，欢迎大家指出，若有任何问题，请在下方讨论。

[https_pan.baidu.com_s_14FKO_wiMG85s2kgo-1aEZg]: https://pan.baidu.com/s/14FKO_wiMG85s2kgo-1aEZg
[20201222171028599.png_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/05/24/46fd446135d04148b1dad95cdffd1435.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2JhaWR1XzMzMjU2MTc0_size_16_color_FFFFFF_t_70_pic_center]: https://image.dandelioncloud.cn/pgy_files/images/2024/05/24/d9faf99e25b94aeca35b9e94ccd93639.jpg
[20201222205506201.jpg]: https://image.dandelioncloud.cn/pgy_files/images/2024/05/24/b625e5acf3fe42c980a070d3cf61658c.jpg
[20201222205506207.jpg]: https://image.dandelioncloud.cn/pgy_files/images/2024/05/24/8e052081caed4a14bab88b2932904c40.jpg
[20201222205506202.jpg]: https://image.dandelioncloud.cn/pgy_files/images/2024/05/24/65c59f3eedc3420092ed04548cbeb697.jpg
[20201222205536329.jpg]: https://image.dandelioncloud.cn/pgy_files/images/2024/05/24/8969781886e64def9e7c20e9cdb98934.jpg
[20201222205536325.jpg]: https://image.dandelioncloud.cn/pgy_files/images/2024/05/24/b21d695fd5344e998770d1a84031ed84.jpg
[20201222205536333.jpg]: https://image.dandelioncloud.cn/pgy_files/images/2024/05/24/3a52741945664098aa748b30a4589fd1.jpg
[20201222205558644.jpg]: https://image.dandelioncloud.cn/pgy_files/images/2024/05/24/8b768b5b7a9e44ff87bc55df40fb014b.jpg
[20201222205558645.jpg]: https://image.dandelioncloud.cn/pgy_files/images/2024/05/24/7c506581d704416085794f9cc79d5fe9.jpg
[20201222205558629.jpg]: https://image.dandelioncloud.cn/pgy_files/images/2024/05/24/2b72f3ff66034b3b80923c2be6d05040.jpg