Python+OpenCV：尺度不变特征变换（SIFT）的加速健壮特性（SURF）

曾经终败给现在 2022-12-23 15:26 194阅读 0赞

## Python+OpenCV：尺度不变特征变换（SIFT）的加速健壮特性（SURF） ##

### 理论 ###

In SIFT, Low approximated Laplacian of Gaussian with Difference of Gaussian for finding scale-space.

SURF goes a little further and approximates LoG with Box Filter.

Below image shows a demonstration of such an approximation.

One big advantage of this approximation is that, convolution with box filter can be easily calculated with the help of integral images.

And it can be done in parallel for different scales.

Also the SURF rely on determinant of Hessian matrix for both scale and location.

![20201125175130705.jpg][]

For orientation assignment, SURF uses wavelet responses in horizontal and vertical direction for a neighbourhood of size 6s.

Adequate gaussian weights are also applied to it.

Then they are plotted in a space as given in below image.

The dominant orientation is estimated by calculating the sum of all responses within a sliding orientation window of angle 60 degrees.

Interesting thing is that, wavelet response can be found out using integral images very easily at any scale.

For many applications, rotation invariance is not required, so no need of finding this orientation, which speeds up the process.

SURF provides such a functionality called Upright-SURF or U-SURF.

It improves speed and is robust upto ±15 degree.

OpenCV supports both, depending upon the flag, **upright**. If it is 0, orientation is calculated. If it is 1, orientation is not calculated and it is faster.

![20201125182036636.jpg][]

For feature description, SURF uses Wavelet responses in horizontal and vertical direction (again, use of integral images makes things easier).

A neighbourhood of size 20sX20s is taken around the keypoint where s is the size. It is divided into 4x4 subregions.

For each subregion, horizontal and vertical wavelet responses are taken and a vector is formed like this, ![20201125182206902.png][].

This when represented as a vector gives SURF feature descriptor with total 64 dimensions. Lower the dimension, higher the speed of computation and matching, but provide better distinctiveness of features.

For more distinctiveness, SURF feature descriptor has an extended 128 dimension version. The sums of dx and |dx| are computed separately for dy<0 and dy≥0.

Similarly, the sums of dy and |dy| are split up according to the sign of dx , thereby doubling the number of features. It doesn't add much computation complexity.

OpenCV supports both by setting the value of flag **extended** with 0 and 1 for 64-dim and 128-dim respectively (default is 128-dim).

Another important improvement is the use of sign of Laplacian (trace of Hessian Matrix) for underlying interest point.

It adds no computation cost since it is already computed during detection. The sign of the Laplacian distinguishes bright blobs on dark backgrounds from the reverse situation.

In the matching stage, we only compare features if they have the same type of contrast (as shown in image below).

This minimal information allows for faster matching, without reducing the descriptor's performance.

![20201125182451340.jpg][]

In short, SURF adds a lot of features to improve the speed in every step.

Analysis shows it is 3 times faster than SIFT while performance is comparable to SIFT.

SURF is good at handling images with blurring and rotation, but not good at handling viewpoint change and illumination change.

### SURF in OpenCV ###

####################################################################################################
    # 图像尺度不变特征变换加速鲁棒特征(SIFT SURF, Scale-Invariant Feature Transform Speeded-Up Robust Features)
    def lmc_cv_image_sift_surf_detection():
        """
            函数功能: 图像尺度不变特征变换加速鲁棒特征(SIFT SURF, Scale-Invariant Feature Transform Speeded-Up Robust Features)。
        """
    
        stacking_images = []
        image_file_name = ['D:/99-Research/Python/Image/Butterfly01.jpg',
                           'D:/99-Research/Python/Image/Butterfly02.jpg',
                           'D:/99-Research/Python/Image/Butterfly03.jpg',
                           'D:/99-Research/Python/Image/Butterfly04.jpg']
        for i in range(len(image_file_name)):
            # 读取图像
            image = lmc_cv.imread(image_file_name[i])
            image = lmc_cv.cvtColor(image, lmc_cv.COLOR_BGR2RGB)
            result_image = image.copy()
            gray_image = lmc_cv.cvtColor(image, lmc_cv.COLOR_BGR2GRAY)
    
            # Create SURF object. You can specify params here or later.
            # Here I set Hessian Threshold to 400
            surf = lmc_cv.xfeatures2d.SURF_create(400)
            # Find keypoints and descriptors directly
            keypoints, descriptors = surf.detectAndCompute(gray_image, None)
            print(f"关键点个数: {len(keypoints)}")
    
            # Check present Hessian threshold
            print(f"Hessian阈值: {surf.getHessianThreshold()}")
    
            # We set it to some 50000. Remember, it is just for representing in picture.
            # In actual cases, it is better to have a value 300-500.
            surf.setHessianThreshold(50000)
    
            # Again compute keypoints and check its number.
            keypoints, descriptors = surf.detectAndCompute(gray_image, None)
            print(f"关键点个数: {len(keypoints)}")
    
            # For Python API, flags are modified as:
            # cv.DRAW_MATCHES_FLAGS_DEFAULT,
            # cv.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS,
            # cv.DRAW_MATCHES_FLAGS_DRAW_OVER_OUTIMG,
            # cv.DRAW_MATCHES_FLAGS_NOT_DRAW_SINGLE_POINTS
            result_image = lmc_cv.drawKeypoints(gray_image, keypoints, None, color=(255, 0, 0),
                                                flags=lmc_cv.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
    
            # Check upright flag, if it False, set it to True
            print(f"upright 标志: {surf.getUpright()}")
            surf.setUpright(True)
    
            # Recompute the feature points and draw it
            keypoints = surf.detect(gray_image, None)
            result_image = lmc_cv.drawKeypoints(gray_image, keypoints, None, color=(255, 0, 0),
                                                flags=lmc_cv.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
    
            # Find size of descriptor
            print(f"描述子个数: {surf.descriptorSize()}")
    
            # That means flag, "extended" is False.
            print(f"扩展模式: {surf.getExtended()}")
    
            # So we make it to True to get 128-dim descriptors.
            surf.setExtended(True)
            keypoints, descriptors = surf.detectAndCompute(gray_image, None)
            print(f"描述子个数: {surf.descriptorSize()}")
            print(f"描述子形状: {descriptors.shape}")
    
            # stacking images side-by-side
            stacking_image = np.hstack((image, result_image))
            stacking_images.append(stacking_image)
    
        # 显示图像
        for i in range(len(stacking_images)):
            pyplot.figure('SURF Keypoints Detection %d' % (i + 1))
            pyplot.subplot(1, 1, 1)
            pyplot.imshow(stacking_images[i], 'gray')
            pyplot.title('SURF Keypoints Detection')
            pyplot.xticks([])
            pyplot.yticks([])
        pyplot.show()
    
        # 根据用户输入保存图像
        if ord("q") == (lmc_cv.waitKey(0) & 0xFF):
            # 销毁窗口
            pyplot.close('all')
        return

[20201125175130705.jpg]: /images/20221120/96ccfdbf17494841ad3cc8e50e0af129.png
[20201125182036636.jpg]: /images/20221120/801902c8ffef4511b007ad795027e6ce.png
[20201125182206902.png]: /images/20221120/f7240f5bd93e4f9396471cfbaa71cc28.png
[20201125182451340.jpg]: https://img-blog.csdnimg.cn/20201125182451340.jpg