VGG16-based Feature Fusion For Image Kyepoint Description

Abstract

The detection and description of feature points or keypoints play a vital role in numerous computer vision tasks. While deep learning-based methods have achieved remarkable results in various fields, they still require extensive training and resources to develop and optimize the network for specific tasks. In this work, we propose a novel method for extracting keypoint descriptors from natural images using a pre-trained VGG16 network that is specifically trained for classification tasks. Unlike previous works, our proposed approach does not require further training or modifications of the pre-trained network, which reduces the overall computational complexity and increases the efficiency of the method. The proposed method is divided into two stages: keypoint detection and description. In the keypoint detection stage, a traditional detector is employed to locate the image keypoints. In the keypoint description stage, a downscaled version of the input image is fed into the VGG16 network, and the first five convolutional layers are used to generate a rich descriptor by combining the descriptors at keypoint positions from various layers. By utilizing a sparse random projection technique, we decrease the size of the introduced descriptor, making it more efficient for keypoint matching. Our evaluation results demonstrate that our proposed method is highly competitive with traditional hand-crafted descriptors and outperforms some of them in terms of geometric transformation estimation. Additionally, our method is versatile and can be applied to different image domains by using task-specific pre-trained networks. Overall, our proposed approach offers an efficient and effective method for keypoint description with competitive performance to traditional methods and without requiring extensive training or modifications to the pre-trained network.