Compact and discriminative approach for encoding spatial-relationship of visual words

Abstract

The Bag-of-Visual-Words approach has been successfully used for video and image analysis by encoding local features as visual words, and the final representation is a histogram of the visual words detected in the image. One limitation of this approach relies on its inability of encoding spatial distribution of the visual words within an image, which is important for similarity measurement between images. In this paper, we present a novel technique to incorporate spatial information, called Global Spatial Arrangement (GSA). The idea is to split the image space into quadrants using each detected point as origin. To ensure rotation invariance, we use the information of the gradient of each detected point to define each quarter of the quadrant. The final representation uses only two extra information into the final feature vector to encode the spatial arrangement of visual words, with the advantage of being invariant to rotation. We performed representative experimental evaluations using several public datasets. Compared to other techniques, such as the Spatial Pyramid (SP), the proposed method needs 90% less information to encode spatial information of visual words. The results in image retrieval and classification demonstrated that our proposed approach improved the retrieval accuracy compared to other traditional techniques, while being the most compact descriptor.