The main objective of Content-Based Image Retrieval (CBIR) systems is to retrieve a ranked list containing the most similar images of a collection given a query image, by taking into account their visual content. Although these systems represent a very promising approach, in many situations is very challenging to assure the quality of returned ranked lists. Supervised approaches rely on training data and information obtained from user interactions to identify and then improve low-quality results. However, these approaches require a lot of human efforts which can be infeasible for many systems. In this paper, we present two novel unsupervised measures for estimating the effectiveness of ranked lists in CBIR tasks. Given an estimation of the effectiveness of ranked lists, many CBIR systems can, for example, emulate the training process, but now without any user intervention. Improvements can also be achieved on several unsupervised approaches, such as re-ranking and rank aggregation methods, once the estimation measures can help to consider more relevant information by distinguishing effective from non-effective ranked lists. Both proposed measures are computed using a novel image representation of ranked lists and distances among images considering a given dataset. The objective is to exploit the visual patterns encoded in the image representations for estimating the effectiveness of ranked lists. Experiments involving shape, color, and texture descriptors demonstrate that the proposed approaches can provide accurate estimations of the quality in terms of effectiveness of ranked lists. The use of proposed measures are also evaluated in image retrieval tasks aiming at improving the effectiveness of rank aggregation approaches. © 2013 IEEE. 341348Datta, R., Joshi, D., Li, J., Wang, J.Z., Image retrieval: Ideas, influences, and trends of the new age (2008) ACM Computing Surveys, 40 (2), pp. 51-560Da S Torres, R., Falcao, A.X., Content-based image retrieval: Theory and applications (2006) Revista de Inforḿatica Téorica e Aplicada, 13 (2), pp. 161-185Ferreira, C.D., Dos Santos, J.A., Da S Torres, R., Gonçalves, M.A., Rezende, R.C., Fan, W., Relevance feedback based on genetic programming for image retrieval (2011) Pattern Recogninion Letters, 32 (1), pp. 27-37Dos Santos, J.A., Ferreira, C.D., Da S Torres, R., Gonçalves, M.A., Lamparelli, R.A., A relevance feedback method based on genetic programming for classification of remote sensing images (2011) Information Sciences, 181 (13), pp. 2671-2684Kontschieder, P., Donoser, M., Bischof, H., Beyond pairwise shape similarity analysis (2009) Asian Conference on Computer Vision, pp. 655-666Yang, X., Bai, X., Latecki, L.J., Tu, Z., Improving shape retrieval by learning graph transduction (2008) European Conference on Computer Vision (ECCV'2008), 4, pp. 788-801Jiang, J., Wang, B., Tu, Z., Unsupervised metric learning by selfsmoothing operator (2011) IEEE International Conference on Computer Vision (ICCV'2011), pp. 794-801Yang, X., Prasad, L., Latecki, L., Affinity learning with diffusion on tensor product graph (2013) Pattern Analysis and Machine Intelligence, IEEE Transactions on, 35 (1), pp. 28-38Yang, X., Latecki, L.J., Affinity learning on a tensor product graph with applications to shape and image retrieval (2011) IEEE Conference on Computer Vision and Pattern Recognition (CVPR'2011), pp. 2369-2376Pedronette, D.C.G., Da S Torres, R., Exploiting pairwise recommendation and clustering strategies for image re-ranking (2012) Information Sciences, 207 (1), pp. 19-34Shen, X., Lin, Z., Brandt, J., Avidan, S., Wu, Y., Object retrieval and localization with spatially-constrained similarity measure and knn re-ranking (2012) IEEE Conference on Computer Vision and Pattern Recognition (CVPR'2012), pp. 3013-3020Qin, D., Gammeter, S., Bossard, L., Quack, T., Van Gool, L., Hello neighbor: Accurate object retrieval with k-reciprocal nearest neighbors (2011) IEEE Conference on Computer Vision and Pattern Recognition (CVPR'2011), pp. 777-784. , juneSchwander, O., Nielsen, F., Reranking with contextual dissimilarity measures from representational bregmanl k-means (2010) International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP'2010), 1, pp. 118-122Pedronette, D.C.G., Da S Torres, R., Image re-ranking and rank aggregation based on similarity of ranked lists (2011) Computer Analysis of Images and Patterns (CAIP'2011), 6854, pp. 369-376Pedronette, D.C.G., Da S Torres, R., Exploiting contextual information for rank aggregation (2011) International Conference on Image Processing (ICIP'2011), pp. 97-100Pedronette, D.C.G., Da S Torres, R., Exploiting contextual information for image re-ranking and rank aggregation (2012) International Journal of Multimedia Information Retrieval, 1 (2), pp. 115-128Latecki, L.J., Lakmper, R., Eckhardt, U., Shape descriptors for non-rigid shapes with a single closed contour (2000) IEEE Conference on Computer Vision and Pattern Recognition (CVPR'2000), pp. 424-429Pedronette, D.C.G., Da S Torres, R., Shape retrieval using contour features and distance optmization (2010) International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP'2010), 1, pp. 197-202Da S Torres, R., Falcao, A.X., Contour salience descriptors for effective image retrieval and analysis (2007) Image and Vision Computing, 25 (1), pp. 3-13Arica, N., Vural, F.T.Y., BAS: A perceptual shape descriptor based on the beam angle statistics (2003) Pattern Recognition Letters, 24 (9-10), pp. 1627-1639Ling, H., Jacobs, D.W., Shape classification using the innerdistance (2007) IEEE Transactions on Pattern Analysis and Machine Intelligence, 29 (2), pp. 286-299Ling, H., Yang, X., Latecki, L.J., Balancing deformability and discriminability for shape matching (2010) European Conference on Computer Vision (ECCV'2010), 3, pp. 411-424Stehling, R.O., Nascimento, M.A., Falcao, A.X., A compact and efficient image retrieval approach based on border/interior pixel classification (2002) ACM Conference on Information and Knowledge Management (CIKM'2002), pp. 102-109Huang, J., Kumar, S.R., Mitra, M., Zhu, W.-J., Zabih, R., Image indexing using color correlograms (1997) IEEE Conference on Computer Vision and Pattern Recognition (CVPR'97), pp. 762-768Swain, M.J., Ballard, D.H., Color indexing (1991) International Journal on Computer Vision, 7 (1), pp. 11-32Weijer De J.Van, Schmid, C., Coloring local feature extraction (2006) European Conference on Computer Vision (ECCV'2006), (PART 2), pp. 334-348Ojala, T., Pietikäinen, M., Mäenpää, T., Multiresolution gray-scale and rotation invariant texture classification with local binary patterns (2002) IEEE Transactions on Pattern Analysis and Machine Intelligence, 24 (7), pp. 971-987Kovalev, V., Volmer, S., Color co-occurence descriptors for queryingby-example (1998) International Conference on Multimedia Modeling, p. 32Tao, B., Dickinson, B.W., Texture recognition and image retrieval using gradient indexing (2000) Journal of Visual Comunication and Image Representation, 11 (3), pp. 327-342Brodatz, P., Textures: A photographic album for artists and designers (1966) DoverYoung, H.P., An axiomatization of borda's rule (1974) Journal of Economic Theory, 9 (1), pp. 43-52Cormack, G.V., Clarke, C.L.A., Buettcher, S., Reciprocal rank fusion outperforms condorcet and individual rank learning methods (2009) ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 758-759Khudyak Kozorovitsky, A., Kurland, O., Cluster-based fusion of retrieved lists (2011) Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, Ser. SIGIR '11, pp. 893-902