Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)Emerging technologies in health care aim at reducing unnecessary visits to medical specialists, minimizing overall cost of treatment and optimizing the number of patients seen by each doctor. This paper explores image recognition for the screening of diabetic retinopathy, a complication of diabetes that can lead to blindness if not discovered in its initial stages. Many previous reports on DR imaging focus on the segmentation of the retinal image, on quality assessment, and on the analysis of presence of DR-related lesions. Although this study has advanced the detection of individual DR lesions from retinal images, the simple presence of any lesion is not enough to decide on the need for referral of a patient. Deciding if a patient should be referred to a doctor is an essential requirement for the deployment of an automated screening tool for rural and remote communities. We introduce an algorithm to make that decision based on the fusion of results by metaclassification. The input of the metaclassifier is the output of several lesion detectors, creating a powerful high-level feature representation for the retinal images. We explore alternatives for the bag-of-visual-words (BoVW)-based lesion detectors, which critically depends on the choices of coding and pooling the low-level local descriptors. The final classification approach achieved an area under the curve of 93.4% using SOFT-MAX BoVW (soft-assignment coding/max pooling), without the need of normalizing the high-level feature vector of scores.601233913398Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)MicrosoftFundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)FAPESP [2010/05647-4, 2011/15349-3]CNPq [307018/2010-5, 304352/2012-8]