Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)In this paper, we consider meta-recognition, an approach for postrecognition score analysis, whereby a prediction of matching accuracy is made from an examination of the tail of the scores produced by a recognition algorithm. This is a general approach that can be applied to any recognition algorithm producing distance or similarity scores. In practice, meta-recognition can be implemented in two different ways: a statistical fitting algorithm based on the extreme value theory, and a machine learning algorithm utilizing features computed from the raw scores. While the statistical algorithm establishes a strong theoretical basis for meta-recognition, the machine learning algorithm is more accurate in its predictions in all of our assessments. In this paper, we present a study of the machine learning algorithm and its associated features for the purpose of building a highly accurate meta-recognition system for security and surveillance applications. Through the use of feature-and decision-level fusion, we achieve levels of accuracy well beyond those of the statistical algorithm, as well as the popular 'cohort' model for postrecognition score analysis. In addition, we also explore the theoretical question of why machine learning-based algorithms tend to outperform statistical meta-recognition and provide a partial explanation. We show that our proposed methods are effective for a variety of different recognition applications across security and forensics-oriented computer vision, including biometrics, object recognition, and content-based image retrieval.7412141224ONR STTR [N00014-07-M-0421]ONR SBIR [N00014-09-M-0448]ONR MURI [N00014-08-1-0638]DHS SBIR [NBCHC080054]Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)MicrosoftFundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)ONR STTR [N00014-07-M-0421]ONR SBIR [N00014-09-M-0448]ONR MURI [N00014-08-1-0638]DHS SBIR [NBCHC080054]FAPESP [2010/05647-4]