The indexing automation has been discussed by researches in the area of Information Science however the discussions have not been so clear on the use of indexing software. Thus, it is necessary to know the indexing software, as well as its application in the analysis of documentary contents. To do so, it is proposed, here, to investigate both the consistency of indexing and the exhaustiveness and precision of the information retrieval, by means of comparative analysis between SISA (Sistema de Indizacion Semi-Automatico) automatic index and BIREME ( Centro Latino-Americano e do Caribe de Informação em Ciencias da Saude) manual indexing. The aim of this paper is to contribute to the theoretical development of the indexing automation and the improvement of SISA. Thus, SISA application and evaluation was used based on the calculation of the consistency indexes between the two types of indexing, and the calculation of the exhaustiveness and precision indexes in information retrieval, by means of searching into BDSISA and BIREME databases, composed by descriptors taken from SISA and manual indexing respectively. The differences among the terms used in scientific papers comparing to the DeCS ones were the main difficult factor to achieve higher consistency indexes in the indexing. These differences influenced the exhaustiveness and precision indexes in the information retrieval, showing that it is necessary to improve the documentary language used by SISA software and to incorporate linguistic methods.