Polarity of comments and internal consistency in peer review reports on scientific research articles

Abstract

Peer review reports on scientific articles means a key genre to explain how scientific knowledge is collectively constructed. In these reports, reviewers write a recommendation for publication along with a series of comments. The quality analysis of the peer review process has been commonly conducted based on indicators, such as rejection rates and the agreement level among evaluators. However, a more qualitative investigation on the process still remains outstanding. This work aims at describing the polarity ( positive, negative and neutral) of comments corresponding to 56 peer review reports belonging to the journal Onomazein and determining whether this proportion is consistent with the reviewers recommendation, i.e. Accepted, Accepted with major revisions, Accepted with minor revisions and Rejected). After the analysis of 1.472 comments, it was possible to determine that the highest proportion of comments is negative, independent of the decision. The analyzed processes also showed a high level of consistency. The more favorable the recommendation, the higher the proportion of positive comments, and, consequently, the less the proportion of negative comments.