Can standards-referenced, large-scale assessment data lead to improvement in the education system?

Abstract

This paper documents the development of performance standards for the Trinidad and Tobago primary school national assessments of educational achievement. Performance standards are written expectations of student achievement operationalized in defensible cutscores. A major argument in this paper is that these standards are necessary to evaluate quality in the education system because they directly address the question of "how good is good enough?" Standards-referenced measurement systems are the basis of both compensatory and accountability systems. Standard-setting procedures in the 2005 and 2006 national assessments of educational achievement are described, followed by an evaluation of procedural validity using quantitative and qualitative data gathered from panellists. The findings indicated that while panellists appear confident about procedures and outcomes, cognitive complexity and organizational inefficiency could prove to be critical constraints. While the introduction of student performance standards appears useful; by itself, it cannot lead to significant education improvement unless there is also a coherent policy for effective data use within a national evaluation system. In developing such a policy, consideration must be given to choosing between (a) low versus high examination stakes, (b) compensatory versus accountability policies, and (c) school-based versus centralized management of test processes. At the very least, stakeholders must understand the purpose of national assessments of educational achievement