Why A-F Ratings for Schools Do Not Make the Grade

The issuance of A-F ratings for Texas schools this week was greeted with a torrent of criticism from educators. Even though the ratings were incomplete, based on last year’s data, and did not count for purposes of state accountability sanctions, the criticism was fully justified.

Our counterparts at the Texas Association of School Administrators have published a timely article by expert John Tanner, with the research group Test Sense, explaining what the author calls the “A-F accountability mistake.” The entire article is worth your time, but Tanner’s executive summary of the piece tells the story quite well, and we reprint that summary here:

The reduction of school quality to a single mark is the purpose of A-F school rating systems. The argument is that a grade will signal a level of quality and make it difficult for low-rated schools to escape scrutiny. Advocates of such rating systems use terms such as “simple,” “clear,” and “transparent” to describe them, and frequently cite competition and subsequent improvement as key outcomes. Former Gov. Jeb Bush of Florida and a number of organizations he supports are the most vocal proponents of such systems. Florida adopted its system in 1999 and sixteen other states have since followed. Texas is scheduled to implement its A-F rating system for the 2017-18 school year.

Research on such systems is surprisingly inadequate given the prevalence of A-F as a policy tool. What does exist is almost universally negative. Florida cites significant gains in the first few years of its program, a fact that is a primary argument in support of such systems. Nevertheless, by Florida’s own admission, the majority of the “gains” were due to changes in the rules, a fact not shared with the Texas Commission on Next-Generation Assessments and Accountability when the Bush-supported organizations offered testimony on this topic in 2016.

Most states with A-F rating systems have adjusted the rules to their systems following implementation so the results more closely match the public and policymakers’ expectations for the distribution of grades. These adjustments call into question the logic behind such systems: It appears they are only declared successful once they reflect a preconceived notion of expectations, not an objective reality.

The few basic rules behind A-F appear simple on the surface but generate an inordinate number of behind-the-scenes calculations and numerous additional rules that render the results unusable for informing change. In many cases schools that perform in a statistically similar manner are awarded vastly different grades, while schools that perform quite differently are awarded similar grades. The reduction to a single grade tends to downplay achievement gaps. In a study of the Oklahoma system, gaps were shown to be wider in higher graded schools than in lower graded schools, and lower graded schools were shown to be performing better with subgroups than higher graded schools.

Based heavily on standardized test scores, A-F school rating systems tend to assign grades in which the socioeconomic status of the school is the single best predictor of the grade, ignoring the efforts being made in some of the most challenged educational environments.

The reduction of a school to a single grade has the tendency to color the judgments and subsequent actions of the entire school, even though each school is a diverse place with the need to serve all students. Reducing a school to a single grade has the predictable effect of telling a school with a good grade that all is well and telling a school with a bad grade that all must change, even though neither can ever be accurate.

Conclusion: Rating schools and districts with A-F letter grades is a policy idea that fails every criterion put forth as a reason for having it. It is neither simple nor transparent. It misrepresents a large proportion of what happens in schools by reducing an entire school to a single mark that can only be partially appropriate given the complexity of schooling. In the end, A-F school ratings do more harm than good. They create confusion among educators, and fail to offer the public useful or accurate information about their schools.