Federal court ruling deals another mammoth blow to the misuse of standardized testing

CONTACT:  Rob D’Amico, 512-627-1343

Houston ISD’s use of ‘black box’ system for evaluating teachers compared to a ‘house of cards’

Decision follows another legal action nixing state rules on using test scores for teacher evaluations

A Thursday ruling in the federal lawsuit filed by the Houston Federation of Teachers against Houston ISD sends a clear message to school districts about the perils of using standardized test scores to evaluate teachers, said Texas AFT President Louis Malfaro. A Texas AFT lawsuit settled the previous day against Texas Education Commissioner Mike Morath nixed state rules specifying the use of test scores for the state’s new teacher evaluation system.

“We’ve seen two significant rulings this week that deliver a mammoth blow to the misuse of standardized test scores for teacher evaluations, and school districts would be foolhardy to keep using this deeply-flawed way of measuring the effectiveness of their teachers,” Malfaro said.

“Houston ISD’s system of making employment decisions—such as whether to reward teachers or fire them—with the use of a secret formula based on students’ test scores has been put through the wringer with our lawsuit against the district,” Malfaro said. “And what has been squeezed out is an astounding account of just how unfair and damaging the district’s evaluation system has been.”

In 2014, the Houston Federation of Teachers, the local affiliate of Texas AFT—along with several teachers—sued Houston ISD over its use of the Education Value-Added Assessment System (EVAAS). The district had contracted with a company called SAS to use EVAAS as a statistical methodology measuring a student’s performance on standardized tests. A teacher’s EVAAS score was supposed to measure the effect, or added value, of a teacher on a student’s academic growth over the school year. The school district then used this unsound methodology for decisions about teacher evaluation, bonuses, and termination, yet it is a “black box” system in which the methodology is considered proprietary and confidential.

“With the black box system, teachers couldn’t challenge the methodology for their evaluations, couldn’t see how they were supposed to improve their instruction, and thus many teachers—including those awarded with teachers of the year honors—were labeled as low-performing, with teachers even wrongly terminated for low EVAAS scores,” said Zeph Capo, Houston Federation of Teachers president. “We argue in our lawsuit that this violates the employees’ due process rights under the constitution.”

The Thursday ruling by the U.S. District Court, Southern Region, determined that HISD’s system violates teachers’ due process rights and denied Houston ISD’s attempt to dismiss the lawsuit.  In the ruling, Judge Stephen Wm. Smith found that the Houston Federation of Teachers may proceed with the lawsuit on a procedural due process claim, due to teachers’ lack of sufficient information to meaningfully challenge terminations based on low EVAAS scores.

“The language used by the judge makes it pretty clear that the district doesn’t have a leg to stand on in defending the defective evaluation system,” Capo said.

In the ruling, Judge Smith noted: “When a public agency adopts a policy of making high stakes employment decisions based on secret algorithms incompatible with minimum due process, the proper remedy is to overturn the policy….”

Part of the EVAAS formula for teacher evaluations.
Part of the EVAAS formula for teacher evaluations.

The judge also noted that Houston ISD admitted to the existence of incorrect EVAAS scores, and it even acknowledged in an online “Frequently Asked Questions” piece that those mistakes would not be fixed.  The district wrote: “Once completed, any re-analysis can only occur at the system level. What this means is that if we change information for one teacher, we would have to rerun the analysis for the entire district, which has two effects: one, this would be very costly for the district, as the analysis itself would have to be paid for again; and two, this re-analysis has the potential to change all other teachers’ reports.”

Judge Smith noted: “The remarkable thing about this passage is not simply that cost considerations trump accuracy in teacher evaluations, troubling as that might be. Of greater concern is the house-of-cards fragility of the EVAAS system, where the wrong score of a single teacher could alter the scores of every other teacher in the district. This interconnectivity means that the accuracy of one score hinges upon the accuracy of all. Thus, without access to data supporting all teacher scores, any teacher facing discharge for a low value-added score will necessarily be unable to verify that her own score is error-free.”

On Wednesday, Texas AFT settled a separate lawsuit against the commissioner of education’s rules for the state’s new teacher evaluation model—the Texas Teacher Evaluation and Support System (T-TESS).

Texas AFT filed suit one year ago in state district court to challenge the authority of the Texas commissioner of education to dictate how to measure student performance for the purpose of teacher appraisals.  Texas AFT filed the suit because the commissioner attempted to enshrine in state rules the misuse of test scores as part of an exclusive list of permissible methodologies for teacher evaluation.

As part of a negotiated settlement of the Texas AFT lawsuit, the commissioner agreed to strike from state rules any reference to specific measures of student performance, including so-called “value-added data based on student state assessment results.” The commissioner has agreed to revise current state regulations so that school districts are free to use any method they choose to measure how teachers’ students progress academically.

“Both of these legal actions are a victory for school districts that want to develop meaningful tools to appraise and support teachers,” Malfaro said.  “The defects of the Houston system illustrate problems that could have spread statewide if the commissioner had succeeded in his attempt to dictate local appraisal methodologies. Local school districts, working in collaboration with their teachers and staff, now have a clear path to develop meaningful teacher evaluation systems that support good teaching rather than reinforce a broken system of over-testing and over-reliance on standardized tests.”

After the Houston Federation of Teachers’ federal lawsuit was filed and the union continued to fight the evaluation system, Houston ISD abandoned its reliance on the secretive and proprietary EVAAS system when the Houston ISD School Board voted last summer not to continue its contract with SAS. “Our legal actions will send a strong signal to school districts that systems using value-added methodologies aren’t accurate and merely exacerbate the overemphasis on testing, to the harm of both teachers and students,” Capo said.

Texas American Federation of Teachers represents more than 65,000 teachers, paraprofessionals, support personnel, and higher-education employees across the state. Texas AFT is affiliated with the 1.6-million-member American Federation of Teachers.

Tags: ,