Evaluating and Improving Teachers’ Effectiveness—What We Know and What It Means for Policy-Makers

Texas AFT’s Austin symposium on teacher effectiveness yesterday was a partial corrective for years of unbalanced presentations at state legislative hearings, which have promoted “silver bullet” concepts like value-added measurement of teachers’ impact based inordinately on students’ test scores. The symposium gave lawmakers and educational stakeholders a clear picture of current scholarship on the limits and potential uses of current methodologies, including value-added—and the lively discussion provided a clear sense of direction for productive policy-making.

Brian Gill of Mathematica Research suggested that value-added models have a role to play as one part of a broader evaluation of teachers’ work, by appropriately focusing on student achievement growth and by offering a quantitative cross-check on qualitative findings based on observation of teachers in the classroom. But he acknowledged that a recent study by colleagues at Mathematica found even the best value-added models are highly error-prone; with one year’s worth of achievement data, they found that there was a one-in-three chance of misclassifying a teacher as a substandard performer.

Charlotte Danielson, an internationally recognized expert on the design of evaluation systems to improve teaching, sharply criticized the narrowing of the curriculum that inevitably goes along with an inordinate stress on narrowly focused state achievement tests. The narrow scope of achievement testing is the greatest weakness of value-added methods, she said, and the use of test results for individual, high-stakes personnel decisions does not match up well with the reality that teaching is teamwork. School-wide performance measures and rewards are preferable for this reason, she said, encouraging the professional conversation and collaboration among colleagues that lies at the heart of teacher improvement. Danielson called for a parallel evolution of student assessment and teacher evaluation to reflect the true complexity of both learning and teaching. With that approach, she maintained, we can meet the dual goals of evaluation—quality assurance and advancement of professional learning.

Lauren Resnick of the Institute for Learning at the University of Pittsburgh talked about the tension between those two goals, noting that parents and legislators “want the bottom line.” She stressed the need for communicating with stakeholders who have little patience for the “inside baseball” analysis of teachers’ practices. She called for “fast prototyping” of better evaluation methods that give the public and policy-makers an intelligible measure of progress. However, she said, we need to be measuring progress based on mastery of a high-level, well-rounded curriculum, in contrast to the narrow assessments we have now.

AFT President Randi Weingarten noted that teachers are the first to tell you that evaluation practices need to change. Drive-by evaluation, based on nothing more than momentary observation, doesn’t produce meaningful guidance to improve teaching, she said—and neither does test-driven evaluation, based far too much on error-prone analysis of standardized achievement-test results.

What’s needed as a foundation, Weingarten said, is a strong curriculum and assessments geared to critical thinking and the active application of knowledge. With that foundation in place, evaluations should be part of a system of continuous improvement. Evaluations based on multiple measures (including classroom use of best practices as well as student achievement) become one tool in that system, along with targeted training, time for teachers to reflect and act on what does and doesn’t work in the classroom, and peer assistance. Meanwhile, as we put greater weight than ever on teachers’ impact on student learning, Weingarten said, we also have to recognize and improve the other factors affecting learning—such as children’s health, nutrition, and safety.

So it’s necessary but not enough just to alter teacher evaluations, Weingarten asserted. To reach the goal of an educational system that leaves the factory model behind and prepares students to be productive in a knowledge-based global economy, we need a comprehensive approach to improving teacher effectiveness, she said. And in hundreds of districts around the country, with AFT’s support and assistance, teachers and administrators and parents are collaborating on locally designed experiments to achieve this sort of comprehensive reform.

Weingarten’s comments echoed the policy recommendations that Lauren Resnick presented to the Senate Education Committee at an interim hearing last July. Resnick advised against a top-down state mandate of a new evaluation system. We need first to “figure out what works,” she said, and the best way forward would be for the state to set some guidelines and then encourage local “trials” of alternative approaches to the evaluation and improvement of teachers’ effectiveness. The state could provide incentives for this work, and a crucial part of the bargain with local districts would be rigorous, impartial evaluation of the results of these local trials to determine what works and what doesn’t.