A new front is opened in the war on standardized testing.
Seven HISD teachers and their union are suing the school district to try to end job evaluations tied to students’ test scores, arguing the method is arbitrary, unfair and in violation of their due-process rights.
The lawsuit, filed in federal court late Wednesday, could have far-reaching implications as more districts and states use student test data to grade teachers.
The Houston case focuses on the district’s use of a specific, privately developed statistical model that analzes test data to try to gauge a teacher’s effectiveness.
In some cases, according to the lawsuit, teachers see their scores fluctuate from year to year, while other results are based on tests not aligned to the state curriculum. The lawsuit also argues that all teachers aren’t treated equally, and they can’t adequately challenge their ratings because the formula is too opaque.
For example, the lawsuit says, Andy Dewey, a social studies teacher at Carnegie Vanguard High School, received such high scores in 2012 that he qualified for the district’s top performance bonus; his results the next year dropped significantly.
“Mr. Dewey went from being deemed one of the highest performing teachers in HISD to one making ‘no detectable difference’ for his students,” the lawsuit said.
Dewey has told the Houston Chronicle previously that he does not understand why his scores vary when he and his fellow social studies teachers — they are rated as a team — don’t change their instruction significantly from year to year.
The system at the center of the lawsuit generally is called “value added.” It uses complex statistics to estimate how well students should perform on standardized tests based on their own past performance. Teachers whose students score better than expected get the best ratings, whether or not the students passed the test.
To do the analysis, HISD contracts with a North Carolina company, whose model is called the Education Value-Added Assessment System, or EVAAS.
You can see a copy of the lawsuit here, the press release from the AFT is here, and some background is here. The Texas AFT has an illustration of the EVAAS formula here. I am not opposed in theory to the idea of value-added evaluations. This is basically what the sabermetric revolution in sports has been all about, coming up with ways to measure performance and determine the value of players in various sports. In sports, however, the relationship between the various metrics – runs created, points per possession, DVOA, etc – has been demonstrably linked to the teams’s actual on-field performance. They also show what sort of things a given player needs to do in order to be valuable. Finally, there are multiple systems that have been created to measure value, and they have risen or fallen based on their usefulness and accuracy. I don’t know how much any of this is true for EVAAS. I do know that teachers should have a clear understanding of what is expected of them, and they should have some input on their evaluation. I’ll be very interested to see how this goes. The Trib and K-12 Zone have more.