Understanding Government

The Obama administration drew a line in the sand in late July when it outlined how state education departments could get a slice of a $4.3 billion federal grant program called Race to the Top. The guidelines, in essence, said to states battered by the recession: adopt most of Barack Obama and Education Sec. Arne Duncan’s vision of education reform — or fund schools on your own. In addition, states must lift caps on the number of charter schools and start judging teachers based on student performance. Hinging on the question of student performance is a crucial part of the Obama administration’s education reform: merit pay, which means that teachers will earn tenure and their salary (or less dramatically, their annual bonuses) based not on seniority or their advanced degrees, but instead on evaluations of their teaching performance. These evaluations tend to link teacher performance to student performance on high-stakes standardized tests.

Aug. 31 marked the end of the public comment period on the Race to the Top grant guidelines. Merit pay may be the most politically explosive part of the Obama-Duncan education agenda. But the stakes aren’t only political — there is still no clear evidence that merit pay works. Nonetheless, Obama and Duncan may be on to something that will have impact far after the headlines have faded away.

A National Education Strategy

When Barack Obama signed the American Reinvestment and Recovery Act, or stimulus bill, this February, he instantly changed Washington’s role in education. The stimulus bill gave $115 billion to the Education Dept. – a federal agency that in 2008 had a total budget of $45 billion. A lot of this money went to replenish state education departments who made cuts; at the start of the summer, 46 states had budget shortfalls. But not all, or even close to all, of the $115 billion is allocated to cover budget gaps at the state level. In Illinois, for example, the state education department recently announced that it was slashing $475 million from its budget, with everything from early childhood education to Chicago Public School janitorial jobs getting cut. Yet instead of handing Illinois $475 million to keep existing programs, the federal Education Dept. wants the state to apply for grants like Race to the Top so they can start new, federally-desired programs.

The Race to the Top grants will go to states “that are creating the conditions for education innovation and reform,” according to the application guidelines. What this means, in part, is that state legislatures must eliminate any law that prohibits “linking student achievement and student growth data to teachers for the purpose of teacher and principal evaluation.” The result is that many states, forced this year to make big education cuts to balance their budget, have had a sudden change of heart about merit pay. The Texas legislature last week approved the largest merit pay program in the nation – three months after they had rejected an almost identical plan. California Gov. Arnold Schwarzenegger has basically told his education department and the state legislature to do whatever necessary for the state to qualify for Race to the Top.

Several states, Tennessee being one example, have experimented with quantitative teacher evaluations in the past. But the Obama administration has successfully pressed many states to take another look at their teacher evaluation and also teacher payment methods. “Why has merit pay become such a big issue?” says Linda Lenz, publisher of Catalyst, a Chicago-based education magazine. “Because the president made it so.”

Should we be upset about this muscular federal role? Yes, says Diane Ravitch, an education historian at New York University and a top official in the George H.W. Bush education department “The Department of Education should respect the requirements of federalism,” Ravitch wrote in comments on the Race to the Top guidelines, “and look to states to offer their best ideas rather than mandating policies that the current administration likes.” Many other education experts share Ravitch’s view – but many also support Obama in this effort. “Given that merit pay is unproven, it is unlikely that individual states or districts will do this,” says Gary Ritter, an education policy professor at the University of Arkansas. “So I think, then, this is a great place to use federal stimulus dollars. The Feds will be able to look at the individual programs states create and learn what works best.” Lenz of Catalyst agrees, arguing that states have had multiple chances over many years to do their own teacher accountability reform. “I’m willing to give Washington sort of a chance here,” she says. “Because if you look at what states have actually done, you trust them less.”

Teacher Concerns

One reason why some states have been skittish about merit pay is that the issue is a lightning rod among teacher’s unions. As other labor unions have imploded or quietly shrunk in influence, the National Education Association and American Federation for Teachers have remained well-endowed supporters of the Democratic Party, with significant influence on party education policy. But this symbiotic relationship grew more complicated when liberal lions George Miller and the late Ted Kennedy wrote the House and Senate version, respectively, of the 2002 No Child Left Behind law, a standardized test-based education policy that teacher’s unions opposed.

The Obama administration has furthered this more complex Democratic Party-teacher’s union dynamic. Teachers booed Arne Duncan at the NEA annual meeting in July when he mentioned merit pay. But they also cheered him for most of the rest of the speech, and some teachers have praised Duncan and Obama for pushing teacher accountability while also constructively engaging with teacher’s unions. The Race to the Top guidelines, though, offended NEA. “It is inappropriate to require that states be able to link data on student achievement to individual teachers for the purpose of teacher and principal evaluation,” the NEA wrote in their comments on Race to the Top. “And we certainly should not primarily base additional compensation on whether students meet particular testing targets on a particular day.”

One argument NEA makes against merit pay is that it has “little or no research base of success.” On the other hand, there is also little or no research against merit pay. “The jury is still out,” says Susan Burns, a senior staff member at the Vanderbilt University National Center on Performance Incentives. “The research is pretty thin.” Burns is working on a study funded by the Education Department that looks at two different merit pay schemes, including one in the Nashville school district. Ritter of the University of Arkansas points out that teacher merit pay was not pulled out of thin air a couple of years ago. But in other ways, he says, the issue really is pulled out of thin air. “Merit pay is one of those educational strategies that comes to prominence every 20 years of so when political leaders declare we need more teacher accountability,” Ritter says. “What’s different now is that new technology has enabled better data collections systems. The ability to collect data on value a teacher adds to a school is relatively new.”

But there is legitimate concern about how accurate this data collection can be in evaluating teachers. A coterie of education advocacy groups including the Austin, Texas-based Data Quality Campaign, argue that data collection systems have evolved, so it is possible to isolate how much value an individual teacher adds to the performance of an individual student. “Right now there’s a system in Tennessee that other states are implementing that works,” says Kate Kowlaski, a senior associate at the Data Quality Campaign. The idea behind the system in place in Tennessee and a dozen other states is that each student and each teacher is assigned a specific identification number. From there, specific information about individual students – like their income status or whether English is not their first language – can be disaggregated to isolate the affects an individual teacher might have on their education. But any system that is about more than just judging teachers on how students score on the high-stakes test “takes two years to fully implement,” says Kowlaski.

Two years is a long time in the world of education policy, where much-touted ideas like school vouchers are currently dying slow deaths. Also, once these data systems are implemented, what will they be used for? Giving teachers feedback or determining teacher’s salary and tenure? Burns of Vanderbilt says teachers are right to be concerned about a data-based evaluation that could determine their livelihood. “It’s a conundrum,” Burns says. “There are probably appropriately sophisticated formulas that exist to calculate what value a teacher adds. The problem is that they are very complicated and very difficult for most of us to understand.” In other words, teachers might not know –and principals might not be able to explain – why they have received unsatisfactory ratings.

The Data on Data

Even if data collection systems are accurate, there are larger concerns: Reform advocates say teachers are far and away the most important part of a student’s education – people that are supposed to open a world of possibilities for students and inspire them to become engaged and productive citizens. “A string of four above-average teachers in a row,” says Burns, paraphrasing from a 2006 Brookings Institution study. “Can close the gap between an economically advantaged and economically disadvantaged student.” Is it fair to judge these professional near-saviors by crunching numbers? A recent study by the Economic Policy Institute (EPI), a liberal Washington think tank, showed that professional evaluation and merit pay based on numerical measures are used infrequently and often ineffectively in both the public and private sector. “Workers will often strive to produce what is measured at the expense of what is not, even if what is not measured is highly valuable to the firm,” the study, released this May, concludes. In one such case, military recruiters who were paid for their number of recruits subsequently recruited soldiers who proved academically incompetent or were later found to have criminal records. The study approvingly quotes sociologist Donald Campbell, who declared 30 years ago that “public goals are too complex to reduce to simply quantifiable measures – and attempts to do so corrupt public service.”

The EPI study swings hardest against evaluating teachers based on student performance on standardized tests. Since the 2002 federal No Child Left Behind law, test performance has been used to evaluate schools, but not individual teachers. Study co-author Richard Rothstein writes that even this indirect evaluation of teachers based on student tests is a mess: “Attempts to hold schools accountable for math and reading test scores have corrupted education by reducing the attention paid to other important curricular goals; by creating incentives to ignore students who are either above or far below the passing point on tests, by misidentifying failing and successful schools because of test unreliability, by converting instruction into test preparation that has little lasting value, and by gaming which borders (or may include) illegality.” But few serious advocates of merit pay for teachers seek to have test scores used to the extent that NCLB uses them to assess schools. The merit pay schemes that have been introduced on the state and district levels have involved some student test data, but also more qualitative measures – like principals observing teachers in the classroom and rating them on a five-point scale. “I don’t think we will ever have this conversation about merit pay without standardized test scores being one of the measures,” says Burns. “But we are far from knowing what works best. There are all kinds of merit pay schemes all over the country. We are just beginning to discern what we should reward teachers for.”

In Chicago, for example, the more qualitative the teacher evaluation scheme, the more likely it wins acceptance from teacher’s unions. The Chicago Public Schools began a pilot program in 2007 in 43 elementary schools in which principals evaluate teachers on areas like classroom instruction and environment. Similar qualitative evaluations have been adopted in cities like Cincinnati and Denver with teacher union support. A recent study by the University of Chicago Consortium on School Research finds that teachers like the pilot program as a “formative tool” to improve principal-teacher feedback but not a “summative tool” to determine their pay or tenure. The Consortium’s Sarah Rae Stoelinga , who co-authored the study, says that teachers feel the current feedback is “hands down a more fair way than the status quo for teachers to get real information about their performance.”

That status quo is a “checklist” where a principal observes a teacher twice a year and them ranks them as “strong,” “weak” or “does not apply” in professional standards, instruction and classroom environment. Unlike the pilot program, the checklist provides no definition or criteria for judging a teacher strong or weak. But teachers remain reluctant to have these evaluations determine their pay and tenure, and they tend to oppose adding quantitative data on student performance to evaluations. “The timeline keeps getting pushed back to expand the program,” Stoelinga says. “The value added measures will be difficult to implement. It will be a challenge to get the Chicago Teacher’s Union on board.”

Future Merits

One aspect of merit pay that the Economic Policy Institute study struggled with is the fact that that while few professions have direct pay for performance, many have indirect forms of performance pay that teaching lacks. Top lawyers, for example, win more clients or move up in their law firms. The best sales clerks get assigned more hours by supervisors or get promoted. So, unlike many workers, teachers are usually not rewarded for good performance by increased responsibilities – or increased money. Their pay is usually based on seniority, on continuing education credits, and on what advanced degrees they have. Career advancement usually means the move to an administrative position, which moves teachers out of teaching. The University of Arkansas’s Gary Ritter points out another way good teachers are promoted — they are able to move from a bad school to a good school. “If I’m a teacher, I can’t be rewarded more money, but I can take an easier assignment,” Ritter says. “That’s exactly the wrong incentive to present to good teachers.”

Merit pay could be the right incentive as part of a mix of educational programs that are both bold and modest. President Obama tends to favor an incremental approach — what the New Republic called a “nudge-ocracy” where instead of making sweeping overhauls, the administration proposes changing the incentives of a situation so actors behave differently. If used in this way – to nudge teachers to be more conscientious about their teaching and how their students are performing, merit pay could be a part of a broad national educational strategy: Washington provides an education mandate, but also money and flexibility so states can meet that mandate.

The first wave of Race to the Top funds will be handed out in early 2010. What bears monitoring is whether states will implement programs sensitive to teacher concerns and sophisticated in their use of both quantitative and qualitative data. Ultimately, the biggest test for merit pay won’t be current ideological battles, but if teacher quality and student performance indeed changes and school district have the intelligence and patience to carry the new policy through. “There are waves of change all the time,” Stoelinga says. “And we don’t know what the level of commitment will be – tune in next year.”