AI-generated from sources

The Honest Case for Payment by Results, and Why It Keeps Failing

12 min read

In Brief

  • Payment by results keeps returning because its premise is sound: money raised from the public owes the public visible proof of what it buys, and uninspected schools or teachers paid whatever happens in class have long drawn charges of negligence.
  • Its weakness lies in the proof: the only evidence cheap enough to pay on is a standardized score, and once money depends on it, scores can rise through teaching to the test, narrowed curricula or lowered standards rather than through learning.
  • Arthur Conan Doyle, who condemned paying spiritualist mediums by results, described the mechanism exactly: when pay depends on a phenomenon that may not appear, the paid are tempted to fake it.
  • Several of the system's harshest 19th-century critics accepted tests as necessary and useful; what they opposed was turning an occasional, simple check into the basis of pay.

In 1880, Auberon Herbert gave payment by results the fairest hearing it ever got from an opponent. Writing in Popular Science Monthly, he granted that the system was simple, easy to administer and fairly equal. Then he turned the same qualities against it. By reducing managers, teachers and pupils to the single aim of satisfying regulations, it made success in passing standards and success in education the same thing, and turned schooling into "one long, unbroken grind" [1]. Merit and indictment sit in a single sentence of his text, and the debate has never quite escaped it.

More than a century later, an American president stated the bargain from the other side. Because Washington was giving more to schools, George W. Bush said in 2001, it would "expect more in return" and require states to test every year; how else, he asked, could anyone know whether schools were teaching and children learning [2]? Between these two voices runs an argument that keeps resetting because each side holds half the truth. The premise of payment by results is sound: money raised from the public owes the public a visible account of what it bought. Its failure lies in the proof. The only evidence cheap enough to pay on is one that a school can learn to produce without producing the learning behind it.

Schools nobody inspected, and the case for a ledger

The honest case starts from what happens when nobody looks. In 1860 the Cornhill Magazine described the old-established upper-class schools of its own country as institutions on which no government inspector ever reported, staffed by teachers never trained to teach. They were, it charged, mere money speculations run for the masters' profit, with everything left to self-interest and chance [3]. A Popular Science Monthly writer of 1873 found the same defect in the higher institutions: their work could not be brought to judgment or submitted to fixed tests. Exposing it to public criticism, he argued, would raise their standards [4]. Accountability, in this view, is not an insult to teachers. It is the condition of quality.

Adam Smith traced the problem to its economic root. In The Wealth of Nations he charged that the endowment of schools and colleges had corrupted the diligence of public teachers. Without public institutions, he reasoned, nothing would be taught for which there was no demand [5]. The logic is the one payment by results rests on: a salary guaranteed whatever happens in the classroom rewards nothing in particular. Smith's remedy was the market rather than the inspector. But his diagnosis supplied the inspector's motive.

Public money turns this suspicion into a right. William Bennett Munro, weighing federal grants to the states in 1922, held that a government granting money for any purpose has "an undeniable right" to make sure it is being properly spent, even as he recorded the fear that such grants would bring national control of the schools [6]. Some eight decades later, Bush described his accountability policy in the same terms. Washington had insisted on measurement in return for federal money, he said, while leaving the states to develop their own tests; what it expected was results [7]. The arrangement meets Munro's fear without surrendering his right: the payer checks the output and leaves the method alone.

Without measurement, the alternative was not freedom but guesswork, or worse, display. Randolph Bourne observed in 1917 that the public liked to see its money go into handsome buildings. One town's seventy-five-thousand-dollar high school drove its neighbor to build a hundred-thousand-dollar one, and the money did not go into teachers' salaries [8]. Measured results promised to replace this emulation of façades with evidence. Charles Hubbard Judd, an advocate of testing, recalled a Board of Education made up, as he put it, of some of the city's cheapest politicians, who attacked tests as a fad; experience, he answered, showed teachers precisely where their work was strong and where it was weak [9]. Bush gave the argument a moral edge, calling the neglect of a school's failure to teach every child the worst kind of discrimination [10].

Reading and arithmetic, where the test and the skill coincide

The 19th-century critics conceded more than their reputation suggests. An 1889 contributor to Popular Science Monthly, whose concern was higher education, allowed that elementary schools were a different case. Compulsory, supported by rates and taxes, administered by public bodies and teaching mainly the rudiments, they had a curriculum "more or less mechanical and capable of mechanical tests" [11]. A decade earlier, a writer in the Contemporary Review reported from experience that a well-conducted elementary school could teach natural science, modern languages and political economy and still be self-supporting under the system of government grants [12]. On these accounts, paying on results need not confine a good school to the bare elements. It fit best where teaching was most mechanical.

This concession is the strongest ground the defenders of testing hold. Bush answered critics who complained of teaching to the test with a blunt syllogism: if the test covers basic math and reading, teaching to it means teaching math and reading, "and that is the whole idea" [13]. The reply, given more than a century after the 1889 concession, holds exactly as far as that concession does—where the skill is mechanical enough for the test and the thing tested to coincide. Herbert's objection begins at that boundary. A system that pays on standards passed leads everyone involved to treat passing standards and being educated as the same thing . That identity is true only for the narrowest part of what a school does.

Beyond the boundary, measurement changes what it measures. Edmond Holmes argued that the real results of education lie in the child's heart, mind and soul, beyond the reach of any measuring tape or weighing machine. If teachers must be tested, the test has to be external; teacher and child will then aim at external results, and their value as symbols of inward growth will be wholly ignored [14]. Holmes's point is not that nothing can be measured. It is that a sign, once it becomes the target, stops signifying. Arlo Bates drew the practical rule: tests of proficiency are necessary, but studying for them directly is fatal to the higher uses of study [15].

Cost is what pushes the system toward the weakest kind of sign. Mihai Nadin notes that standardized tests dominated by multiple-choice answers make evaluation cheap, and that they reshape patterns of teaching and learning in the process [16]. The two facts are linked. A test cheap enough to give every child every year must ask what can be checked mechanically, and a test with money behind it will be taught to. The very cheapness that makes paying on results feasible is what makes the results easy to manufacture.

Conan Doyle's mediums, or how proof gets manufactured

The sharpest description of the mechanism comes from far outside the classroom. Arthur Conan Doyle, who believed some mediums' phenomena to be genuine, condemned the practice of paying them by results. A medium who never got results would soon get no payments, he observed, and only an annuity independent of results could remove the temptation "to substitute pretended phenomena when the real ones are wanting" [17]. Transposed to schools, the parable is exact. When the real outcome is slow, invisible and uncertain, and pay depends on a visible sign of it, the sign will be produced by other means.

Set beside Smith, Doyle completes a dilemma. Pay independent of results, Smith warned, corrupts the diligence of teachers . Pay dependent on results, on Doyle's evidence, tempts the paid into fraud . Each witness's remedy is the other's disease. Payment by results is born of the first diagnosis and keeps dying of the second, which is why the argument swings back and forth instead of settling.

In the classroom, the pretended phenomenon rarely looks like fraud. Holmes described what happens when a teacher's own professional welfare depends on an examiner's verdict: he naturally holds himself responsible for "every stroke and dot" his pupil makes. Suppressing the child's natural activities becomes the central feature of the program, and the child is not allowed to do anything the teacher can do for him [18]. The result on paper is the pupil's; the work behind it is the teacher's. Nothing has been faked, and yet the proof no longer shows what it claims to show.

When the stakes reach the level of states, the bar itself moves. Barack Obama, judging the No Child Left Behind law, called its goals admirable but said that teachers were being forced to teach to a test, that history and science were being squeezed out, and that some states had lowered their standards to avoid having their schools labeled failures [19]. Bush, defending his reforms, pointed to rising math scores and to minority children narrowing the achievement gap [20]. The two testimonies are less contradictory than they look. Rising scores are what a working reform would produce—and also what a lowered bar would produce. The proof cannot tell them apart, and that is precisely the problem.

Money sharpens every temptation, as older critics of rewards saw. F. Max Müller believed that the mere knowledge that their reports would be scrutinized was enough to keep even the most reprobate teachers from bearing false witness in favor of their pupils. Pecuniary rewards such as scholarships, by contrast, placed unnecessary temptation before all parties; the proper reward for a good examination, he held, was honor [21]. An 1876 Popular Science Monthly writer saw the same force at work in endowed prizes, which set students racing in whatever direction a donor's whim indicated [22]. Scrutiny, in Müller's account, keeps proof honest. Payment bends it.

The tarantula dance, or why the bargain keeps returning

If the flaw is so well known, the return of the system needs explaining. Another 1889 writer in Popular Science Monthly offered one answer: everyone has a reason to keep it going. Parents, governments, schools, colleges and universities combine to stimulate competitive examination; none quite likes it, yet all keep up the "tarantula dance" until examination becomes the master of education [23]. Holmes added that the examination system controlled education in every Western country in exact proportion as that country was progressive and up to date [24]. The demand for proof, on this reading, is no passing fashion. As Holmes saw it, it advances with modernity itself.

A second answer is that the system manufactures its own supporters. An 1895 Popular Science Monthly writer warned that an imperfect educational system "creates the public opinion by which itself is judged." A parent of 35 was a pupil only twenty years before; if the spirit of competition was stamped into him, he will want it stamped into his children [25]. The bargain returns not only because taxpayers ask for proof. It returns because the voters asking were themselves schooled to believe that a score is what a school produces.

Yet the old critics did not reject measurement; they rejected its use as a wage. The 1889 writer judged examination useful as long as it remained spontaneous, occasional and simple, with an examiner free to use his own discretion. Its mischief began when it was organized into a trade [26]. In 2007, Obama drew a strikingly similar line: testing should work like medical testing, diagnosing a student's needs, and should support learning rather than serve as punishment or mere accounting [27]. Across more than a century, the distinction holds. A test that informs can stay honest because nobody gains by faking it; a test that pays invites the medium's temptation.

The honest proof exists, but it costs. A Popular Science Monthly writer of 1899, discussing public charity, accepted that paying salaries to investigators could be wiser than spending the money on relief, provided the investigation exposed fraud [28]. Applied to schools, the lesson is that verification resistant to gaming—an examiner's discretion, an inspector's judgment—is a cost to be paid, not a waste to be cut. John R. Commons, writing of an educational test in connection with the Fourteenth Amendment, added a condition from a different field. Such a test, which judged adults rather than schools, was rational in his view only when the state made an honest and diligent effort to equip every man to pass it [29]. Proof, in other words, is owed in both directions.

✻

Payment by results—funding schools and teachers according to what their pupils can show on a test—rests on a premise that several of its sharpest critics granted: money raised from the public owes the public a visible account of what it bought . Schools run for their masters' profit without inspection, teachers paid whatever happened in class, and public money poured into ever grander buildings gave that demand its force . The system fails for a different reason. The proof cheap enough to pay on is a sign, and a sign with money attached gets produced by other means: a teacher guiding every stroke of the pupil's pen, a curriculum narrowed to tested subjects, a state lowering its bar .

What this long quarrel between payers and teachers suggests is less a choice between measuring and trusting than a separation between the test that informs and the test that pays, along with an acceptance that honest proof costs discretion, judgment and salaries . Charles Wagner, writing of private life rather than schools, warned against becoming "mere accountants" who count as worthless whatever cannot be drawn up in figures on a ledger [30]. Public schools cannot do without a ledger. But a school paid for what it can show will, sooner or later, show what it is paid for.