AI-generated from sources

From the Lot to the Ranked List, Why Republics Once Trusted Chance More Than Their Assessors

13 min read

In Brief

  • Ancient republics never used the lot blindly: Athens scrutinized candidates and Venice drew the electors of the doge by lot, so that chance cancelled faction while screening kept out the incapable.
  • Carl Schurz preferred a ranked list to a pass examination because discretion among the qualified, applied to thousands of obscure posts, would become a cloak for favoritism.
  • By 1922 a civics textbook conceded that competitive tests reliably weed out the unfit but cannot reliably pick the best, which makes the examination a threshold rather than a true ranking.
  • Research funding reproduces the same dilemma: as success rates fall, reviewers must make fine distinctions among proposals that all clear the bar of competence.

Every republic that hands out offices or prizes must answer a question older than any written constitution: who decides who is worthy, and who keeps watch over the one who decides? Over the long run the answer has swung between two instruments that look like opposites, chance and judgment. Aristotle already treated the method of appointment as a marker of the regime itself, classing the lot as the democratic way of filling magistracies and election as the oligarchic one, and he described constitutions that deliberately borrowed from both [1]. More than two thousand years later, Carl Schurz built the American case for civil service reform on a conviction that seemed to leave chance no place at all: only a competitive test open to every qualified person could drive favoritism out of public employment [2].

Between these two poles runs a story less of progress than of displaced trust. The ancient designs paired a scrutiny of candidates with a mechanism that no patron could steer, so that the final choice escaped the hands of faction [3, 4]. The 19th-century reformers replaced that mechanism with the ranked list, wagering that a board of examiners could measure fine degrees of merit and that this measurement would be as impartial as a draw while being wiser [5, 6]. The wager drew hesitation from the outset, and in time some of its own defenders conceded that the instrument separated the fit from the unfit far more reliably than it separated the good from the best [7, 8]. That concession quietly reopens a problem the old republics had solved in their own way.

Screening and the Draw in the Ancient Republics

In Aristotle's typology, the way a city fills its offices is never a neutral technique. The lot belongs to democracy because it treats every eligible citizen as interchangeable; election belongs to oligarchy because it presupposes that some can be preferred over others; a property qualification is oligarchic, its absence democratic . The mixed or constitutional regime, in his account, takes the principle of election from oligarchy and the disregard of qualification from democracy . What matters for the long history of selection is the underlying idea: every procedure distributes power, and choosing a procedure is already choosing whose judgment will count.

Election, for Aristotle, carried a specific danger that later debates would rediscover under other names. Where no census regulates who may stand, those who seek office court the people with flattery until the popular will is set above the laws, and a democracy can slide into something new and worse [9]. Demagogues, he observes, grow great by referring everything to the assembly, so that the authority of every magistracy is undermined [10]. Election, in this light, is itself a competition, but one whose winners are selected for their skill at pleasing rather than for their fitness to govern. The lot neutralized precisely that skill: no one can flatter a draw.

The draw was not, however, indifferent to quality. Aristotle's description of the Athenian constitution shows a procedure of scrutiny in which candidates could be accused and in which the whole body voted, so that an unprincipled candidate who had rid himself of his accusers might still be disqualified before the court . Chance operated inside a frame of verification. A historian writing in 1905, more than two millennia after Aristotle and looking back at Venice, describes the same logic in a more elaborate form: open suffrage invites tumult or corruption, the secret ballot prevents the first without securing against the second, and the lot alone risks placing incapable persons in positions of trust, so the Venetian scheme left the choice of the doge to electors who had themselves been drawn by lot . Each instrument was used to cancel the characteristic vice of the other.

The lot had its detractors, and their objection is the one the 19th century would make its own. The editorial commentary printed with Plato's Laws counts the use of the lot among the worst features inherited from the old Athenian democracy and regrets that no attempt is made to proportion representation to merit [11]. Joseph Addison, reflecting on lotteries in The Spectator, supplies the counterargument without intending to: before the draw, every ticket has the same claim to good fortune, and no reason can be given to prefer one to another [12]. The lot is defensible exactly where reasons for preference run out. Its proper domain is the set of candidates whose claims cannot honestly be told apart, which is why the ancient designs always placed a filter before it.

The Examination as a Weapon Against Patronage

The reformers of the American civil service faced an enemy the ancients would have recognized: the personal will of the patron, which Schurz, in the vocabulary of his time, called spoils politics. His formula was unambiguous: favoritism could be excluded from the public service at large only by competitive tests freely accessible to all qualified persons, with the best merit properly demonstrated securing the best title to appointment . The fundamental rule, as he restated it later, was that appointments must not depend on the arbitrary will of the appointing power but solely on merit shown in open competition, supplemented by a period of probation . The diagnosis was ancient; the remedy was new. Instead of removing the final choice from human hands by chance, the reform entrusted it to a measurement presented as objective.

Why insist on ranking rather than on a simple threshold? Schurz's answer is revealing. A pass examination might work for a handful of conspicuous posts, he argued, but applied to thousands of positions out of the public eye it would quickly become a cloak for arbitrary favoritism [13]. The party machine, he warned, would try either to capture the examinations themselves or to colonize the posts left outside them [14]. Advocates of competitive entry to military schools reasoned the same way: patronage combined with a pass examination drags the average ability of those selected down toward the minimum required, whereas competition pulls it up toward the maximum of all who apply . The ranked list was thus not only a measure of merit. It was a device for eliminating the discretion that survives once a threshold has been cleared, which is exactly the function the Venetian draw had performed.

The reform also had to defend its democratic credentials. Its opponents claimed that making appointments depend on examinations would remove public offices from the reach of the people and breed an office-holding aristocracy [15]. Schurz replied with statistics, insisting that the fear of a service monopolized by the college-educated had been refuted by the record [16]. The exchange inverts Aristotle's scheme in an instructive way, across more than two thousand years: where the ancient philosopher saw preference among candidates as the oligarchic element , the reformers presented selection by merit as democratic because access to the test was open . In Aristotle's own terms, the merit system was a mixture, combining a democratic disregard of birth and connection with an oligarchic principle of preference.

Even sympathetic observers did not hide the experimental character of the enterprise. Alexander Bain, writing in 1877, described competitive examinations for the public service as one of those radical innovations that might lead to great consequences, while acknowledging that many still hesitated over whether it would produce good or evil . The examination entered public life not as a settled science of merit but as a bet against patronage, with its own costs left to be discovered.

Doubts About the Measure

The first critiques targeted the hidden premise of the bet. An anonymous contributor to the Popular Science Monthly in 1877 observed that, before anyone could be examined, someone had to fix the standards by which candidates would be judged, and that the whole policy rested on those preliminary standards [17]. A measure is only as sound as the definition of merit it encodes. Another writer in the same journal, twelve years later, went further: people trusted to govern must be trusted to choose capable agents, and if examinations were needed to restrain jobbery, he preferred to fight jobbery directly rather than distort education to reward the best-crammed candidates [18]. Both accepted the diagnosis of patronage while doubting that measurement was the right cure.

A second line of criticism concerned what the test actually selects. A text of 1901 states plainly that no reliable device for catching capacity had been invented: the machinery lets go some it should keep and keeps some it should let go, and it tends to reward the smartest, who are sometimes the shallowest [19]. Bertrand Russell, in 1918, objected that competitive scholarships teach the very young to value knowledge for its usefulness in examinations rather than for its intrinsic interest [20]. The defenders of competitive entry to military schools had already been forced to answer the charge that examinations reward the best instructed rather than the most capable, and they could only reply that a properly conducted competition need not produce that result [21]. Another essayist suggested why such instruments spread nonetheless: methods built like machines make results independent of personal endowment and supply accredited reputations at moderate cost [22]. The appeal of measurement lay in its reliability and cheapness rather than in its precision.

Schurz himself conceded more than his polemics suggest. He granted that a good title to promotion might rest on practical efficiency and executive ability, qualities whose evidence an examination in the ordinary sense cannot supply [23]. His insistence that competition be supplemented by probation points in the same direction . The examination, even in the eyes of its most determined champion, measured some things and not others.

The decisive admission came from a textbook of civics. William Bennett Munro acknowledged in 1922 that the merit system does not always pick out the best among applicants and that no system of competitive tests is infallible, while insisting that civil service tests do succeed in weeding out the unfit and in protecting the service against appointments based on nothing but political influence . The claim is precise: the examination is trustworthy as a threshold and doubtful as a ranking. Leslie M. Shaw, in a book whose title betrays its polemical intent, went further, calling the belief that civil service examinations secure the most efficient a serious delusion, and noting that the three names certified for each vacancy were unlikely to be the three best qualified [24].

This left the reformers in a dilemma. If the fine ranking above the threshold was unreliable, the obvious alternative was to let appointing officers choose freely among all who had passed, and Carl Russell Fish recorded in 1905 that bills of exactly this kind were regarded by the friends of reform as attempts to cripple the merit system [25]. Schurz had explained why: discretion among the qualified is where favoritism returns . Between a ranking that pretends to a precision it lacks and a discretion that reopens the door to patronage, the ancient pairing of a filter with a draw offers a third path . None of the reformers in these texts proposes it; the logic of their own admissions nonetheless points toward it.

Scarce Prizes and Overburdened Judges

The same structure reappears wherever a public body distributes scarce prizes. In the allocation of research money, peer review long held the position the competitive examination once held in the civil service: it became the undisputed method of decision and acquired legitimacy both among scientists and among the wider public [26]. Yet the conditions of its exercise changed. The success rate of grants from the American National Institutes of Health fell from 34% in 2001 to 19% in 2012, and rates below 25% were common among public funders elsewhere [27]. When most applicants clear the bar of competence and only a fraction can be funded, reviewers are asked to perform exactly the fine discrimination Munro had doubted among civil servants .

The costs of such competition echo the 19th-century complaints with striking fidelity. Martin Weller describes researchers spending months preparing bids, a waste that goes unnoticed because academics have been enculturated into it [28]. More than a century earlier, the critic of 1889 had protested that examinations distorted the education they were meant to assess ; in both cases the test reshapes the activity it claims merely to measure. An executive order on federal hiring makes a parallel observation about the civil service itself: the competitive process, even at entry level, had come to favor applicants with substantial prior experience and to erect a barrier against students and recent graduates [29]. Like the objection about the best instructed , it shows a measure acquiring biases of its own.

The remedies proposed in these texts remain within the paradigm of the assessor. Advocates of reform in grant review call for greater transparency, so that researchers and the public can understand how public money is spent and trust the integrity of the process [30]. That answer mirrors Schurz's reliance on independent boards and public records rather than questioning the ranking itself . The older alternative stays implicit: Addison's observation that, among claims of equal standing, no reason can justify preferring one ticket to another , and the Venetian practice of entrusting the final choice to chance once incapacity had been guarded against . The corpus does not document a revival of that remedy. It documents the conditions under which the remedy becomes thinkable.

✻

Across these texts a single adversary persists: the personal will that bends public choices toward friends, factions or flatterers. Each age armed itself with a different instrument, and each instrument carried its own flaw. Election rewarded the skill of pleasing ; the lot ignored differences of merit ; the competitive examination claimed to rank what it could only sort . The ancient republics combined screening with the draw so that chance would neutralize faction while scrutiny kept out the incapable . The reformers of the 19th century chose the ranked list because they feared discretion more than imprecision , and their critics accepted the diagnosis while contesting the cure .

The question these sources leave open is not whether chance is better than judgment, but where judgment stops being reliable. Munro placed that limit precisely at the threshold that separates the fit from the unfit . Above it, the choice lies between three options that the long history of republics has already tried: a ranking that overstates its accuracy, a discretion that invites patronage, or a draw among the qualified that confesses its ignorance openly . The old republics bet that a confessed ignorance was more honest than a pretended precision. Their bet deserves to be judged on its merits rather than dismissed as an archaic superstition.