Skip to main content

Elections should be grounded in evidence, not blind trust

Philip Stark, professor of statistics | January 5, 2021

Co-authored by Edward Perez, global director of technology development at the OSET Institute, and J. Alex Halderman, a professor of computer science and engineering at the University of Michigan. 

President Donald Trump’s attempt to pressure Georgia election officials to “find” votes he didn’t win is keeping election integrity in the spotlight. Tomorrow’s Senate runoffs will determine which party controls the chamber, and there’s a high likelihood that this round of voting will also be declared illegitimate by the losers. Even though there is no compelling evidence the 2020 vote was rigged, U.S. elections are insufficiently equipped to counter such claims because of a flaw in American voting. The way we conduct elections does not routinely produce public evidence that outcomes are correct.

Furthermore, despite large investments since 2016, voting technology remains vulnerable to hacking, bugs, and human error. A report by the National Academies into the 2016 election process concluded that “there is no realistic mechanism to fully secure vote casting and tabulation computer systems from cyber threats.” The existence of vulnerabilities is not evidence that any particular election outcome is wrong, but the big-picture lesson from 2020 is that ensuring an accurate result is not enough. Elections also have to be able to prove to a skeptical public that the result really was accurate.

We need evidence-based elections: processes that create strong public evidence that the reported winners really won and the reported losers really lost, despite any problems that might have occurred. Every step in election administration—from technology choices to voter eligibility checks, physical security, the canvass, and audits—should flow from that requirement.

Currently, only four states (Colorado, Nevada, Rhode Island, and Virginia) have statutory requirements for risk-limiting audits, the most rigorous type of post-election audits. But evidence-based elections are possible in all states with existing voting technology by following a few steps in a publicly transparent, verifiable manner. Here’s what an evidence-based election would look like:

  • Voters hand-mark paper ballots to create a trustworthy, durable paper vote record. Voters who cannot hand-mark a ballot independently are provided assistive technologies, such as electronic ballot marking devices. But because these devices are subject to hacking, bugs, and software misconfiguration, the use of such ballot-marking devices should be limited.
  • Election officials protect the paper ballots to ensure no ballot has been added, removed, or altered. This requires stringent physical security protocols and ballot accounting, among other things.
  • Election officials count the votes, using technology if they choose. If the technology altered the outcome, that will (with high confidence) be corrected by the steps below.
  • Election officials reconcile and verify the number of ballots and the number of voters, with a complete canvass to ensure that every validly cast ballot is included in the count.
  • Election officials check whether the paper trail is trustworthy using a transparent “compliance audit,” reviewing chain-of-custody logs and security video, verifying voter eligibility, reconciling numbers of ballots of each style against poll book signatures and other records, and accounting for every ballot that was issued.
  • Election officials check the results with an audit that has a known, large probability of catching and correcting wrong reported outcomes—and no chance of altering correct outcomes. The inventory of paper ballots used in the audit must be complete and the audit must inspect the original hand-marked ballots, not images or copies.

None of these steps stands alone. An unexamined set of paper ballots, no matter how trustworthy, provides no evidence. Conversely, no matter how rigorous, audits and recounts of an untrustworthy paper trail provide no evidence that the reported winners won. Auditing or recounting machine-marked ballots or hand-marked ballots that have not been kept secure can check whether the reported outcome reflects that paper trail, but cannot provide evidence that the reported winners won.

To conduct risk-limiting audits properly, jurisdictions must keep better track of ballots than they might otherwise. Election officials who have conducted RLAs report greater confidence in their own operations, partly because they know where every ballot is. By contrast, outsourcing audits, as Georgia did after the November vote, may prevent such process improvements. It is the responsibility of election officials (and not a third party) to ensure and demonstrate that the paper trail includes no more and no less than every validly cast ballot, and that the reported result is what those ballots show.

Moving to evidence-based elections requires public education, funding, and training for election officials. If the general voting population uses hand-marked paper ballots, Georgia and many other states have everything they need to conduct evidence-based elections except laws and regulations. Other states may need new voting equipment. Well designed, publicly owned, open-source voting technology could save taxpayers millions within a few years, alleviate concerns that private corporations are using their products and services to alter election outcomes, and speed up a national transition to efficient evidence-based elections.

A secure election gets the right outcome and convinces the public. No matter what candidate they vote for, every American should be concerned about gaps in the evidence to support the reported outcome. Official assurances without public evidence do not persuade skeptics, and a lack of public confidence in the machinery of elections is an existential threat to our democracy. Let’s make sure all states adopt evidence-based elections with hand-marked paper ballots, demonstrably secure chain of custody, thorough canvasses, compliance audits, and risk-limiting audits. That way, we all win.

Comments to “Elections should be grounded in evidence, not blind trust

  1. Casting a ballot framework respectability is a significant piece of decisions, yet different angles need our consideration, as well — like how data is created and conveyed about the applicants or measures on the polling forms, how qualification to cast a ballot is resolved and controlled, and how evenhandedly the chance and simplicity of casting a ballot is encouraged. Danger restricting reviews are not what should be highlighted in the event that we have little command over who reserves, claims, and works the methods for correspondence, and no powerful method of forestalling misuse previously. What’s more, obviously how applicants are created/prepared is likewise significant.

  2. Voting system integrity is an important part of elections, but other aspects need our attention, too — like how information is generated and distributed about the candidates or measures on the ballots, how eligibility to vote is determined and administered, and how equitably the opportunity and ease of voting is facilitated. Risk-limiting audits are not the thing that needs to be spotlighted if we have little control over who funds, owns, and operates the means of communication, and no robust way of preventing abuse beforehand. And of course how candidates are developed/groomed is also very important.

    That said, regarding voting systems, what are the best examples currently? Who is handling voting well?

    • I certainly agree that the voting system is not the only vulnerable or weaponizable part of U.S. elections. Disinformation, barriers to registration, barriers to casting a vote, and many other things can and have been used to disenfranchise targeted groups. Here’s a writeup of some of the issues in Georgia from 2018 and earlier:

      That said, we *do* know how to conduct elections in a way that produces strong evidence that the reported outcomes accurately reflect how people who did vote in fact voted. There are individual jurisdictions that do a great job, but most of the country doesn’t do that.

      This isn’t about risk-limiting audits per se. More fundamentally, it’s about (1) having a demonstrably trustworthy record of votes and (2) transparently checking the reported results against that trustworthy record. Risk-limiting audits save labor compared to a full hand count when the reported results are correct. (When the reported results are incorrect, risk-limiting audits are designed to progress to a full hand count.) But neither audits nor recounts are probative unless the paper trail they rely on is demonstrably trustworthy. In most U.S. jurisdictions, the paper trail is not demonstrably trustworthy, because of how it was created or curated.

      Regarding good examples: Colorado is doing a particularly good job as a state, but still has some gaps. For instance, Colorado does not do enough to check eligibility determinations. They audit only two contests per county, selected by the SoS. And they do not provide the public enough information to verify that the audits did not stop prematurely. Inyo County, CA, is doing an exemplary job of “soup-to-nuts” trustworthy elections, taking great pains to verify the trustworthiness of the paper trail at all stages of the process, including the canvass and the audit, and to provide public evidence. See

  3. If no high stakes election (President, Congress, Governor) in modern times has been shown
    to have been rigged/hacked, does that not establish an extremely high statistical expectation
    of accuracy which is in fact an empirical validation?

    Similarly did not the laborious hand recounts (Wisconsin, Pennsylvania, others) verify the initial results
    and hence validate the current process?

    The radical nihilists and poor losers (Donald Trump, Ted Cruz) will challenge the most absolutely
    perfect system you could design by sowing seeds of doubt without any supporting evidence.

    Numbers people like yourself, like engineers, have a very difficult time understanding how
    psychology can overturn irrefutable evidence.  For example the Sandy Hook school massacre
    is believed to be a hoax by people who have not a shard of evidence to refute the funerals and gravestones.

    The crackpots and their manipulators are an unfortunate fact of life.

    But perhaps there could be a small paper receipt like from the library self-checkout machine that
    could later be used as a hard copy verification of that person’s vote on a random audit if needed.

    But again, the poor losers like Trump & Cruz
    will dream up crazy legal cases to challenge any and all higher veracity voting and auditing methods.
    They can allege without any proof that dead people voted, that people voted multiple times,
    that the vote was not cast by the voter connected to that ballot, and so on and so forth.

    • I agree that some fraction of the electorate is impervious to evidence, and will never be convinced.

      That said, the issue we are raising is that our voting systems–as currently used–would *not* generate evidence of rigging or hacking if indeed it had occurred. The lack of evidence of rigging or fraud is not evidence that rigging or fraud did not occur, because the systems and procedures are not designed to detect problems nor to determine whether, despite whatever problems might have occurred, outcomes are correct. That’s why we need to shift from procedure-based elections to evidence-based elections.

      By analogy, our current system is like a brain surgeon saying “I used a sterile scalpel and followed proper surgical procedure, therefore, the patient is fine.” In US elections, officials in effect say, “I used certified equipment and followed the regulations, therefore the outcome is correct.” In both cases, we should be looking at the patient to assess whether the patient is in fact fine.

      In the last few years, our nation has spent hundreds of millions of taxpayer dollars (>$300MM in Los Angeles County alone; another >$100MM in Georgia; similar sums in PA) on systems that cannot generate affirmative evidence that outcomes are correct–even though they generate a “paper trail.” We should be starting from the premise that the purpose of a voting system is to provide evidence that would convince a reasonable person that the reported winners really won and the reported losers really lost. Instead, we are prioritizing things like rapid reporting and the gleam of modern technology. Absent a trustworthy, durable, tamper-evident record of the votes, no post hoc procedure can justify confidence in outcomes.

Leave a Reply

Your email address will not be published. Required fields are marked *