A roll of the dice and off goes Colorado to audit elections in a new way

Colorado Secretary of State Wayne Williams explains what’s next after multi-colored 10-sided dice were used Friday to establish a “seed” to randomly select ballots for each county to audit. (SOS photo/Judd Choate)

A process to audit Colorado’s elections in a different manner drew national attention Friday when participants at the Colorado Secretary of State’s office plucked names from Rockies and Broncos baseball caps to see who would roll 20 colored 10-sided dice. The numbers were used to come up with a “seed” to randomly select ballots from the Nov. 7 election for the counties to audit.

From left, U.S. Election Assistance Commission chairman  Matt Masterson, Arapahoe County Clerk Matt Crane and Boulder County Clerk Hillary Hall wait for dice to be rolled, the first step in randomly selecting ballots for each county to audit during the RLA. (SOS photo)

The light-hearted ceremony kicked off work that began in 2009 when the Colorado General Assembly passed legislation requiring every county after every election to create a risk-limiting audit, a procedure that provides strong statistical evidence that the election outcome is right and has a high probability of correcting a wrong outcome.

“It was an incredibly successful first effort,” said the Secretary of State’s Dwight Shellman, the county support manager.  “I’m really proud of our team and of all the county clerks. We are already in the process of working with the clerks and interested stakeholders to collect lessons learned to make the process even better in the future.”

The Secretary of State’s office will release a report Monday on the first steps of the audit.

Before this year, Colorado counties conducted “random audits” following each election.  Random audits provide some evidence an election outcome is correct, but they do not yield the statistical level of confidence provided by RLAs, as risk-limiting audits are called. One election official likened the difference between the two audits to testing water that comes out of a faucet as opposed to testing equipment at a water treatment plant to make sure everything is OK.

Staffers with the Colorado Secretary of State at Friday morning’s risk-limiting audit events. Left to right, deputy elections director Hilary Rudy, Jessi Romero and Danny Casias, who conducted much of the training for risk-limiting audits and also extensively tested the RLA software; Melissa Polk; and IT security chief Rich Schliep. (SOS photo)

Colorado’s RLA initially stalled because the technology wasn’t available, and then the state had to adjust in 2013 to becoming a mostly all-mail ballot state. The legislature  mandated that the statewide audit be conducted after the 2017 coordinated election.

Because of that, early this year the Secretary of State’s office created a “representative group” of county clerks, subject matter experts and election integrity advocates to craft administrative rules prescribing the procedures for RLAs.  SOS staffers then developed and conducted a series of trainings statewide to help county clerks and their staffs learn how to conduct a risk-limiting audit.

After the dice were rolled, SOS staffers entered the resulting 20-digit number into the Secretary of State’s RLA software, developed by Free & Fair. The software then randomly selected certain ballots from ballots in each county. The software compared the audit boards’ reports of the voter markings to the manner in which they were interpreted by the voting machines. If there were no discrepancies between the human and machine interpretations, the audit stopped and the county could certify official election results.  But some discrepancies discovered in the first round may require the county to proceed to another round before the risk limit is satisfied and the audit can be concluded.

The progress was documented on Twitter, and The Denver Post’s John Frank wanted to know:

Weld County weighed in on its audit:

As did Mesa County:

Some counties, such as Crowley and Denver, are expected to complete the audit on Monday. Six counties that did not have any issues for the ballot and so held no elections did not participate in this year’s RLA. Neither did Jackson and San Juan counties, which hand count their ballots.

Election observers gathered at the Arapahoe County clerk’s election warehouse Friday to watch and help staffers locate and retrieve specific ballots. From left to right: clerk staffer Jonathan Layman, U.S. EAC commissioners Thomas Hicks and Matt Masterson, Colorado Secretary of State Wayne Williams and Arapahoe County Clerk Matt Crane. (SOS picture)

Among those observing the process in Colorado were Matt Masterson and Thomas Hicks, two members of the U.S. Election Assistance Commission, and EAC staffer Jerome Lovato, who until recently worked at the Colorado Secretary of State’s office on the RLA project.

Also observing was the Rhode Island Board of Elections and other Rhode Island officials. Rhode Island might implement RLAs as early as the 2018 election but must be ready by its 2020 contest. (When Rhode Island’s election director, Rob Rock, joked that he’d rather be pulling names out of Patriots hat than a Broncos hat, he was told, “Then the die would have to be deflated.“)

Stephanie Singer and Neal McBurnett of Free & Fair, the company that developed the software to conduct the audit, were present as well as several academics who developed the theory and statistical algorithms for risk-limiting audits: Philp Stark of the University of California at Berkeley, Mark Lindeman of Columbia University, and Ron Rivest of the Massachusetts Institute of Technology.

Elizabeth Steele of Colorado Common Cause, John Marion of Common Cause of Rhode Island, and Susan Greenhalgh of the Verified Voting Foundation also observed.

The first name drawn to roll the dice was election activist gadfly Harvie Branscomb, who has been hanging around Colorado elections for a while now. There was plenty of laughter and one person cracked, “Now we know it’s rigged!”