OpenTally/docs/validation.md

# Validation

STV-counting software is frequently validated empirically by comparing the results of election counts to those generated by independent implementations. See, for example, [[1–5]](#references). The table describes the empirical validation performed on OpenTally to date.

| Method | Election | Comparator | Included test case |
|-|-|-|-|
| Scottish STV | [2007 Glasgow council Linn ward election](https://web.archive.org/web/20121004213938/http://www.glasgow.gov.uk/en/YourCouncil/Elections_Voting/Election_Results/ElectionScotland2007/LGWardResults.htm?ward=1&wardname=1%20-%20Linn) | eSTV 2.0.16 (official) | ✓ |
| OpenTally Meek | [Reverse engineered ballots for the ERS97 model election](https://yingtongli.me/blog/2021/01/04/ers97.html) | [Algorithm 123](https://www.dia.govt.nz/diawebsite.NSF/Files/meekm/%24file/meekm.pdf) | ✓ |
| Meek STV (2006) | Reverse engineered ballots for the ERS97 model election | [OpenSTV 1.7](https://github.com/Conservatory/openstv) | ✓ |
| Meek STV (New Zealand) | Reverse engineered ballots for the ERS97 model election | OpenSTV 1.7, [Hill's nzmeek 6.7.7](https://yingtongli.me/blog/2021/07/08/nzmeek.html) | ✓ |
| Australian Senate STV | [2019 Tasmanian Senate election](https://results.aec.gov.au/24310/Website/SenateDownloadsMenu-24310-Csv.htm) | EasyCount (official) | ✓ |
| Australian Senate STV | [2019 NSW Senate election](https://results.aec.gov.au/24310/Website/SenateDownloadsMenu-24310-Csv.htm) | EasyCount (official) | |
| Australian Capital Territory STV | [2020 Kurrajong Legislative Assembly election](https://www.elections.act.gov.au/elections_and_voting/2020_legislative_assembly_election/ballot-paper-preference-data-2020-election) | [eVACS 2020](https://www.elections.act.gov.au/elections_and_voting/electronic_voting_and_counting) (official) | ✓ |
| NSW Local Government STV | [2021 City of Albury Council election](https://pastvtr.elections.nsw.gov.au/LG2101/albury/councillor) | PRCC Vote Count (official) | ✓ |
| Victorian Legislative Council STV | [2022 Northern Metropolitan Region Legislative Council election](https://www.vec.vic.gov.au/results/state-election-results/2022-state-election-results/results-by-region/northern-metropolitan-region-results) | Results sheet (official) | ✓ |
| Minneapolis STV | [2009 Minneapolis Board of Estimate & Taxation election](https://vote.minneapolismn.gov/results-data/election-results/2009/bet/) | Results sheet (official) | ✓ |
| Minneapolis STV | [2013 Minneapolis Parks & Recreation Commissioner At Large election](https://vote.minneapolismn.gov/results-data/election-results/2013/park-board-at-large/) | Results sheet (official) | ✓ |
| Minneapolis STV | [2021 Minneapolis Board of Estimate & Taxation election](https://vote.minneapolismn.gov/results-data/election-results/2021/bet/) | Results sheet (official) | ✓ |
| Minneapolis STV | [2021 Minneapolis Parks & Recreation Commissioner At Large election](https://vote.minneapolismn.gov/results-data/election-results/2021/park-board-at-large/) | Results sheet (official) | ✓ |
| Cambridge STV | [2003 Cambridge City Council election](https://web.archive.org/web/20070204083508/http://stv.sourceforge.net/) | OpenSTV 1.7, [ChoicePlus Pro 2.1](https://www.votingsolutions.com/cpdetail.htm) (official) | ✓ |
| Dáil Éireann STV | [2002 Dublin North election](https://electionsireland.org/counts.cfm?election=2002&cons=96) | Results sheet (official) | ✓ |
| van der Craats (‘Wright’) STV | [EVE Online CSM 15 election](https://www.eveonline.com/news/view/meet-the-new-council) | [ccp-wright-stv](https://github.com/ccpgames/ccp-wright-stv) (official) | ✓ |
| PRSA 1977 | [*Proportional Representation Manual*](https://www.prsa.org.au/publicat.htm#p2) [example 1](https://www.prsa.org.au/utopiatc.pdf) | [Model result](https://www.prsa.org.au/example1.pdf) (official) | ✓ |
| PRSA 1977 | 40 elections from [stvdb](https://gitlab.com/RunasSudo/stvdb) | [count.nl (RunasSudo version)](https://gitlab.com/RunasSudo/prsa_count) | ✓ |
| ERS97 | [Reverse engineered ballots for the ERS97 model election](https://yingtongli.me/blog/2021/01/04/ers97.html) | [Model result](https://www.electoral-reform.org.uk/latest-news-and-research/publications/how-to-conduct-an-election-by-the-single-transferable-vote-3rd-edition/#sub-section-24) (official) | ✓ |
| ERS97 | [Joe Otten/eSTV ballots for the ERS97 model election](https://web.archive.org/web/20020606014623/http://estv.otten.co.uk/) | [Model result](https://www.electoral-reform.org.uk/latest-news-and-research/publications/how-to-conduct-an-election-by-the-single-transferable-vote-3rd-edition/#sub-section-24) (official) | ✓ |
| ERS76 | Ballots adapted from Joe Otten/eSTV ERS97 | Model result (official) | ✓ |
| Church of England | Joe Otten/eSTV ballots for the ERS97 model election | [eSTV 1.47](https://web.archive.org/web/20040607021930/http://www.electoral-reform.org.uk/votingsystems/estv.htm) | ✓ |

# References

1. Wichmann BA. Checking two STV programs. *Voting Matters*. 2000 Apr; (11): 6–8. <http://www.votingmatters.org.uk/ISSUE11/P4.HTM>
2. Wichmann BA. Validation of implementation of the Meek algorithm for STV. London: McDougall Trust; 2000 Apr 28. <http://www.votingmatters.org.uk/RES/MKVAL.pdf>
3. Koopman P, Hubbers E, Pieters W, Poll E, de Vries R. Testing the eSTV program for the Scottish local government elections. Nijmegen (NL): Radboud University; 2007 Mar 30. <https://research.utwente.nl/en/publications/testing-the-estv-program-for-the-scottish-local-government-electi>
4. Conway A, Blom M, Naish L, Teague V. An analysis of New South Wales electronic vote counting. *ACSW '17: Proceedings of the Australasian Computer Science Week multiconference*. New York: Association for Computing Machinery; 2017 Jan. [doi: 10.1145/3014812.3014837](http://doi.org/10.1145/3014812.3014837)
5. Abate P, Dawson J, Goré R, Gray M, Norrish M, Slater A. *Formal methods applied to electronic voting systems*. Canberra: Australian National University; c2003. <https://users.cecs.anu.edu.au/~rpg/EVoting/>