[E-voting] The IES election management system

Adrian Colley aecolley at spamcop.net
Wed Jul 5 12:24:42 IST 2006


The CEV2 sections on the software in the voting machine and on the
software in the count PC both have long lists of defects.  To people
without much programming experience, it might appear that they're both
criticised equally.  In fact, the voting-machine software gets a
generally clean bill of health; the only criticisms of the code are
focused on the style and process of creating the code.

In contrast, the count PC ("election management system") software is
savaged in a diplomatic sort of way.

p. 104: "Analysis of the design and development processes has
identified little evidence to support a claim that the software is
dependable and little documentation to support this claim was
available.  Significant areas where the documentation is incomplete
are the design processes and evidence of testing."
While it isn't unusual to lack evidence of a design process, it's very
strange to lack evidence of testing.

"The behaviour of the code does not appear to be consistent.  For
example potential divide by zero operations are not all protected; the
protections applied are not all consistent and in some cases appear to
have been applied at a later date as a batch job that may not have
been completely thought out."
This says that someone ran a global search-and-replace on the source
code looking for divisions and adding "protection" (which probably
means adding a check for division by zero).  It implies that it was
slapped on as an afterthought, and no real analysis was done to see
where division-by-zero might happen.  The related problem of overflow
is almost certainly untested if division-by-zero got so little
attention.  (Overflow is where the result of a computation is too
large to be represented in the chosen datatype, so the computation
either stops with an error or (all too commonly) gets the wrong result
and carries on using it anyway.)

"There is some question as to the effectiveness of any version control
procedures being applied to the software development."
No doubt this is because of the large number of regressions reported
in the first report regarding the divide-and-get-remainder part of
vote transfers.  (A regression is an accidental reinstatement of
incorrect program code, causing cries of "hey, I thought we fixed that
bug already!".)

"No documentation concerning coding style appears to exist.  No
recognisable coding standard appears to have been followed."
It's no big deal that the coding style hasn't been documented; but the
second sentence is a polite way of saying that the source code is a
huge unreadable mess.

"The code does not appear to use either abstraction or separation of
concerns, which ease both understanding and future maintenance."
This means that nothing is done to reduce the complexity of the
design; instead the code is mashed into one place.  It's technically
possible to do this right (if the program is very small or very
carefully rigorously derived from a formal design), but it usually
means that it has been slapped together without regard for the lessons
learned from the last 40 years of software engineering (which I'll
bravely summarise as "complexity leads to bugs").

"The code has some questionable characteristics.  The function of its
exception handling is not clear (and an exception may cause the
database to end up in a malformed state); the statements containing
real divide operators have protection that may not be appropriate; and
some potential divide operations do not have any protection."
Ouch!  These characteristics are much worse than "questionable".  I
seem to recall an earlier (Nathean?) review which criticised the same
code for catching exceptions but ignoring them instead of recovering
from them.  This review seems to indicate that the exception handling
isn't much better.  Malformed databases are not fun; if you're lucky,
you don't get any results out of them.

p. 106: "Given the apparent immaturity of the software there is also a
requirement for evidence of changes in development procedures, full
commenting of the code, and documentation of the full software
development, maintenance and testing process."
This is the most overt criticism of the IES code, and the requirement
amounts to "back to the drawing board".

"Consequently, given that there were doubts about the design and
development process, and since several minor failings had already been
identified before detailed functional analysis took place, it is
questionable whether the integrity of this software can be
sufficiently assured for use in a real election without significant
modification, documenting, analysis and testing."
In other words: it would be easier to throw it away and start again
than to try to fix the code that was delivered.

"In these cases, it was found that in circumstances where a number of
candidates are tied and it is necessary to determine which candidate
should be eliminated, the election management software makes an
incorrect determination as to which of the tied candidates should be
selected for this purpose.  The action taken by the election
management software as a result of this determination may cause a
candidate to be incorrectly eliminated, thus affecting the outcome of
the election."
Another counting bug!

p. 107: "A further issue with the election management software was
also indicated as a result of examining the error described above.  It
was noted that, in addition to selecting the incorrect candidate for
elimination, the software makes a further error by reporting
incorrectly through its user interface that a different candidate has
been selected.  Thus the error in selection is compounded by an error
in reporting the selection."
An actual case of the IES PC displaying one thing on the screen and
doing another thing internally.  I think this should be talked up,
because it's a central problem with the DRE approach.

 --Adrian.



More information about the E-voting mailing list