[E-voting] Bugs in the PTB report

[Adrian Colley]

That PTB (Physikalisch-Technische Bundesanstalt) report contains
some interesting descriptions of show-stopping flaws, coupled with
conclusions to the effect that they aren't serious enough to justify
stopping the show.  In
<URL:http://minds.cs.may.ie/~lovelace/E-Voting/FoI_request/PB_ESI1_IRL_v2_833-130-02.doc>
on page 8 it says:
> 1.3 The voting machine shall be designed in such a way that any
> unauthorised alterations of a technical nature will come to notice.

> The type, the version number and the individual identifier (serial)
> number (ID) of the voting machine and of the programming/reading
> unit are given on a type plate.  Every ballot module is identified
> by the individual ID-number.  The version numbers of the voting
> machine, the connection (communication) board and the five display
> boards and the ID-numbers of the voting machine and the installed
> ballot module can be displayed on, and printed out by, the voting
> machine.  This enables the polling staff to compare the type, the
> version numbers and the individual ID-numbers with the values given
> by the manufacturer in documentation (user manual) or e.g. listed
> in an authorized certificate.  These data are stated in section 2.1
> of this Test Report.

> OK

The discussion simply doesn't justify the conclusion.  There's much
talk about version and ID numbers, and nothing about detecting
fraudulent alterations.  Perhaps the general impossibility of
assuring a lack of alterations was weighing on the authors' minds.
Or perhaps they trusted that would-be vote riggers would obligingly
bump the version numbers when replacing components in the voting
machines?

> 1.4 In the case of a processor-controlled machine, any alteration
> of the installed software by an unauthorised person will be detected.

> The program version numbers and checksums of the installed software
> for the main control board, the connection (communication) board
> and the five display boards can be displayed on, and printed out
> by, the voting machine.  This enables the polling staff to compare
> these program version numbers and checksums with the values given
> by the manufacturer in documentation (user manual) or e.g. listed
> in an authorised certificate.  These data are stated in section 2.1
> of this Test Report.

> OK

As will be seen below, the checksums and version numbers are not
reliable and are easily faked.  On the back of trusting those
numbers, the PTB decided incorrectly that the criteria of mandatory
detection of unauthorised changes (hardware and software) were
satisfied.

In the "Test report - Voting machine ESI1 - Software for elections
in Ireland"
<URL:http://minds.cs.may.ie/~lovelace/E-Voting/FoI_request/Encl-2_SW_Test_Report_ESI1_.doc>
on page 8 it says:
> 1.3-1   The diagnostic software detects changes that influence
> proper functioning of the VM.
[...]
> MB: The software provides a checksum test to be carried out at the
> start-up. This test checks the ROM area in which the program is
> burned in and compares the new checksum to a default value burned
> in at the end of the ROM area. In case of differences, the voting
> machine is shut down.

> Besides that internal checksum, there is an external checksum which
> contains the internal one. This external checksum is calculated
> separately for the even and the odd memory at the start-up. It is
> displayed or printed out on request and is also part of the voting
> result printout.

All three checksums (1 inner, 2 outer) declared above are calculated
and output under the control of the mainboard program, which is
read from the EPROM being checksummed.  It should be obvious that
a malicious program could easily output the "right" checksum in
each case, and would therefore escape any detection attempt that
relies on the checksum.  It should be equally obvious that this is
a problem, as it amounts to failing to detect an intentional
unauthorised modification to the mainboard software.

And indeed the report authors realised this, as on page 11 it says:
> 1.4-1   Any alteration of the installed software by an unauthorised
> person should be detected.

> MB, CB, DB: The persistent program copies are held in ROM memory
> chips. Changes in these memory chips lead to failure in the ROM
> test (see requirement 1.3-1) followed by a shut down of the voting
> machine.

> An exchange of the ROM chips including fraudulent presentation of
> the correct checksums cannot be avoided by software but by means
> of sealing only.

Aha, "sealing"!  What this means isn't defined, but whatever it might
be clearly isn't there, as the report notes in its recommendations
(on page 56):
> 4. The voting machine needs physical sealing to prevent the exchange
> of program chips. The exchange cannot be detected by software.

So, rigging a Nedap ESI1 voting machine is a matter of opening it
and substituting compromised EPROM chips, attaching a fresh seal
of whatever kind the Department has settled on, and closing it up.
And, we know that there was no seal when the PTB tests were done.
Nonetheless, they had little hesitation in signing off on the
no-undetected-modifications criteria.

As to the software quality, another of the PTB recommendations was:

> 3. In the next software version the initialisation of local variables
> and the use of default cases in switch constructs should be improved.

Initialisation of local variables?  In C, those are always stack
variables, and every C programmer knows that stack variables are
undefined (i.e. they don't default to zero values) if used before
being initialised.  Could this really mean what it appears to mean?
On page 6 of the same document, it says:
> Potential Problems:

> The software was checked for several constructs which might lead
> to problems. In high-safety software such constructs should be
> avoided. They are usually forbidden by programming rules issued at
> the manufacturer's. Nedap uses a static analyser (lint) to detect
> a certain set of problematic constructs during development, so
> constructs from this set do not occur in the software.

> The following problems were found:
> Potentially problematic construct                     Number of occ.
>                                                        MB  CB  DB

> Local automatic variables which are not initialised    *)  *)  *)
[...]

> *) Most of the local automatic variables are not initialised.

I know what you're thinking -- "lint?!".  (Every Unix/C programmer
goes through a lint phase before realising that lint rarely catches
actual bugs but usually produces a lot of unnecessary warnings.)
It would be charming that someone's still using lint for a real
project, if it wasn't for the seriousness of the project it's being
used on.  But what I'm trying to draw attention to is the note that
most local variables are not initialised.  This could mean that the
programmers are assuming that they will default to zero, just like
static variables, and so are using the variables before assigning
to them.  Or, it could mean that local variable declarations lack
a redundant initialiser that the report authors strangely insist
on.  I think we should be told.  Someone's thinking fuzzily here,
and it isn't just me.

The same section later says:
> Compared to the overall code size (about 25000 lines of code), the
> number of occurrences of problematic constructs is very small. At
> present, no occurrence leading to a programming error ("bug") was
> found. As they may all lead to errors, when the software is maintained,
> it is recommended improving the code in the next update.

> Summary: The requirement is fulfilled. The software complies with
> the current principles of Software Engineering. It is recommended
> for the next versions avoiding certain problematic constructs.

If I understand this correctly, the dodgy code is acceptable to the
report authors because it is outweighed by the sanity of the rest
of the codebase.  In a world where a NASA probe was lost because
of a single-character error in coding its software, this is an
attitude unworthy of a software specialist.  The conclusion should
be qualified, at least, by the substitution of "almost complies"
for "complies".
<URL:http://nssdc.gsfc.nasa.gov/database/MasterCatalog?sc=MARIN1>

In <Encl-2_SW_Test_Report_ESI1_.doc> (page 5):
> 1.2-1   The software manufacturer shall have and apply an appropriate
> development model.

> The document Project Overview gives a short description of the Nedap
> life cycle model for software development. According to this
> description, each project includes the requirements analysis phase
> (results: functional requirements, technical requirements), the
> design and implementation phase (results: implementation plan,
> source code, software description, test specification), the test
> phase (results: test plan, test results, changed source code), and
> the release phase (results: update list, projected changes). According
> to this life cycle model, the documentation contains the Functional
> Specification, several Technical Specifications, the Detailed
> Software Design, the test documentation, and several update lists.

> During the software development, a tool for configuration management
> of source code (QVCS), a make utility (make), and a static analyser
> (lint) are used.

> The life cycle model used by Nedap is in keeping with the project
> size and the manpower involved in the development.

> Summary: The requirement is fulfilled. An appropriate development
> model is used at the manufacturer's.

This description clearly indicates that the long-discredited
"waterfall" model of software development was used.  This is the
model most frequently used for small projects, and it has given us
the world of unreliable software that we have all come to know (see
Hanna, M., "Farewall to Waterfalls", Software Magazine, May 1995,
pp. 38-46).  Where formal methods were justified, the developers
have instead used the most informal of methods.  The report authors
appear satisfied with this lack of process, on the ground that the
"project size" is small -- this is another way of saying that high
standards were not required because it was decided not to require
high standards.

--------
That's what I found in a single pass through the FoI'd PTB documents.
I recall Margaret's saying that one part of the code was not properly
reviewed because the reviewers couldn't understand the Dutch
comments -- where is that?  And does anyone know of any other
material carelessness by e-voting consultants that I've missed?

 --Adrian.

-- 
GPG 0x43D3AD19 17D2 CA6E A18E 1177 A361  C14C 29DB BA4B 43D3 AD19
http://user-aecolley.jini.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.stdlib.net/pipermail/e-voting/attachments/20031106/8fea2dba/attachment.bin