Report to Congressional Requesters United States General Accounting Office GAO October 2001 ELECTIONS Status and Use of Federal Voting Equipment Standards GAO-02-52 Page i GAO-02-52 Voting Equipment Standards Letter 1 Results in Brief 2 Background 4 FEC Has Developed Voting Equipment Standards but Has Not Maintained Them 6 NASED’s Process for Testing and Qualifying Voting Equipment 12 Conclusions 15 Matters for Congressional Consideration 16 Recommendations for Executive Action 16 Agency Comments and Our Evaluation 16 Appendix I Objectives, Scope, and Methodology 19 Appendix II Comments From the Federal Election Commission 21 Appendix III System Requirements and Supporting Life-Cycle Processes and Our Analysis 27 Tables Table 1: Types of Requirements and Processes Satisfied in FEC’s 1990 Voting Equipment Standards 10 Table 2: Types of Requirements and Processes Not Satisfied in FEC’s 1990 Standards but Satisfied in the Draft Standards 11 Table 3: NASED Qualified Equipment by Voting Equipment Category 15 Figures Figure 1: States That Require the Use of FEC’s Voting Equipment Standards 7 Figure 2: FEC’s Revised Standards Development Time Frame 8 Figure 3: Simplified Diagram of the Voting Equipment Standards Development and Implementation Process 13 Contents Page 1 GAO-02-52 Voting Equipment Standards October 15, 2001 The Honorable Thomas Daschle Majority Leader The Honorable Trent Lott Minority Leader United States Senate The Honorable Christopher J. Dodd Chairman The Honorable Mitch McConnell Ranking Minority Member Committee on Rules and Administration United States Senate Events surrounding the last presidential election have brought to light a number of issues about the conduct of elections that extend to the people, processes, and technology used to administer these elections. We were asked by several congressional committees and members to review aspects of elections throughout the United States. In response, we are issuing a series of reports. To date, we have issued reports on the scope of congressional authority in election administration and voting assistance to military and overseas citizens. 1 Other forthcoming reports will examine voting accessibility for people with disabilities; the factors that affected the uncounted votes in the November 2000 presidential election; and the people, processes, and technology challenges affecting the conduct and administration of elections. We are also issuing a capping report that draws upon our extensive body of work to identify the main issues and challenges confronting our nation’s election system and to delineate an analytical framework that Congress could use as it weighs the merits of various reform proposals. This report focuses on the status and use of federal voting equipment standards. These standards define minimum functional and performance requirements for voting equipment, such as accurately recording votes cast. The standards also define minimum life-cycle management processes 1Elections: The Scope of Congressional Authority in Election Administration (GAO-01- 470, March 13, 2001) and Elections: Voting Assistance to Military and Overseas Citizens Should Be Improved (GAO-01-1026, September 28, 2001). United States General Accounting Office Washington, DC 20548 Page 2 GAO-02-52 Voting Equipment Standards for voting equipment developers to follow, such as quality assurance. As agreed with your offices, our objectives were to (1) identify the Federal Election Commission’s (FEC) role regarding voting equipment and assess how well FEC is fulfilling its role and (2) identify the National Association of State Election Directors’ (NASED) process for testing and qualifying voting equipment against FEC’s standards. Details of our objectives, scope, and methodology are contained in appendix I. No federal agency has been assigned explicit statutory responsibility for developing voting equipment standards; however, the Federal Election Commission assumed this role by developing voluntary standards in 1990 for computer-based systems,2 and the Congress has supported this role with appropriations. These standards describe specific performance benchmarks, and address many—but not all—types of system requirements. In 1997, the Commission initiated efforts to evaluate the 1990 standards to identify areas to be updated and in 1999, it initiated efforts to update the standards. We reviewed available drafts of these updated standards and found they describe most—but again not all—types of system requirements. The Commission plans to issue revised standards in 2002. This update is necessary because the Commission has not proactively maintained them, thus allowing them to become out of date. According to Federal Election Commission officials, the Commission has not proactively maintained the standards because it has not been assigned explicit responsibility to do so. Without current, relevant, and complete voting equipment standards, states may choose not to follow them, resulting in the adoption of disparate standards that could drive up the cost of voting equipment and produce unevenness among states in the capabilities of their respective equipment. No federal agency has been assigned responsibility for or assumed the role of testing voting equipment against the federal standards. Instead, the National Association of State Election Directors, through its Voting Systems Committee, has assumed responsibility for implementing the federal voting equipment standards by accrediting independent test authorities,3 which in turn, test equipment against the standards. To this 2Performance and Test Standards for Punchcard, Marksense, and Direct Recording Electronic Voting Systems (January 1990). 3Independent test authorities are contractors independent from the voting equipment vendors and are responsible for testing voting equipment to ensure that they meet the Commission’s standards. Results in Brief Page 3 GAO-02-52 Voting Equipment Standards end, the committee has developed procedures to accredit the independent test authorities.4 According to the test authorities, testing is generally iterative, in which the voting equipment vendors are provided an opportunity to correct deficiencies identified during testing and resubmit the modified equipment for retesting. When testing is successfully completed, the independent test authorities notify the National Association of State Election Directors that the equipment has satisfied testing requirements.5 As of July 3, 2001, the Association had qualified 21 different voting equipment, representing 10 vendors. Because development, maintenance, and implementation of voting equipment standards are very important responsibilities, we are raising matters for congressional consideration regarding the explicit assignment of responsibility in these areas. Additionally, we are making recommendations to the Federal Election Commission’s commissioners aimed at improving its efforts to update its 1990 standards. In written comments on a draft of this report, the Commission stated that it generally agreed with most of our observations and recommendations, and that it agreed with our matter for congressional consideration. However, the Commission commented that it disagreed with our conclusions that it has not proactively maintained its voting equipment standards and that its draft updates to those standards do not satisfy security certification and quality assurance requirements. While the Commission deserves credit for recognizing the need over a decade ago for voting equipment standards and developing and updating them, despite a lack of explicit statutory responsibility to do so, we disagree that its efforts to update its 1990 standards have been proactive. Nine years elapsed from the time the Commission issued the standards and the time the Commission began updating them; this is too long a period between updates given advances in the field and is the major reason that the current standards are out of date. Further, while we agree that the draft standards address quality assurance, we are concerned that they omit 4According to the National Association of State Election Directors, accreditation signifies formal recognition that the independent test authority possesses or will acquire the competence to design and perform specific test methods applicable to voting equipment hardware and software and that the test authority has adequately demonstrated its competence for voting equipment testing. 5Independent test authorities notify the Election Center, which serves as the National Association of State Election Directors’ Secretariat and maintains the list of the association’s approved voting equipment. Page 4 GAO-02-52 Voting Equipment Standards essential aspects of quality assurance, such as quality assurance plans and process reviews. Regarding the draft standards satisfaction of security requirements, subsequent to commenting on a draft of this report, the Commission provided us additional draft standards, which address the requirements for security certification. Therefore, we have modified this report to recognize this new information. Last, we acknowledge the Commission’s position, as stated in its comments, for not including certain quality assurance standards areas in the revised draft standards was the result of deliberative and collaborative interaction among NASED and Commission staff, and we have modified this report to reflect this change. In 1975 Congress created the FEC to administer and enforce the Federal Election Campaign Act. To carry out this role, FEC discloses campaign finance information, enforces provisions of the law such as limits and prohibitions on contributions, and oversees the public funding of presidential elections. Within FEC, the Office of Election Administration (OEA) serves as a national clearinghouse for information regarding the administration of federal elections. As such, OEA assists state and local election officials by developing voluntary voting equipment standards, responding to inquiries, publishing research on election issues, and conducting workshops on all matters related to election administration. In addition, it answers questions from the public and briefs foreign delegations on the U.S. election process, including voter registration and voting statistics. FEC consists of six voting members, appointed by the President and confirmed by the Senate. To encourage nonpartisan decisions, no more than three commissioners can be members of the same political party, and at least four votes are required for most official Commission actions. FEC’s budget for fiscal year 2001 is $40.4 million, and of that amount, $804,000 is allocated to support OEA functions. FEC has 357 full-time staff, of which 5 are allocated to OEA functions. The voting methods used in the United States can be placed into five categories: paper ballots, mechanical lever machines, punch cards, optical scan, and direct recording electronic. The last three methods use Background Voting Equipment Used in the United States Page 5 GAO-02-52 Voting Equipment Standards computer-based equipment. Three of the five—paper ballots, punch cards, and optical scan—use some kind of paper ballot to record voters’ choices.6 • Paper Ballot. Voters use a paper ballot listing the names of the candidates and issues and record their choice by placing a mark in a box next to the candidate’s name or issue. After making their choices, the ballots are dropped into a sealed ballot box to be manually tabulated. • Mechanical Lever. Voters pull a lever next to the candidate’s name or issue and the machine records and tallies the votes using a counting mechanism. Write-in votes must be recorded on a separate document. Election officials tally the votes by reading the counting mechanism totals on each lever voting machine. • Punch Card. Voters can use one of two basic types of punch cards— Votomatic or Datavote. In both instances, voters use a computer-readable card to cast their vote. The Votomatic uses a computer-readable card with numbered boxes that correspond to a particular ballot choice. The choices corresponding to those numbered boxes are indicated to the voter in a booklet attached to a vote recording machine, with the appropriate places to punch indicated for each candidate and ballot choice. The voter uses a simple stylus to punch out the box corresponding to each candidate and ballot choice. In the Datavote, the names of the candidates and issues are printed on the card itself—there is no ballot booklet. The voter uses a stapler-like punching device to punch out the box corresponding to each candidate and ballot choice. To tally the votes in both instances, the ballots are fed into a computerized tabulation machine that records the vote by reading the holes in the ballots. • Optical Scan. Voters use a computer-readable paper ballot listing the names of the candidates and issues. The voters record their choices by using an appropriate writing instrument to fill in a box or oval, or complete an arrow next to the candidate’s name or issue. The ballot is then fed into a computerized tabulation machine, which senses or reads the marks on the ballot, and records the vote. • Direct Recording Electronic. Voters use a ballot that is printed and posted on the voting machine or displayed on a computer screen listing the names of the candidates and issues. Voters record their choices by pushing a button or touching the screen next to the candidate’s name or issue. When a voter is finished, the vote is submitted by pressing a vote button, which 6A more detailed description of each type of voting method and the associated equipment can be found in Elections: Perspectives on Activities and Challenges Across the Nation (GAO-02-03, October, 2001). Page 6 GAO-02-52 Voting Equipment Standards stores the vote in a computer memory chip. Election officials tally the votes by reading the votes totaled on each machine’s computer chip. While neither FEC nor any other federal agency has explicit statutory responsibility to develop voting equipment standards, the Congress has appropriated funds for FEC to develop and update the standards. FEC first issued voting equipment standards in 1990. These standards identify minimum functional and performance requirements for punch card, optical scan, and direct recording electronic voting equipment, and specify test procedures to ensure that the equipment meet these requirements.7 The functional and performance requirements address what voting equipment should do and delineate minimum performance thresholds, documentation provisions, and security and quality assurance requirements. The test procedures describe three stages of testing: qualification, certification, and acceptance. According to FEC’s standards document: • Qualification testing is the process by which a voting equipment is shown to comply with the requirements of its own design specification and with the requirements of FEC standards. • Certification testing, generally conducted by individual states, determines how well voting equipment conform to individual state laws and requirements. • Acceptance testing is generally performed by the local jurisdictions procuring voting equipment and demonstrates that the equipment, as delivered and installed, satisfies all the jurisdiction’s functional and performance requirements. The standards are voluntary; states are free to adopt them in whole, in part, or reject them entirely. To date, 38 states require that voting equipment used in the state meet FEC standards either in total or in part.8 Figure 1 shows these states. 7The FEC standards address only computer-based systems; therefore, the standards do not address the paper and mechanical lever machine voting methods. 8This includes the District of Columbia. Four of the 38 states reported that they followed the FEC standards in part. FEC Has Developed Voting Equipment Standards but Has Not Maintained Them Page 7 GAO-02-52 Voting Equipment Standards Figure 1: States That Require the Use of FEC’s Voting Equipment Standards Source: GAO survey results. In September 1997, FEC initiated efforts to evaluate its voting equipment standards and identify areas to be updated, and in July 1999, FEC initiated efforts to revise the standards.9 As part of this revision, FEC has been working closely with state and local election officials and vendors to incorporate industry comments on the draft standards. FEC plans to issue the revised standards in multiple volumes: volume I is to include the 9FEC has contracted with American Management Systems to support FEC in revising the voting equipment standards. Page 8 GAO-02-52 Voting Equipment Standards functional and performance requirements for voting equipment; volume II is to provide the detailed test procedures, including information to be submitted by the vendor, tests to be conducted to ensure compliance with the standards, and the criteria to be applied to pass the individual tests. Figure 2 depicts FEC’s time frames for revising the standards. Figure 2: FEC’s Revised Standards Development Time Frame Source: FEC data. Organizations such as the Department of Defense and the Institute of Electrical and Electronics Engineers have developed guidelines for various types of systems requirements and for the processes that are important to managing the development of any system throughout its life cycle. These types of systems requirements and processes include, for example: • Security and Privacy Protection. Requirements defining the security/privacy environment, types of security needed (e.g., data confidentiality and fraud prevention), risks the system must withstand, safeguards required to withstand those risks, security/privacy policies that must be met, accountability (i.e., audit trails), and criteria for security certification.10 10Security certification is the technical evaluation that establishes the extent to which a computer system, application, or network design and implementation meets specified security requirements. FEC’s Voting Equipment Standards Address Most Key Types of Systems Requirements Page 9 GAO-02-52 Voting Equipment Standards • Human Factors. Requirements defining the usability of the system, including considerations for human capabilities and limitations, and the use and accessibility of the system by persons with disabilities. • Documentation. Processes for recording information produced during the system development life cycle, which includes identifying documents to be produced; identifying the format, content, and presentation items for each document; and developing a process for reviewing and approving each document. • Configuration Management. Processes to establish and maintain the integrity of work products through the system development life cycle, including developing a configuration management plan, identifying work products to be maintained and controlled, establishing a repository to maintain and control them, evaluating and approving changes to the work products, accounting for changes to the products, and managing the release and delivery of products. • Quality Assurance. Processes to provide independent verification of the requirements and processes used to develop and produce the system, which include developing a quality assurance plan, determining what system development product and process standards are supposed to be followed, and conducting reviews to ensure that the product and process standards are followed. While FEC’s 1990 standards satisfy most of these areas, they do not satisfy all. For example, in the area of security, the standards do not address the security/privacy environment in which the voting equipment must operate, the types of security to be provided, the risks the equipment must withstand, the security/privacy policies that must be met, or the criteria for security certification. Further, the standards do not specify requirements for voting equipment usability, taking into account human capabilities and limitations, or the use and accessibility of the voting equipment by persons with disabilities. Table 1 summarizes the types of requirements and processes satisfied in FEC’s 1990 voting equipment standards. Page 10 GAO-02-52 Voting Equipment Standards Table 1: Types of Requirements and Processes Satisfied in FEC’s 1990 Voting Equipment Standards Types of requirements and processes Satisfied in the 1990 standards? Functional and performance Yes System quality Yes Security and privacy protection No Human factors No System external interface Yes Installation-dependent data Yes System environment Yes Design constraints Yes Physical Yes Safety Yes Personnel Yes Training Yes Logistics Yes Documentation No Configuration management No Quality assurance No Source: GAO analysis. As part of FEC’s current effort to revise the 1990 standards, it has made improvements in all five of the areas in which we identified missing types of requirements and processes. For example, in the area of human factors, the draft standards now include requirements for the use and accessibility of voting equipment by persons with disabilities.11 Further, for documentation, the draft standards include requirements for identifying documents to be produced; defining the format, content, and presentation items for each document; and developing a process for reviewing and approving each document. In addition, in the area of security, the standards now address security types, risks, safeguards, policies, accountability, and certification. While FEC has made improvements, the draft standards do not satisfy two areas—human factors and quality assurance. For example, in the area of human factors, the draft standards do not address requirements for equipment usability, including considerations for human capabilities and 11Our review of these standards did not include validating that the requirements are correct and complete, beyond determining whether the standards addressed all the requirements and process elements. Page 11 GAO-02-52 Voting Equipment Standards limitations. Finally, the draft standards do not yet specify the development of a quality assurance plan or the performance of quality assurance reviews to ensure that the equipment development process requirements are being met. Table 2 summarizes the types of requirements and processes not satisfied in FEC’s 1990 voting equipment standards but satisfied in the draft standards. Appendix III provides a detailed description of the requirements and process types and our complete analysis of FEC’s 1990 voting standards and draft standards. Table 2: Types of Requirements and Processes Not Satisfied in FEC’s 1990 Standards but Satisfied in the Draft Standards Type of requirements and processes Satisfied in the draft standards? Security and privacy protection Yes Human factors No Documentation Yes Configuration management Yes Quality assurance No Source: GAO analysis. In the area of quality assurance, FEC stated in its written comments on a draft of this report that its decision to not include quality assurance process reviews in the revised standards was the result of deliberative and collaborative interaction among NASED’s Voting System Committee and FEC staff. In addition, FEC did not include equipment usability because it was determined not to be an area of immediate concern by the election community during FEC’s evaluation of the standards to identify areas to be updated. FEC agrees that equipment usability should be addressed in the standards and has stated that it will fully do so once resources are available. Beyond this stated commitment, FEC has not established any specific plans or allocated specific resources for doing so. Until FEC addresses these missing requirements, the voting equipment standards’ value and utility will be diminished. Given the pace of today’s technological advances, standards must be proactively maintained to ensure that they remain current, relevant, and complete. Standards-setting bodies, such as the American National Standards Institute and the National Institute of Standards and Technology, require that standards be revised or reaffirmed at least once every 5 years. This is particularly important with voting equipment standards, which must respond to technological developments if they are FEC Has Not Proactively Maintained the Voting Equipment Standards Page 12 GAO-02-52 Voting Equipment Standards to be current, complete, and relevant, and are to be useful to state and local election officials in assuring the public that their voting equipment are reliable. FEC has not proactively maintained its voting equipment standards. As previously stated, FEC is only now updating the 1990 standards. Because FEC has not proactively maintained the standards, they have become out of date. Vendors are using new technology and expanding voting equipment functions that are not sufficiently covered by the 1990 standards. For example, the 1990 standards do not address election management systems, which are used to prepare ballots and programs for use in casting and tallying votes, and to consolidate, report, and display election results. According to a NASED committee representative and the Independent Test Authority (ITA) responsible for testing election management systems, the lack of adequate standards to address election management software has forced them to interpret the current voting equipment standards to accommodate the development of this new software. Further, according to these representatives, these interpretations have not been documented and formally shared with FEC. As mentioned earlier, FEC is updating its standards, and the draft standards now address election management systems. FEC officials acknowledge the need to actively maintain the standards, but state that they have not done so because they have not been assigned explicit responsibility. By not ensuring that voting equipment standards are current, complete, and relevant, states may choose not to follow them, resulting in states adopting disparate standards. In turn, this could drive up the cost of voting equipment being designed to multiple standards and produce unevenness among states in the capabilities of voting equipment. No federal agency, including FEC, has been assigned explicit responsibility for testing voting equipment against FEC standards, and no federal agency has assumed this role. Rather, NASED has assumed responsibility for implementing the standards.12 To do so, NASED established a voting systems committee, which comprises selected state and local election officials and technical advisers. This committee 12NASED, which comprises chief election officials from each state and territory of the United States, provides a forum for state election officials to share information about their duties, responsibilities, methods of operation, and suggestions for improving election laws. NASED’s Process for Testing and Qualifying Voting Equipment Page 13 GAO-02-52 Voting Equipment Standards accredits ITAs to test and qualify voting equipment against FEC standards. Figure 3 illustrates the voting equipment standards program, from the development of voting equipment standards through the testing and qualification of voting equipment. Figure 3: Simplified Diagram of the Voting Equipment Standards Development and Implementation Process Source: GAO, based on FEC and NASED information. To accredit the ITAs, the NASED committee has developed requirements and procedures, which include provisions for NASED to periodically reaccredit the ITAs and conduct on-site inspection visits, both of which Page 14 GAO-02-52 Voting Equipment Standards are important to ensuring that the accredited laboratories continue to comply with all requirements.13 To date, the committee has not reaccredited or inspected ITAs because, according to NASED committee representatives, they rely on the committee’s technical advisers’ ongoing conversations with ITA officials and the officials’ participation in committee meetings to ensure that the ITAs are fulfilling their responsibilities effectively. Currently, three ITAs are approved to test voting equipment against the FEC standards. In 1994, the NASED committee accredited Wyle Labs to test the hardware and machine-resident software components of proprietary vote cast and tally equipment. In February 2001, Metamor (previously PSINet) applied for accreditation to conduct qualification testing of vote tabulation and election management software.14 Also in 2001, SysTest applied for accreditation to conduct qualification testing of voting tabulation and election management software. While both Metamor and SysTest have been granted an interim approval to test voting equipment, NASED has not yet accredited either. To test voting equipment, voting equipment vendors submit requests for testing to the ITAs, who then prepare a test procedure. The test procedure details the software and hardware testing requirements that the voting equipment will be tested against and is based on both the FEC voting equipment standards and the vendors’ design specifications. According to ITA officials responsible for testing voting equipment, the testing process is generally an iterative one. Vendors are provided an opportunity to correct deficiencies identified during testing and resubmit the modified voting equipment for retesting. At the end of testing, the ITA completes a test report and notifies the Election Center that the voting equipment has successfully satisfied testing requirements.15 The Election 13NASED, Accreditation of Independent Testing Authorities For Voting System Qualification Testing Handbook (May 1, 1992). 14Software qualification testing does not include testing the software which is in a permanent machine resident status (e.g., programmed on a read only memory chip) nor the hardware and operating systems on which the software operates. Also, in 1997, the committee accredited Nichols Research to test software and the integration of the software with vote cast and tally equipment. However, as of December 2000, this responsibility was moved to PSINet. 15The Election Center is a nonprofit organization dedicated to training local election officials in election administration and serves as the NASED voting systems committee secretariat, assisting NASED in implementing national voting equipment standards. Page 15 GAO-02-52 Voting Equipment Standards Center then assigns a NASED number to the specific equipment model and firmware release that was tested and maintains the list of qualified voting equipment.16 Each time a vendor issues a new model or software release, the vendor is to submit a request for testing to the ITAs in order to qualify the new model or release. As of July 3, 2001, NASED had qualified 21 models of voting equipment and 7 election management systems, representing 10 vendors. See table 3 for a breakout of the types of equipment qualified. Table 3: NASED Qualified Equipment by Voting Equipment Category Equipment category Number qualified Punch card 1 Optical scan 6 Direct recording electronic 14 Election management systemsa 7 Total 28 aNot specifically a separate voting equipment category. Rather these systems support other voting equipment categories by preparing ballots and programs for use in casting and tallying votes, and consolidating, reporting, and displaying election results. Source: NASED data, as of July 3, 2001. The ITAs stated that the testing process generally takes about 2 to 3 months. This is contingent, however, upon the vendors having the proper documentation in order. If documentation is missing or incomplete, the process may take longer. According to the ITAs, the cost of qualification testing ranges from $40,000 for vote cast and tally equipment to $75,000 for vote tabulation and election management software. While not explicitly provided for in legislation, FEC and NASED have assumed and are performing important roles by developing voting equipment standards and testing and qualifying equipment against these standards, respectively. Given the current pace of technological change for voting equipment, the degree to which these standards are actively maintained and the extent to which they are appropriately applied, can have a direct bearing on the capabilities of voting equipment. This, in turn, can affect the successful conduct of national, state, and local elections. Therefore, it is important that responsibility for these roles be clearly 16Firmware is software that is embedded in a hardware device that allows reading and executing the software, but it does not allow modification. Conclusions Page 16 GAO-02-52 Voting Equipment Standards assigned. By doing so, the appropriate federal role in these important areas can be deliberated, decided, and explicitly defined, thereby avoiding another situation where the standards are allowed to become out of date. It is also important that these roles be executed effectively. In the case of FEC’s ongoing update of the standards, this means that requirements for equipment usability, and quality assurance should be developed. As part of the ongoing debate and deliberation over election reform in general, and the federal role in voting equipment standards in particular, the Congress may wish to consider assigning explicit federal authority, responsibility, and accountability for voting equipment standards, including proactive and continuous update and maintenance of the standards. Given that no federal or state entity has been assigned explicit authority or responsibility for testing voting equipment against the FEC standards, the Congress may wish to consider what, if any, federal role is appropriate, regarding implementation of the standards, including the accreditation of ITAs and the qualification of voting equipment. To improve the quality of FEC’s voting equipment standards, we recommend that the FEC Commissioners direct the OEA Director to accelerate the development of requirements for equipment usability, including considerations for human capabilities and limitations. To improve the quality of FEC’s current efforts to update the voting equipment standards, we also recommend that the FEC Commissioners direct the OEA Director to develop requirements for quality assurance, including developing a quality assurance plan and conducting quality assurance process reviews. In its written comments on a draft of this report (reprinted in appendix II), the FEC Chairman and Vice Chairman stated that FEC generally agrees with most of our observations and recommendations, including that human factors are not being addressed in the revised voting equipment standards and that FEC needs to accelerate their development in future iterations of the standards. Additionally, FEC agreed with our matter for congressional consideration. Nevertheless, FEC commented that it was concerned with the report’s portrayal of the Commission as being insufficiently proactive in revising voting equipment standards, stating that its efforts have been as timely as Matters for Congressional Consideration Recommendations for Executive Action Agency Comments and Our Evaluation Page 17 GAO-02-52 Voting Equipment Standards possible given certain practical constraints, which it described in a chronology of events and circumstances. FEC also commented that it disagrees with the draft report’s characterization of the Commission’s ongoing efforts to update security and quality assurance standards as incomplete, describing how both areas are being addressed. Subsequent to providing us with its written comments on a draft of this report, FEC also provided us with additional draft standards that address security requirements. Accordingly, we have modified this report, including our recommendations, to reflect this new information. We do not agree with either of FEC’s other two points of concern. Regarding FEC’s concern with the report’s portrayal of the Commission as being insufficiently proactive in revising voting equipment standards, FEC states in its comments that 7 years elapsed from the time that the standards were first issued in 1990 to the time that FEC first began evaluating them to identify areas that needed to be updated. Further, it states that another 2 years elapsed between the time FEC began evaluating the standards and the time it began updating them. We recognize that FEC is performing, through its own initiative, an important role in developing and updating the standards, and deserves credit for doing so. However, in our view, allowing 9 years to pass before beginning to update the standards, regardless of the practical circumstances that FEC cites, is too long and does not constitute a proactive maintenance process and is the primary reason that the current standards are out of date. Regarding FEC’s disagreement with the report’s characterization of the Commission’s ongoing efforts to update quality assurance standards as incomplete, we do not question, and in fact state in this report, that the draft standards address requirements for quality assurance. However, our main concern is that important and relevant aspects of quality assurance requirements, such as quality assurance plans and process reviews, respectively, are not addressed. Concerning FEC’s decision to omit quality assurance standards areas from the revised draft standards, we modified this report to reflect FEC’s position that its decision resulted from deliberative and collaborative interaction among NASED and FEC staff and was not, as we were told during the course of our review by the OEA Director, areas that were overlooked. We are sending copies of this report to the Chairmen and Ranking Minority Members of the Senate Appropriations Subcommittee on Treasury and General Government and the House Appropriations Page 18 GAO-02-52 Voting Equipment Standards Subcommittee on Treasury, Postal Service, and General Government; the Director of the Office of Management and Budget; and the Chairman and Vice Chairman of FEC. Copies will also be available at our Web site at www.gao.gov. If you have any questions, please contact me at (202) 512- 6240 or by email at hiter@gao.gov. Key contributors to this assignment were Deborah A. Davis, Richard Hung, and Eric Winter. Randolph C. Hite Director, Information Technology Systems Issues Appendix I: Objectives, Scope, and Methodology Page 19 GAO-02-52 Voting Equipment Standards The objectives of our review were to (1) identify Federal Election Commission’s (FEC) role regarding voting equipment and assess how well FEC is fulfilling its role and (2) identify the National Association of State Election Director’s (NASED) process for testing and qualifying voting equipment against FEC’s voluntary voting equipment standards. To identify FEC’s role regarding voting equipment, we researched FEC’s statutory and legislative role in developing and maintaining voting equipment standards. To further identify FEC’s role, we reviewed relevant documents, including the Plan to Update the Voting Systems Standards,1 the standards update project contract, project work plans, and legislative proposals, and interviewed key FEC officials, including the Director, OEA. To assess FEC’s voting equipment standards, we examined relevant guidelines and procedures for the development of system requirements. Specifically, we examined the Department of Defense’s Data Item Description for System/Subsystem Specifications; the Institute of Electrical and Electronics Engineers’ Standard 12207 on Software Life Cycle Processes, and the Software Engineering Institute’s Software Development Capability Maturity Model™ 2 and identified 13 types of systems requirements and 3 supporting life-cycle processes that are important in the development of any system. We then compared these types of requirements and processes against FEC’s 1990 voting equipment standards to determine if all key elements were addressed. In those areas where variances were noted, we compared the types of requirements and processes against relevant sections of volumes I and II of the draft standards to determine whether FEC had addressed any of these missing requirements.3 We only reviewed those portions of the draft standards for which we identified missing types of requirements and processes in the 1990 standards. In addition, our review of the standards did not include validating that the requirements are correct and complete beyond determining whether the standards addressed all of the requirements and process key elements. 1ManTech Advanced Systems International, Inc., February 9, 1999. 2The Software Engineering Institute is operated by Carnegie Mellon University as a federally funded research and development center sponsored by the Department of Defense. Capability Maturity Model is a registered mark of Carnegie Mellon University. 3We examined all but two sections (hardware standards and software/firmware standards) in volume I and those sections pertaining to security and certification in volume II. Appendix I: Objectives, Scope, and Methodology Appendix I: Objectives, Scope, and Methodology Page 20 GAO-02-52 Voting Equipment Standards To identify NASED’s process for testing and qualifying voting equipment against FEC’s voting equipment standards, we interviewed officials from NASED, the Election Center, and the two independent test authorities (ITA). We also reviewed documentation describing NASED’s process, NASED’s Accreditation of Independent Testing Authorities For Voting System Qualification Testing Handbook,4 ITAs’ generic test plans, and NASED’s policies, procedures, and by-laws. We also provided a copy of relevant parts of this report to the Chairman of the NASED Voting System Committee for comment. The Chairman stated that the report accurately reflected the NASED process. We also contacted officials in the State Election Director's offices in each of the 50 states and the District of Columbia to determine which states required that their voting equipment be in compliance with FEC's standards. We did not verify the officials' responses. We performed our work at FEC headquarters in Washington, D.C., NASED, the Election Center, and the independent test authorities from March 2001 through September 2001, in accordance with generally accepted government auditing standards. 4NASED, May 1, 1992. Appendix II: Comments From the Federal Election Commission Page 21 GAO-02-52 Voting Equipment Standards Appendix II: Comments From the Federal Election Commission Note: GAO comments supplementing those in the report text appear at the end of this appendix. See comment 2. See comment 1. Appendix II: Comments From the Federal Election Commission Page 22 GAO-02-52 Voting Equipment Standards Appendix II: Comments From the Federal Election Commission Page 23 GAO-02-52 Voting Equipment Standards See comment 6. See comment 5. See comment 4. See comment 3. Appendix II: Comments From the Federal Election Commission Page 24 GAO-02-52 Voting Equipment Standards Appendix II: Comments From the Federal Election Commission Page 25 GAO-02-52 Voting Equipment Standards The following are GAO’s comments on the Federal Election Commission letter dated July 18, 2001. 1. See comments 2, 5, and 6. 2. We do not dispute either the chronology of events provided in FEC’s comments or its statement that it does not have explicit statutory authority to develop and revise the standards. We provide the relevant elements of this chronology in this report. Additionally, we state in this report that FEC has assumed and is performing an important role by developing and revising the standards, despite its lack of explicit statutory responsibility. We do not agree with FEC’s comment that it has been proactive in updating the voting equipment standards. As FEC acknowledges in its comments, 7 years elapsed from the time the standards were first issued in 1990 to the time that FEC initiated efforts to assess the standards to identify areas that needed to be updated. During that time, considerable experience with the standards was accumulating, as vendors were developing voting equipment to meet the FEC standards and ITAs were testing against them. Since then, additional experience has been gained with the standards as vendors have continued to develop voting equipment to meet the standards, and ITAs have continued to test vendors’ equipment against the standards. For example, we state in this report that ITAs have had to interpret the 1990 standards in the testing process to accommodate vendors’ use of new technologies and expanded equipment functions that are not addressed in the 1990 standards. However, FEC does not formally receive these interpretations, any one of which could be the basis for prompting an update to the standards. In our view, waiting 9 years to begin updating the standards is too long, does not constitute proactive maintenance, and is the primary reason that the current standards are out of date. 3. FEC is correct in stating that we did not assess all of the revised draft standards areas. However, we disagree that this assessment approach ignores the collaborative and dynamic process of NASED’s Voting Systems Committee and FEC’s staff in overseeing the development of the standards for two reasons. First, this report recognizes that FEC worked closely with state and local election officials in revising the standards. Second, this joint FEC and NASED process has no relevance to our findings that certain standards areas do not address GAO Comments Appendix II: Comments From the Federal Election Commission Page 26 GAO-02-52 Voting Equipment Standards the full range of items associated with well-defined system requirements in these areas. As we state in the objectives, scope, and methodology section of this report, our approach was to assess all of the 1990 standards because they are the standards against which voting equipment are currently being developed and independently tested. In assessing drafts of the updated standards, we assumed that those areas in the 1990 standards that we found to be satisfactory would continue to be satisfactory in the updated standards. As long as our findings are limited to the standards area that we assessed, the issue of whether we assessed all or some of the draft standards is not relevant. 4. We acknowledge that FEC’s position, as stated in its comments, concerning standards areas omitted from the revised draft standards is that these were based on decisions resulting from deliberative and collaborative interaction among NASED and FEC staff, and were not, as we were told during the course of our review by the OEA Director, areas that were overlooked. Accordingly, we have modified our report to reflect this position. 5. Subsequent to providing us written comments on a draft of this report, FEC provided us with a copy of volume II of the standards, which includes the tests to be conducted to ensure compliance with the voting equipment standards. Based on our review of the relevant security sections, the standards satisfy the requirement for security certification. We have modified our report, including the recommendations, to reflect this new information. 6. We do not disagree that the draft standards discuss quality assurance and have been strengthened from the 1990 standards. We acknowledge these improvements to the standards in our report. However, as we state, quality assurance includes a number of activities. While FEC’s draft standards include some of these elements, they do not include all of them. Specifically, the draft standards do not address requirements for developing a quality assurance plan and conducting process reviews to ensure that the product and process standards are followed. Appendix III: System Requirements and Supporting Life-Cycle Processes and Our Analysis Page 27 GAO-02-52 Voting Equipment Standards We identified 13 types of system requirements and 3 supporting life-cycle processes that are often associated with complete system requirements. FEC’s 1990 voting equipment standards satisfied 11 of the 13 system requirements areas and none of the life-cycle processes. We reviewed FEC’s draft standards for those areas for which we identified variances in the 1990 standards and found that the draft standards had made improvements in all five areas. However, the draft standards still do not satisfy human factors and quality assurance. A detailed description of the system requirements areas and our complete analysis follow. 1990 Draft Systems requirements Definition/analysis Standards satisfied? Functional/Performance Required system capabilities based on the purpose of the system; also includes parameters for response times, accuracy, capacities, unexpected/unallowed conditions, error-handling, and continuity of operations. 1990 analysis: Identified areas include ballot definition, candidate/measure selection, vote casting, ballot interpretation, voting reports, accuracy and integrity, processing speed, response times, and error and status messages. Yes System quality Quantitative measures of quality including reliability (perform correctly and consistently), maintainability (easily serviced/repaired/corrected), and availability (accessibility to be operated when needed). 1990 analysis: All identified areas included. Yes Security/privacy protection Requirements for maintaining a secure system and protecting data privacy, including (1) security/privacy environment in which system must operate; (2) types of security to be provided (e.g., data confidentiality and fraud prevention); (3) risks the system must withstand; (4) safeguards required; (5) security/privacy policies that must be met; (6) accountability the system must provide (i.e., audit trails); and (7) criteria for security certification. 1990 analysis: Access control identified as a security safeguard, and requirements defined for audit records produced by the system to provide accountability. The other areas, however, are not addressed. Draft analysis: In addition to access controls and audit records, the security/privacy environment, the types of security to be provided, the risks the system must withstand, safeguards necessary, security policies, and criteria for security certification are identified. No Yes Human factors Requirements defining system usability of the system that take into account human capabilities and limitations, along with use and accessibility by persons with disabilities. 1990 analysis: System usability and accessibility by persons with disabilities are not identified. Draft analysis: Requirements for the use and accessibility by persons with disabilities are identified. System usability requirements are not. No No System external interface Characteristics of the interface between the voting system and other systems, including data types, data formats, and timing. 1990 analysis: Removable storage media, communications devices, and printers identified as external interfaces. Yes Appendix III: System Rquirements and Supporting Life-Cycle Processes and Our Analysis Appendix III: System Requirements and Supporting Life-Cycle Processes and Our Analysis Page 28 GAO-02-52 Voting Equipment Standards 1990 Draft Systems requirements Definition/analysis Standards satisfied? Installation-dependent data Requirements for system configuration to meet local operational requirements. 1990 analysis: Requirements defined for voting systems programming in accordance with ballot requirements of the election and the jurisdiction in which the equipment will be used. Yes System environmental The natural environment that the system must withstand during transportation, storage, and operation, including (1) temperature, (2) humidity, (3) rain, and (4) motion/shock. 1990 analysis: Requirements identified for temperature, humidity, rain, transit drop, and vibration. Yes Design constraints Any commercial standards that must be used in the system’s development. 1990 analysis: Vendors are instructed to design equipment in accordance with best commercial and industrial practice; software is to be designed in a modular fashion, preferably using a high-level programming language. Yes Physical The system’s physical characteristics, including size, weight, color, nameplates, markings of parts and serial/lot numbers, transportability, and parts interchangeability. 1990 analysis: All requirements identified. Yes Safety Requirements for preventing or minimizing unintended hazards to personnel, property, and the physical environment. 1990 analysis: All systems shall be designed to meet the requirements of the Occupational Safety and Health Administration Yes Personnel Requirements for who will use or support the system, such as number of workstations and built-in help/training features. 1990 analysis: Vendors instructed to include information on number of personnel and skill level required to maintain the voting system. Yes Training Requirements for training devices and materials to be included with the system. 1990 analysis: Vendors instructed to document information required for system use and operator training, and orientation and training of poll workers, user maintenance technicians, and vendor personnel. Yes Logistics Requirements for system maintenance, software support, and system transportation. 1990 analysis: Vendors instructed to document information required in these three areas. Yes Appendix III: System Requirements and Supporting Life-Cycle Processes and Our Analysis Page 29 GAO-02-52 Voting Equipment Standards 1990 Draft Life-cycle process Definition/analysis Standards satisfied? Documentation The process of recording information produced during the life-cycle process. Describes and records information about a product, the processes used to develop the product, and provides a history of what happened during the development and maintenance of the product. Includes (1) identification of documents to be produced and delivered to customer or tester, (2) identification of format, content, and presentation items for each document, and (3) review and approval process for each document. 1990 analysis: Requirements identify products to be produced, including the content and format of the documents. Review and approval process not specified. Draft analysis: Products to be produced, including the content and format of the documents, as well as the review and approval process is identified. No Yes Configuration management The process to establish and maintain the integrity of work products throughout the life-cycle process; it involves establishing product baselines and systematically controlling changes to them. The process should include (1) developing a configuration management plan, (2) identifying work products to be maintained and controlled, (3) establishing a repository to maintain and control the work products, (4) evaluating and approving changes to the products, (5) accounting for changes to the work products, and (6) managing the release and delivery of them. 1990 analysis: Includes requirements for (1) identifying work products to be maintained and controlled, (2) evaluating and approving changes to the products, and (3) managing the release and delivery of work products. The standards do not include requirements for developing a configuration management plan, establishing a repository to maintain and control the work products, and accounting for changes to the work products. Draft analysis: All areas identified. No Yes Quality assurance The process that provides adequate assurance of the system development process. It typically involves independent review of work products and activities to ensure compliance with applicable development standards and procedures. The process should include (1) developing a quality assurance plan, (2) determining system development product and process standards to be followed, and (3) conducting reviews to ensure that the product and process standards are followed. 1990 analysis: None of these areas specified. Draft analysis: The need to document the hardware and software development process is specified, but a quality assurance plan and quality assurance reviews are not. No No Source: GAO analysis. (310210) The General Accounting Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. The fastest and easiest way to obtain copies of GAO documents is through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO E-mail this list to you every afternoon, go to our home page and complete the easy-to-use electronic order form found under “To Order GAO Products.” The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. General Accounting Office P.O. Box 37050 Washington, D.C. 20013 To order by Phone: Voice: (202) 512-6000 TDD: (301) 413-0006 Fax: (202) 258-4066 GAO Building Room 1100, 700 4th Street, NW (corner of 4th and G Streets, NW) Washington, D.C. 20013 Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm, E-mail: fraudnet@gao.gov, or 1-800-424-5454 (automated answering system). Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800 U.S. General Accounting Office, 441 G. Street NW, Room 7149, Washington, D.C. 20548 GAO’s Mission Obtaining Copies of GAO Reports and Testimony Order by Mail or Phone Visit GAO’s Document Distribution Center To Report Fraud, Waste, and Abuse in Federal Programs Public Affairs