This is the accessible text file for GAO report number GAO-03-476 
entitled 'Defense Acquisitions: Assessments of Major Weapon Programs' 
which was released on May 15, 2003.

This text file was formatted by the U.S. General Accounting Office 
(GAO) to be accessible to users with visual impairments, as part of a 
longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov.

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately.

Report to Congressional Committees:

May 2003:

Defense Acquisitions:

Assessments of Major Weapon Programs:

GAO-03-476:

GAO Highlights:

Highlights of GAO-03-476, a report to Congressional Committees

Why GAO Did This Study:

The weapons the Department of Defense (DOD) develops have no 
rival in superiority. How they are developed can be improved, 
without sacrificing the superiority of the outcome. GAO’s reviews 
over the past 20 years have found consistent problems with weapon 
investments—cost increases, schedule delays and performance 
shortfalls—along with underlying causes, such as pressure on 
managers to promise more than they can deliver. The best 
practices of successful product developments offer a knowledge-based 
approach DOD can use to improve the way it develops new weapons. 

This report is new for GAO, and draws on its work in best practices 
for product development. GAO’s goal for this report is to provide 
congressional and DOD decision makers with an independent, 
knowledge-based assessment of defense programs that identifies 
potential risks, and offers an opportunity for action when a 
program’s projected attainment of knowledge diverges from the best 
practice. It can also highlight those programs that employ practices 
worthy of emulation by other programs. GAO plans to update and 
issue this report annually to the congressional defense committees. 

What GAO Found:

GAO assessed 26 defense programs ranging from the Marine Corps’
Advanced Amphibious Assault Vehicle to the Missile Defense Agency’s
Theater High Altitude Area Defense system. GAO’s assessments are
anchored in a knowledge-based approach to product development that
reflects best practices of successful programs. This approach centers 
on attaining high levels of knowledge in three elements of a new 
product or weapon—technology, design, and production. If a program is 
not attaining this level of knowledge, it incurs increased risk of 
technical problems, accompanied by cost and schedule growth (see 
figure). If a program is falling short in one element, like technology 
maturity, it is harder to attain knowledge in succeeding elements.

All of the programs GAO assessed proceeded with less knowledge at 
critical junctures than suggested by best practices, although several 
came close to meeting best practice standards. GAO also found that 
programs generally did not track statistical process control data, a 
key indicator for production maturity. Program stakeholders can use 
these assessments to recognize the gaps in knowledge early and to take 
advantage of opportunities for constructive intervention—such as 
adjustments to schedule, trade-offs in requirements, and additional 
funding.

GAO has summarized the results of its assessments in an easy to read 
two-page format. Each two-page assessment contains a profile of the 
product that includes a description; a timeline of development; a 
baseline comparison of cost, schedule, and quantity changes to the 
program; and a graphical and narrative depiction of how the product  
development knowledge of an individual program compared to best 
practices. Each program office submitted comments and they are 
included with each individual assessment as appropriate.

What GAO Recommends:

GAO makes no recommendations. Program office comments are included in 
the assessments of each individual program.

www.gao.gov/cgi-bin/getrpt?GAO-03-476.
To view the full report, including the scope and methodology, click on 
the link above. For more information, contact Paul Francis at
(202) 512-4841 or francisp@gao.gov.

[End of section]

Foreword:

Letter:

A Knowledge-Based Approach Can Lead to Better Acquisition Outcomes:

Observations:

Agency Comments:

Scope of Our Review:

Appendixes:

Appendix I: Assessment of Individual Programs:

Introduction:

Advanced Amphibious Assault Vehicle (AAAV):

Airborne Laser (ABL):

Advanced Extremely High Frequency (AEHF) Communications 
Satellite:

AN/APG-79 Active Electronically Scanned Array (AESA) Radar:

AIM-9X Short-Range Air-to-Air Missile:

Advanced Threat Infrared Countermeasures/Common Missile Warning System 
(ATIRCM/CMWS):

Advanced Wideband Satellite (AWS):

Cooperative Engagement Capability (CEC):

CH-47F Improved Cargo Helicopter:

RAH-66 Comanche:

EX-171 Extended Range Guided Munition (ERGM):

Excalibur Artillery Round:

F/A-18E/F Super Hornet:

F/A-22 Raptor:

Joint Air-to-Surface Standoff Missile (JASSM):

Joint Common Missile:

Joint Primary Aircraft Training System (JPATS):

F-35 Joint Strike Fighter (JSF):

Joint Standoff Weapon (JSOW):

National Polar-orbiting Operational Environmental Satellite System 
(NPOESS):

Patriot Advanced Capability 3 (PAC-3) Program:

Space Based Infrared System (SBIRS) High:

Theater High Altitude Area Defense (THAAD):

Tactical Tomahawk Missile:

V-22 Osprey:

Wideband Gapfiller Satellite (WGS) Communications System:

Appendix II: Methodology:

System Profile Assessment:

Product Knowledge Assessment:

Appendix III: GAO Contact and Acknowledgments:

Figures:

Figure 1: Knowledge Build at Key Points in Product Development Reduces 
the Risk of Unknowns:

Figure 2: Graphic Depiction of Best Practices for Technology, Design, 
and Production Knowledge:

Figure 3: Depiction of a Notional Weapon System Program's Knowledge as 
Compared with Best Practices:

Abbreviations:

DOD: Department of Defense:

GAO: General Accounting Office:

USAF: United States Air Force:

USN: United States Navy:

USMC: United States Marine Corps:


Foreword May 15, 2003:

Congressional Committees:

Recent military operations in Iraq have soundly demonstrated the 
superiority of United States military capabilities. The Department of 
Defense (DOD) develops weaponry that is unmatched in levels of 
technological sophistication and lethality. Despite their superiority, 
weapon systems routinely take much longer to field, cost more to buy, 
and require more support than investment plans provide for. In a 
constrained funding environment, unforeseen cost growth in weapon 
systems forecloses other investment choices for the government, both 
within and outside of DOD. DOD's investment in major weapon systems is 
expected to grow considerably in the future as DOD works to keep legacy 
systems while investing in future capabilities such as unmanned 
aircraft, satellite networks, and information communication systems. 
For example, the investment in weapons from fiscal years 2003 through 
2009 will exceed 
$1 trillion. Such an investment clearly requires DOD to be as efficient 
and effective as possible in the development and acquisition of weapon 
systems.

In the last several years, we have undertaken a body of work that 
examines weapon acquisition issues from a different, more cross-cutting 
perspective--one that draws lessons learned from best product 
development practices to see if they apply to weapon system 
development. We found that programs managed with a knowledge-based 
approach--where product knowledge is demonstrated at critical points in 
a development cycle--place themselves on a low-risk path to production. 
These programs are more likely to be executed within cost and schedule 
estimates. We believe that by employing this approach, DOD can still 
field superior weapons without attendant cost and schedule growth.

This report is a new product for GAO. It provides decision makers with 
a snapshot of program performance and risk and is focused on each 
system's developmental progress vis a vis best practices. Each 
assessment is summarized in an easy to read, visually descriptive 2-
page format that provides a fact-based analysis of each program's cost, 
schedule, and development status. We plan to issue this report annually 
in early spring, and we intend to increase the number of systems 
reviewed each year. We have briefed numerous committee staff on the 
product and received positive feedback regarding the report's utility 
and breath of coverage.

The continuing war on terrorism, regional instability, the challenge of 
transforming the military, as well as the federal government's short-
and long-term budget pressures have created a challenging environment 
for DOD. It faces a number of difficult missions that will put its 
strategies and resources under enormous strain. Consequently, it is 
important that weapon systems be acquired using a knowledge-based 
approach to ensure that their development is within cost, schedule, and 
performance parameters. We believe that this report can provide useful 
insights on key risks in development, allow decision makers to take 
corrective actions, and thereby place programs in a better position to 
succeed.

Signed by:

David M. Walker
Comptroller General
of the United States:

Letter May 15, 2003:

Congressional Committees:

The Department of Defense (DOD) is on the threshold of several major 
investments in programs that are likely to dominate budget and 
doctrinal debates well into the next decade. These programs include, 
among others, the Missile Defense Agency's suite of land, sea, air, and 
space defense systems; the Army's Future Combat System; and the Air 
Force and Navy's Joint Strike Fighter. In fiscal year 2003, the 
Congress appropriated 
$127 billion to DOD for the research, development, and procurement of 
weapon systems. Funding for weapon systems is projected to continue 
growing to $182 billion in fiscal year 2009--an increase of over 43 
percent. In total, the investment in weapons from fiscal years 2003 
through 2009 will exceed $1 trillion. Thus, it is essential that sound 
foundations for these and other weapon system investments be laid now 
so that the resulting programs can be executed within estimates of 
available resources.

The challenge of putting new programs on a better footing than their 
predecessors is a daunting one. Clearly, the acquisition process 
produces superior weapons. But it does so at a high price. Weapon 
systems routinely take much longer to field, cost more to buy, and 
require more support than investment plans provide for. These 
consequences reduce the buying power of the defense dollar, delay 
capabilities for the warfighter, and force unplanned--and possibly 
unnecessary--trade-offs among programs.

DOD has undertaken a number of acquisition reforms over the preceding 
two decades in response to those problems, but while there have been 
individual successes, these reforms have not yet yielded consistent 
improvements in program outcomes. More recently, DOD leadership has 
embraced an evolutionary acquisition approach, coupled with time-phased 
requirements. This approach supports developing weapons in smaller, 
more predictable iterations of increasing capabilities, rather than the 
past approach of attempting to achieve a weapon's maximum capability in 
one design leap. DOD is also striving to give programs, such as missile 
defense, more flexibility to make trade-offs between cost, schedule, 
and performance that can lead to better investment decisions. It is 
also currently looking at how to revise its planning, programming, and 
budgeting process that has been in place for over 40 years.

Key to any effort to improve weapon system outcomes is using the 
lessons that can be learned from the best practices of successful 
commercial and defense product development programs. We have found that 
these practices can be collectively described as a knowledge-based 
approach whose success depends on the timely attainment and use of a 
product's technology, design, and production maturity. In this report, 
we compare the knowledge gained on 26 DOD weapon system programs with 
best practices. Our objective is to provide decision makers a means to 
quickly gauge the progress and potential risks--based on demonstrated 
knowledge--of the individual weapon system programs.

A Knowledge-Based Approach Can Lead to Better Acquisition Outcomes:

All product development efforts, whether for a car, a plane, a missile, 
or a satellite, go through a process of building knowledge. Ultimately, 
this process brings together and integrates all of the technologies, 
components, and subsystems needed for the product to work and to be 
reliably manufactured. The product development process can be 
characterized as the reduction of risk and the resolution of unknowns 
through the attainment of knowledge.

About 7 years ago, at the request of the Senate Committee on Armed 
Services, we began an extensive body of work identifying best practices 
in product development, both in DOD and in the commercial sector. Of 
particular interest were cases in which increasingly sophisticated 
products were being developed in significantly less time and at lower 
cost than their predecessors. A major reason for these successes was 
the use of a product development process that was anchored in 
knowledge. Product developers employed specific practices to ensure 
that a high level of knowledge regarding critical facets of the product 
was achieved at key junctures in development. We have characterized 
these junctures as three knowledge points. We have also identified key 
indicators that can be used to assess the attainment of each knowledge 
point. When tied to major events on a program's schedule, they can 
disclose whether gaps or shortfalls exist in demonstrated knowledge, 
which can presage future cost, schedule, and performance problems. 
These knowledge points and associated indicators are defined as 
follows.

* Knowledge point 1: Resources and needs are matched. This level of 
knowledge is attained when a match is made between a customer's needs 
and the developer's technical, financial, and other resources. 
Technology maturity is a particularly important indicator of resource 
availability. A best practice is to achieve a high level of technology 
maturity at the start of product development. This means that the 
technologies needed to meet essential product requirements have been 
demonstrated to work in their intended environment.

* Knowledge point 2: The product design is stable. This level of 
knowledge is attained when the product's design demonstrates its 
ability to meet the customer's requirements. A best practice is to 
achieve design stability at the system-level critical design review, 
usually held midway through development. Completion of engineering 
drawings at the system design review provides tangible evidence that 
the design is stable.

* Knowledge point 3: Production processes are mature. This level of 
knowledge is attained when it is demonstrated that the product can be 
manufactured within cost, schedule and quality targets. A best practice 
is to achieve production maturity at the start of production. This 
means that all key manufacturing processes produce output within 
statistically acceptable limits for quality.

As illustrated in figure 1, the process is building block in nature as 
the attainment of each successive knowledge point builds on the 
proceeding one. While the knowledge itself builds continuously without 
clear lines of demarcation, the attainment of knowledge points is 
sequential. In other words, production maturity cannot be attained if 
the design is not mature, and design maturity cannot be attained if the 
key technologies are not mature.

Figure 1: Knowledge Build at Key Points in Product Development Reduces 
the Risk of Unknowns:

[See PDF for image]

[End of figure]


For the most part, all three knowledge points are eventually attained 
on a completed product. The difference between highly successful 
product developments--those that deliver superior products within cost 
and schedule projections--and problematic product developments is how 
this knowledge is built and how early in the development cycle each 
knowledge point is attained. When knowledge is built more slowly than 
these points suggest, less knowledge is on hand at key decisions or 
events, such as the decisions to start a development program, hold the 
critical design review, and start production. This invites greater 
cost, schedule, and performance risks because (1) problems are more 
likely to be discovered late in the process and will therefore be more 
difficult and costly to correct and (2) a variety of pressures 
encourage program managers to underestimate the difficulties.

It is important to note that successful product developers treat 
technology development as a different and separate effort that precedes 
product development. This treatment of technology development is key to 
reaching the first knowledge point at the start of product development, 
as it is a prerequisite for capturing design and production knowledge 
early in product development. This approach to attaining knowledge puts 
program managers--and programs--in a better position to succeed.

Observations:

When programs proceed with less knowledge than suggested by best 
practices, cost, schedule, and performance problems often result. To 
varying degrees, all the programs we assessed proceeded with lower 
levels of knowledge at critical junctures and thus attained key 
elements of product knowledge later in development. In some programs, 
the consequences of proceeding with early knowledge deficits have 
already been felt. For example:

* The F-22 Fighter began product development with key technologies 
immature--deferring knowledge point 1--and subsequently had only a 
quarter of the desired amount of engineering drawings completed at the 
critical design review--deferring knowledge point 2. The program has 
experienced substantial cost increases and schedule delays in the 
latter stages of development.

* The Patriot Advanced Capability missile also reached knowledge points 
1 and 2 later than best practices. The seeker technology did not 
demonstrate maturity until close to the production decision and the 
design remains unstable. Each seeker still needs to be reworked about 
3 times on average before it passes quality inspections. The cost of 
the seeker has increased by 76 percent and contributed to a 2-year 
delay in the program's schedule.

* The Extended Range Guided Munition program began with only one of its 
20 critical technologies mature--deferring knowledge point 1. While 
progress has been made, program officials do not expect to achieve 
maturity on all technologies until after the design review. The lack of 
mature technologies contributed to subsequent test failures, cost 
increases, and schedule delays.

If programs attain more knowledge as suggested by best practices, they 
are in a better position to succeed in meeting cost, schedule, and 
performance expectations. We found some programs that did attain key 
product knowledge earlier than most. For example:

* The National Polar-orbiting Operational Environmental Satellite 
System program ensured that its pacing technologies were demonstrated 
before committing to product development. The program plans to 
demonstrate three critical sensors on a demonstrator satellite prior to 
their inclusion on the new satellite.

* The Theater High Altitude Area Defense System made significant 
strides in product development, following a problematic preliminary 
development phase. In 2000, we reported that the program's delayed 
demonstration of technologies and components and reliance on full-
system testing to discover problems, was a very costly method to mature 
the system's design and nearly caused the cancellation of the program. 
The program has since structured a product development phase that 
places a much greater emphasis on early demonstration of components, a 
testing program that incorporates sufficient time between tests for 
learning, and a plan to achieve design stability by releasing 90 
percent of engineering drawings by the time of the critical design 
review--knowledge point 2.

In general, we found that the greatest absence of knowledge was in the 
area of production. Almost no programs collected statistical process 
control data, the indicator for production maturity. Unlike technology 
readiness levels, which can be applied at any time, and engineering 
drawing release data, which is captured on all programs, few programs 
collected statistical process control data. While the absence of this 
data does not necessarily mean that production processes were immature, 
attained knowledge could not be assessed against an objective standard. 
Other indicators of production maturity, such as scrap and rework 
rates, can indicate positive trends, but are not prospective--that is, 
they are not useful in guiding preparations for production. To some 
extent, statistical process control data is not being collected because 
DOD has been delegating more responsibility to prime contractors and 
reducing the amount of data requested. The lack of such data may put 
program offices in a disadvantaged position to gain insights about a 
contractor's production progress. We have recently issued a report that 
recommends that DOD collect statistical process control data on its 
weapon system programs and DOD has agreed with this 
recommendation.[Footnote 1]

We conducted our review from September 2002 through May 2003 in 
accordance with generally accepted government auditing standards.

Agency Comments:

DOD did not provide general comments on a draft of this report, but did 
provide technical comments on individual assessments. These comments, 
along with program office comments, are included with each individual 
assessment as appropriate.

Scope of Our Review:

We selected programs for the assessments based on several factors, 
including (1) high dollar value, (2) stage in acquisition, and (3) 
congressional interest. The majority of the 26 programs covered in this 
report are considered major defense acquisition programs by DOD. A 
program is defined as major if its estimated research and development 
costs exceed $365 million or its procurement exceeds $2.19 billion in 
fiscal year 2000 constant dollars.

We plan to include more programs in subsequent years, with a greater 
focus on programs early enough in development that the assessments can 
be used to improve the program's prospects for success, and issue this 
report annually to the congressional defense committees. The individual 
assessment of each program can be found in appendix I. Appendix II 
contains detailed information on our methodology.

:

We are sending copies of this report to interested congressional 
committees; the Secretary of Defense; the Secretaries of the Army, 
Navy, and Air Force; and the Director, Office of Management and Budget. 
We will also make copies available to others upon request. In addition, 
the report will be available at no charge on the GAO Web site at http:/
/www.gao.gov. If you have any questions on this report, please contact 
me at (202) 512-4841 or Paul Francis at (202) 512-2811. Major 
contributors to this report are listed in appendix III.

Signed by:

Jack L. Brock, 
Managing Director
Acquisition and Sourcing Management:

List of Congressional Committees:

The Honorable John W. Warner
Chairman
The Honorable Carl Levin
Ranking Member
Committee on Armed Services
United States Senate:

The Honorable Ted Stevens
Chairman
The Honorable Daniel K. Inouye
Ranking Member
Subcommittee on Defense
Committee on Appropriations
United States Senate:

The Honorable Duncan Hunter
Chairman
The Honorable Ike Skelton
Ranking Minority Member
Committee on Armed Services
House of Representatives:

The Honorable Jerry Lewis
Chairman
The Honorable John P. Murtha
Ranking Minority Member
Subcommittee on Defense
Committee on Appropriations
House of Representatives:

[End of section]

Appendixes :

[End of section]

Appendix I: Assessments of Individual Programs:

Introduction:

For the 26 programs, each assessment provides the historical and 
current program status and offers the opportunity to take early 
corrective action when a program's projected attainment of knowledge 
diverges significantly from the best practices. The assessments also 
identify programs that are employing practices worthy of emulation by 
other programs. If a program is attaining the desired levels of 
knowledge, it has less risk--but not zero risk--of future problems. 
Likewise, if a program shows a gap between demonstrated knowledge and 
best practices, it indicates an increased risk--not a guarantee--of 
future problems. The real value of the assessments is recognizing gaps 
early, which provides opportunities for constructive intervention--
such as adjustments to schedule, trade-offs in requirements, and 
additional funding--before cost and schedule consequences mount.

Our assessment of each program is summarized in two components--(1) a 
system profile and (2) a product knowledge assessment.

The system profile presents a general description of the product in 
development; a picture of the product or a key element of the product; 
a schedule timeline identifying key dates in the program; a table 
identifying the prime contractor; the program office location, and the 
fiscal year 2004 requested funding if available; and a table 
summarizing the cost, schedule and quantity changes to the program.

The rest of the assessment analyzes the extent to which product 
knowledge at the three key knowledge points has been attained. We 
depict the extent of knowledge in a stacked bar graph and provide a 
narrative summary at the bottom of the first page. The second page is 
devoted to a narrative assessment of technology, design and production 
maturity, as well as other program issues identified and comments from 
the program office.

The product knowledge figure is based on the three knowledge points and 
the key indicators for the attainment of knowledge. A "best practice" 
line is drawn based on the ideal attainment of the three types of 
knowledge at the three knowledge points (see fig. 2).

Figure 2: Graphic Depiction of Best Practices for Technology, Design, 
and Production Knowledge:

[See PDF for image]

[End of figure]

The first major point on the best practice line represents two facts: a 
commitment to a new product development has been made and the key 
technologies needed for the new product are mature.

When all critical technologies have reached a technology readiness 
level 7, technology maturity--and thus knowledge point 1--has been 
attained. In our assessment, the technologies that have reached 
technology readiness level 7, a prototype demonstrated in an 
operational environment, are considered mature and those that reach 
technology readiness level 6, a prototype demonstrated in a relevant 
environment, are assessed as attaining 50 percent of the desired level 
of knowledge. Satellite technologies that achieved technology readiness 
level 6 were assessed as fully mature due to the difficulty of 
demonstrating maturity in an operational environment--space. 
(Technology readiness levels are more fully explained in appendix II.) 
The second major point on the best practice line captures technology 
maturity plus design maturity--knowledge point 2. A design is 
considered mature when 90 percent of the engineering drawings have been 
released or deemed releasable to manufacturing. In the successful 
programs we have studied, design maturity is attained about halfway 
through the product development phase. The third major point on the 
best practice line captures the sum of technology maturity, design 
maturity, and production maturity. Production is considered mature when 
all key production processes are in statistical control. Ideally, this 
occurs before the first products for delivery to the customer are 
manufactured. As can be seen, knowledge about the technology, design, 
and production of a new product builds over time. While the knowledge 
itself builds continuously without clear lines of demarcation, the 
attainment of knowledge points is sequential. In other words, 
production maturity cannot be attained if the design is not mature, and 
design maturity cannot be attained if the key technologies are not 
mature.

Data for a given weapon system program is then plotted against the best 
practices line. In the assessments that follow, a brown bar indicates 
the technology knowledge attained by a weapon system program. The 
actual technology readiness levels attained for a program's key 
technologies are measured at the start of development--normally 
milestone II or milestone B in the Department of Defense's (DOD) 
acquisition process. The closer a program's attained knowledge is to 
the best practice line, the more likely the weapon will be delivered 
within its estimated cost and schedule. A knowledge deficit at this 
point--indicated by a gap between the technology knowledge attained by 
the weapon system and the best practices line--means the program 
proceeded with immature technologies and may face a greater likelihood 
of cost and schedule increases as technology risks are discovered and 
resolved. A green bar indicates the design knowledge attained by a 
weapon system program. This is calculated by measuring the percent of 
engineering drawings released to manufacturing. The green bar is 
stacked on top of the brown bar to indicate whether any cumulative gap-
-considering both technology and design--exists at the halfway point of 
product development. A blue bar indicates the production knowledge 
attained by a weapon system program. This is calculated by measuring 
the percentage of key production processes in statistical control. The 
blue bar is stacked on top of the brown and green bars to indicate 
whether any cumulative technology, design, and production gaps exist at 
the time production begins. In some cases, we obtained projections from 
the program office of future knowledge attainment. These projections 
are depicted as dashed bars.

Figure 3 depicts an example of an assessment for a notional weapon 
system.

Figure 3: Depiction of a Notional Weapon System Program's Knowledge as 
Compared with Best Practices:

[See PDF for image]

[End of figure]

An interpretation of this notional example would be that the product 
development began with key technologies immature, thereby missing 
knowledge point 1. Knowledge point 2 was not attained at the design 
review as some technologies were still not mature and only a small 
percentage of engineering drawings had been released. Projections for 
the production decision show that the program is expected to achieve a 
greater level of maturity, but will still fall short. It is likely that 
this program would have had significant cost and schedule increases.

We also found three situations in which programs were unable to provide 
key knowledge indicators. We used three types of labels in the 
knowledge figures to depict those situations. Programs with these 
labels are distinguished from those that have elected not to collect 
data that can be used to assess progress against best practices. First, 
a few programs are planning to collect the relevant knowledge 
indicator, but they have not yet begun collecting it. In these 
situations, we annotate the graph with the phrase "Data unavailable." 
Second, a few programs have not followed the traditional acquisition 
model. For example, one program combined the development start decision 
with the production decision. Another program used commercial off-the-
shelf components, which negated the need to monitor production 
processes. In these situations, we annotate the graph with the phrase 
"Not applicable." Finally, some programs were unable to provide or 
reconstruct the relevant knowledge indicator because the event happened 
too many years ago. In these situations, we annotate the graph with the 
phrase "Not assessed.":

Our assessments of the 26 systems follow.

Advanced Amphibious Assault Vehicle (AAAV).

The Marine Corps' AAAV is designed to transport troops from ships to 
shore at higher speeds and from farther distances than the existing 
AAV-7. It is designed to be more mobile, lethal, reliable, and 
effective in all weather conditions. AAAV will have two variants--a 
troop carrier for 17 Marines and a command vehicle to manage combat 
operations in the field.

[See PDF for image]

[End of figure]

[See PDF for image]

[End of figure]

Advanced 
Amphibious Assault Vehicle (AAAV): AAAV demonstrated most technology 
and design knowledge at critical junctures in the program. At the start 
of the program, all but one of the critical technologies were mature. 
The design was close to meeting best practice standards at the design 
review, signifying the design was stable. Early development of fully 
functional prototypes and other design practices facilitated design 
stability. However, late maturation of the remaining technology may 
lead to some redesign. Also, the demonstration of production maturity 
remains a concern because the program is currently uncertain about 
requiring the contractor to use statistical process controls to achieve 
quality objectives. The AAAV production decision is not scheduled until 
September 2005. Remaining efforts include developmental, operational, 
live fire, and reliability testing.

AAAV Program:

Technology Maturity:

Four of the five critical technologies had demonstrated an acceptable 
level of maturity at the start of product development. The remaining 
technology, moving map navigation, is not expected to achieve maturity 
until the spring of 2003. Program officials stated that maturing this 
technology is contingent on developing and testing system hardware. As 
a backup, program officials said they could carry out the AAAV mission 
using existing technology, but it would not provide full vehicle-to-
vehicle situational awareness.

Design Maturity:

The AAAV design is essentially complete. However, late maturation of 
the new mapping system may lead to some redesign, if testing identifies 
any problems.

At the critical design review, AAAV had completed 77 percent of the 
drawings--not up to the best practice standard of 90 percent, but 
higher than many DOD programs. Early engineering prototypes--fully 
integrated and functional--allowed the program to demonstrate that the 
design worked as required. These early prototypes have completed over 
4,000 hours of testing that resulted in design improvements for 
subsequent prototypes.

To complete development, program plans call for building and testing 
nine development prototypes and one live fire test vehicle. These 
prototypes will be production representative vehicles for 
developmental, operational, live fire and reliability testing. The 
first prototype is scheduled to be available by May 2003.

Production Maturity:

Program officials are developing a production readiness plan to ensure 
vehicles will meet cost, schedule, and quality objectives. At this 
time, they are uncertain whether this plan will require the contractor 
to use statistical process controls, the best practice standard. As the 
prime contractor currently produces the nine developmental prototype 
vehicles, it is not tracking statistical process control data. Instead, 
it is using postproduction inspections, considered less efficient and 
effective than statistical process controls to achieve quality.

Other:

The Marine Corps has recently restructured the AAAV program to add 12 
additional months of testing before the September 2005 production 
decision. This change more than doubles the number of vehicle test 
months previously planned. The change also moves the initial 
operational capability date from September 2007 to September 2008. The 
program estimates a $480 million increase in acquisition costs--
$101 million for added testing, $75 million for development, and 
$304 million for recurring production.

Program Office Comments:

AAAV program officials concurred with our assessment.

Airborne Laser (ABL).

The Missile Defense Agency's ABL is designed to destroy enemy ballistic 
missiles almost immediately after their launch. The system, carried 
aboard a highly modified Boeing 747 aircraft, uses a high-energy 
chemical laser to rupture the skin of enemy missiles; a beam control/
fire control subsystem to guide the laser beam through the aircraft, 
focus the beam on the target, and maintain the beam's quality as it 
travels through the atmosphere; and a battle management subsystem to 
plan and execute the engagement. We assessed all components.

[See PDF for image]

[End of figure]

Airborne Laser (ABL): Only one 
of ABL's critical subsystems has demonstrated acceptable levels of 
maturity. The Missile Defense Agency is developing an initial ABL 
system to demonstrate technology critical to the system's design and 
plans to begin development of a second improved demonstration aircraft 
in 2003. Either of these aircraft, or later improved configurations, 
could be given to the Air Force for operational testing and production 
if system-level tests show that any one of them is capable of 
destroying a threat missile at an operational range. Although the 
agency's development strategy incorporates some knowledge-based 
practices, it is difficult to see how the discipline of a knowledge-
based approach can be achieved when uncertainty exists about whether 
the effort is a technology development or a product development.

[See PDF for image]

[End of figure]


ABL Program:

Technology Maturity:

Only one of ABL's five critical subsystems, the 
aircraft itself, represents mature technology. A second subsystem, 
which directs the laser energy through the aircraft, consists of 
several technologies that have been tested in a simulated environment. 
However, three other subsystems consist of low-fidelity prototype 
technologies that have only been tested in a laboratory environment. 
They include the laser, the battle management subsystem, and the ground 
support subsystem.

Problems associated with maturing technology have consistently been a 
source of cost and schedule growth throughout the life of the program. 
DOD analysts attribute this growth to the increased complexity of 
designing laser subsystems, substantial increases in engineering 
analysis and design, and greater than anticipated aircraft engineering 
complexity.

The program is managed under the Missile Defense Agency's new 
capabilities-based acquisition strategy. This approach develops an 
operational system through a series of block upgrades. The agency plans 
to use the first two blocks, block 2004 and block 2008, to demonstrate 
critical technologies, but if tests show either configuration has any 
battlefield utility, that configuration could be deployed in the event 
of an emergency.

The 2004 configuration will have a 6-module laser, rather than the 14 
modules planned for the production system. The optical components can 
withstand the heat produced by a 6-module laser, but the agency would 
have to redesign optical components for the system to withstand the 
heat associated with an increase in laser power. In addition, the 2004 
configuration is far too heavy to allow the addition of laser modules 
that will likely be needed in an operational ABL system.

To accommodate more modules, a weight reduction program has begun that 
includes redesigning many components and the increased use of composite 
materials. The program is considering whether to use a different 
aircraft configuration that would allow the system's weight to be moved 
forward to relieve stress on the airframe. However, its use would 
require additional design changes.

The Missile Defense Agency has made changes that are expected to 
improve its ability to evolve ABL's critical technologies, including 
adopting a flexible requirements setting process, providing additional 
time and facilities to develop and test these technologies, and 
attaining the knowledge to match the warfighters' needs with 
demonstrated technology. On the other hand, it is not clear whether the 
start of a block represents a technology development or a product 
development. This uncertainty may hamper the application of knowledge 
standards and forfeit the discipline necessary to ensure successful 
product development.

Program Office Comments:

In commenting on a draft of this assessment, program officials 
reemphasized their commitment to spiral development and capabilities-
based acquisition. They plan to use this strategy to improve the 
critical aspects of the system by allowing the pace of technological 
development to dictate the introduction of improved capabilities into 
the system. They believe this strategy is not inconsistent with 
knowledge-based acquisition.

They also mentioned that laser power depends not only on the number of 
laser modules but also on module efficiency, optics, and pointing 
precision. They admit that the laser subsystem should be operated in 
flight before any production decision is made. Program officials are 
conducting emergency operational capability planning to support a 
possible emergency ABL deployment. This decision will be based on the 
potential threat and an assessment of the capabilities ABL may provide.

The program office indicated that all but one of the battle management 
components have been tested in an operational environment. This 
component is the active ranger system, which provides crucial angle 
measurements and range data for engaging ballistic missiles.

Advanced Extremely High Frequency (AEHF) Communications Satellite.

The Air Force's AEHF is a satellite system intended to replace the 
existing Milstar system with improved, survivable, jam-resistant, 
worldwide, secure communication capabilities at lower launch costs. 
First launch of an AEHF satellite is expected in 2006. The system also 
includes a mission control segment with service-specific terminals to 
process satellite information. DOD is negotiating international partner 
participation in the program. We assessed the satellites and mission 
control segments.

[See PDF for image]

[End of figure]

Advanced Extremely High Frequency (AEHF) 
Communications Satellite: The AEHF satellite program demonstrated most 
technology knowledge at development start. Eleven of the 12 critical 
technologies were mature, according to best practice standards. The 
remaining technology is not projected to be mature prior to the 
critical design review, nor does it have a backup technology. However, 
some elements of this technology are mature. The program expects to 
complete 90 percent of its drawings by the critical design review. The 
manufacture of the communications and transmission security subsystem 
is a major challenge facing the program as upgrades are being added 
into the new cryptological equipment. If production of this subsystem 
slips, first launch could slip correspondingly as no backup exists.

[See PDF for image]

[End of figure]

AEHF Program:

Technology Maturity:

Eleven of the 12 critical technologies have reached maturity according 
to best practice standards. The program does not project achieving 
maturity on the remaining technology--the phased array antenna--by the 
design review in June of 2004, nor does it have a backup capability. 
However, some elements of this technology have been demonstrated in an 
operational environment.

Design Maturity:

The program has completed 150 or more of the 6,000 total drawings for 
release to manufacturing. Program officials project completing 90 
percent of drawings by the system critical design review in June 2004. 
The program has completed key segment level preliminary design reviews 
and is expected to complete all design reviews by the second quarter of 
fiscal year 2004. Program officials consider the design and development 
of the satellite subcomponents low risk because those components have 
been used on other space systems. However, the integration of these 
subcomponents into a subsystem, such as the phased-array antenna, has 
yet to be successfully demonstrated at the AEHF satellite frequencies.

Program officials assessed the software development for the mission 
control system as moderate risk and have developed a risk mitigation 
strategy. This strategy includes consulting with the National Software 
Engineering Institute and the Aerospace Corporation and conducting a 
software development capability evaluation. Also, the program office 
has incorporated spiral development and the use of software emulators 
so users and developers can see how the software will look and work. 
Until these actions are completed, software may be at risk for 
unplanned cost and schedule growth.

Production Maturity:

Any future problems with the fabrication of the communications and 
transmission security microprocessor, a component designed to limit 
access to satellite transmissions to authorized users, could delay the 
production schedule and the launch of the first satellite planned for 
December 2006. Program officials have started a number of risk 
reduction efforts, including a chip emulator whose purpose is to 
simulate the communications and transmission security subsystem's 
functions as it is integrated into the AEHF satellite's communications 
subsystem. However, continued complications in fabrication could 
potentially place the entire program at cost, schedule, and performance 
risk.

Other Program Issues:

In December 2002, the Deputy Secretary of Defense decided to change the 
acquisition strategy of AEHF from a five-satellite program to a three-
satellite program. Under the revised strategy, full capability may no 
longer be satisfied by an AEHF-only constellation.

Program Office Comments:

In commenting on a draft of this assessment, program officials stated 
that the program is executing very well since contract definitization 
in August 2002, with cost and schedule variance at less than 1 percent. 
Currently, at approximately 33 percent complete toward first launch, 
the total program is on track and estimated to finish on time and on 
budget. The system preliminary design review has been completed. 
Critical design reviews are on track for completion by Spring 2004. 
Funding cuts have, in the past, caused schedule slips and cost 
increases. Given the focus on the critical design review, the impacts 
of changing requirement will have increasing deleterious effects. The 
program remains focused on addressing critical risks that threaten 
cost, schedule, and performance. New system security requirements 
recently received from the National Security Agency for the space, 
mission control, and terminal segments are being evaluated. After 
aggressive risk management, the most likely impacts include additional 
testing, verification, and program documentation. The program has also 
begun developing engineering models for all of the critical subsystems. 
These efforts are on track and proceeding well.

AN/APG-79 Active Electronically Scanned Array (AESA) Radar.

The Navy's AESA radar is one of the top upgrades for the F/A-18E/F 
aircraft. It is to be the aircraft's primary search/track and weapon 
control radar and is designed to correct deficiencies in the current 
radar. According to the Navy, the AESA radar is key to maintaining the 
Navy's air-to-air fighting advantage and will improve the effectiveness 
of the air-to-ground weapons. When completed, the radar will be 
inserted in new production aircraft and retrofitted into existing 
aircraft.

[See PDF for image]

[End of figure]

AN/APG-79 Active Electronically Scanned Array (AESA) Radar: 
The AESA radar's demonstrated knowledge is difficult to characterize. 
The fact that almost all of the engineering drawings have been 
completed suggests design stability. However, until the technologies 
are demonstrated, the potential for design changes remains. The AESA 
radar is also dependent on other programs that could pose significant 
risk to the radar's cost, schedule, and technical performance. The 
technology and design risks are significant given that the AESA radar 
is only a few months from a production decision. The Navy is currently 
reassessing the radar's technology maturity. Although many of the F/A-
18E/F aircraft will be retrofitted with the AESA radar, full funding 
for the retrofitting has not been budgeted. If the radar is not ready 
for production as scheduled, more aircraft will have to be 
retrofitted.

[See PDF for image]

[End of figure]

AESA Radar Program:

Technology Maturity:

The AESA program's four critical technologies were not mature at the 
start of development in February 2001, and they were not mature as of 
May 2002. The Navy is currently reassessing the maturity of these 
technologies. At the time of its last assessment, two of the 
technologies had been tested using simulation and two had been tested 
in the laboratory. Program officials indicated that they have several 
options for dealing with immature technologies, including utilizing 
backup technologies. Initial flight tests of the radar in an aircraft 
are scheduled for June 2003--concurrent with the production decision. 
All four technologies are not expected to be mature until late 2004.

Design Maturity:

At the design review, 67 percent of the currently projected total 
drawings were completed. In the period between June 2002 and December 
2002, the number of total expected drawings increased by 21 percent. 
Program officials stated that the increase was due to new or modified 
drawings for systems supporting the radar such as the radome, shield, 
and aircraft airframe. Program officials indicated that they currently 
have 98 percent of the drawings complete; however, the technology 
maturation process may lead to more design changes.

Production Maturity:

We could not assess the AESA program's production maturity against best 
practices, as statistical control data was not available.

Other Program Issues:

Program officials estimate that the first low-rate production unit will 
exceed its cost target by 27 percent. Subcontractor development cost 
was considered to be the biggest contributor to this increase. The 
effects of the cost increase may be minimized in low-rate production 
lots 1-3 because of firm fixed price contract options. Program 
officials stated that cost reduction initiatives were underway to 
reduce the cost overruns by half by full-rate production.

Delivery of the first production AESA radars for insertion into F/A-
18E/F aircraft on the production line is scheduled for fiscal year 
2005. As a result, 254 of the planned total buy of 548 F/A-18E/F 
aircraft will not receive the radar as they are being produced. Plans 
are to retrofit the radar onto 136 aircraft at a projected cost of 
$3.14 million each. This cost does not include the cost of new APG-73 
mechanical scanned radars that will be installed in the aircraft until 
AESA radars are available for retrofit. If delays occur in the AESA 
radar deliveries, retrofit costs will increase.

The AESA radar is projected to weigh about 270 pounds more than the 
current radar and will require a more capable cooling system than the 
one currently on the aircraft. The Navy expects some minor degradation 
in aircraft performance, such as slightly decreased range, as a result 
of the increased weight and new cooling system.

The AESA program is linked to a number of other corporate and Navy 
programs. For example, the radar will use a 32-port fiber channel 
fabric module developed by Boeing as a commercial venture. Technical 
difficulties with the module have caused schedule delays and may impact 
cost and performance of the radar. Also, Raytheon is developing some 
hardware and software for the radar with company funds or in 
coordination with other programs. Disruptions in these efforts could 
adversely impact the AESA program.

Program Office Comments:

The AESA program did not provide a general statement in response to our 
review but did provide technical comments that were incorporated where 
appropriate.

AIM-9X Short-Range Air-to-Air Missile.

The AIM-9X is a follow-on version of the existing AIM-9M short-range 
missile for Air Force and Navy fighters. The AIM-9X is designed to be a 
highly maneuverable, launch-and-leave missile; capable of engaging 
targets using passive infrared guidance to provide full day/night 
operations and improved resistance to countermeasures and expanded 
target acquisition. The full capabilities of the AIM-9X will not be 
achieved without completing development of the helmet mounted cueing 
system--a separate development program that we did not assess.

[See PDF for image]

[End of figure]



AIM-9X Program:

Technology Maturity:

All of the AIM-9X critical technologies are mature because they have 
been demonstrated in developmental tests using actual hardware in 
realistic conditions. Specifically, the program used prototypes to test 
new technologies and existing missile components that are being 
employed in a new operational environment.

Design Maturity:

The design of the AIM-9X is complete, and 100 percent of the drawings 
have been released to manufacturing. The AIM-9X program built and 
tested 43 prototypes of various configurations during development to 
help mature the missile's design. Hardware and software performance was 
assessed at subsystem and system levels, and design changes were 
incorporated into the prototypes until a mature and stable missile 
configuration was demonstrated. The AIM-9X program held design reviews 
for the 11 subsystems between October 1997 and March 1998. The early 
design reviews, prototypes, and early testing allowed the program to 
achieve a stable design at the system design review in March 1998. At 
that time, the contractor had released 94 percent of its engineering 
drawings to manufacturing.

Production Maturity:

The AIM-9X program does not contractually require collection of 
statistical process control data on critical manufacturing process, but 
it has undertaken an acquisition strategy to incentivize the contractor 
to reach cost and quality goals. However, the contractor and program 
officials believe that they have significant knowledge about producing 
the missile. The AIM-9X is a variant of the AIM-9M missile and uses 
components produced for other weapon systems, providing the program 
with significant production knowledge. In addition, to improve the 
production capabilities, the contractor built developmental units on 
production equipment. Program officials believe this practice has 
allowed them to mature the manufacturing processes. According to 
program officials, most of the critical processes on the AIM-9X are at 
the subcontractor level and a process exists to attain cost and quality 
goals. This is accomplished primarily by postproduction inspections to 
track production yield, scrap, and rework data. The AIM-9X acquisition 
cost and schedule history shows the program has been able to meet its 
goals.

Program Office Comments:

In commenting on a draft of this assessment, program officials 
acknowledged they did not contractually require collection of 
statistical process control data on critical manufacturing processes. 
Program officials stated their strategy for demonstrating manufacturing 
process maturity includes building, testing, and evaluating production 
representative missiles; conducting multiple readiness reviews; 
utilizing low-rate initial production to test production processes; and 
maturing production processes before full-rate production.

Advanced Threat Infrared Countermeasures/Common Missile Warning System 
(ATIRCM/CMWS).

The Army's and Special Operations' ATIRCM/CMWS is a component of the 
Suite of Integrated Infrared Countermeasures planned to defend U.S. 
aircraft from advanced infrared-guided missiles. The system will be 
employed on Army and Special Operations aircraft. ATIRCM/CMWS includes 
an active infrared jammer, a missile warning system, and a 
countermeasure dispenser capable of loading and employing expendables, 
such as flares, chaff, and smoke.

[See PDF for image]

[End of figure]


Advanced Threat Infrared 
Countermeasures/Common Missile Warning System (ATIRCM/CMWS): The 
ATIRCM/CMWS is scheduled to enter production in May 2003 with no 
assurance that production processes are in control. The CMWS portion of 
the ATIRCM/CMWS program entered limited production in February 2002 to 
meet an urgent need. Full-rate production for ATIRCM/CMWS was delayed 
because of reliability problems, which may indicate that production 
processes were not in control. These problems are, at least in part, a 
consequence of design proceeding with known shortfalls in knowledge: 
key technologies were demonstrated late in development and only a small 
number of design drawings were completed by design review. Resolving 
these knowledge shortfalls has led to cost and schedule increases. 
While the key technologies appear mature, reliability and producibility 
issues could necessitate design changes.



ATIRCM/CMWS Program:

Technology Maturity:

The five critical technologies for the system are mature, but they did 
not mature until after the system design review. Most of the early 
technology development effort was focused on the application to rotary 
wing aircraft. However, when product development began in 1995, the 
requirements were expanded by Office of the Secretary of Defense 
direction to include Navy and Air Force fixed wing aircraft. According 
to program officials, they did not fully anticipate the additional 
technology needed to meet these much more demanding requirements. This 
change caused problems that largely contributed to cost increases of 
more than 150 percent to the development contract. The Navy and the Air 
Force subsequently dropped out of the program, rendering the extra 
effort needless.

Design Maturity:

The basic design of the system is complete, with 100 percent of the 
drawings released to manufacturing. However, reliability and 
producibility issues could require design changes. The design was 
particularly immature at the critical design review, with only 
22 percent of the drawings complete. A major cause was that the 
technology requirements were not well understood until the system 
design review, leading to the discovery that a major redesign was 
needed to meet requirements. It was not until 2 years after the design 
review that 90 percent of the drawings were released and the design was 
considered stable. According to program officials, the immature design 
caused inefficient manufacturing, rework, and testing and contributed 
to the 3-year schedule delay.

Production Maturity:

The ATIRCM/CMWS program does not collect statistical control data on 
its critical manufacturing processes. Program officials have identified 
the absence of statistical process control data as a weakness and 
believe it should be instituted. Despite this shortfall in knowledge, 
the Army entered limited CMWS subsystem production in February 2002 to 
meet an urgent need of the U.S. Special Operations Command.

The program delayed the production decision for the combined system an 
additional year to the currently scheduled May 2003 date primarily due 
to reliability issues. Reliability testing was halted because of 
numerous failures with the ATIRCM subsystem. Reliability failure can be 
an indicator of producibility and process control problems. The program 
plans to build and develop six additional subsystems during 2002 and 
2003. The full-rate production decision for the complete system is now 
scheduled for 2005.

Other Program Issues:

The Army procured an initial 32 systems in fiscal year 2002 that only 
included the CMWS. The Army plans to procure a total of 99 ATIRCM/CMWS 
systems to outfit special operations aircraft between fiscal year 2002 
and 2009.

Program Office Comments:

In commenting on a draft of this assessment, program officials stated 
that the Army eliminated the program's funding for fiscal years 2002 
and 2003. In fiscal year 2003, the Special Operations Command funded 
the urgent procurement of 32 CMWSs. Subsequently, the Army reinstated 
the program for fiscal years 2004-2009. The program office stated that 
the loss of funding in fiscal year 2003 slowed the program markedly. 
The program's acquisition strategy remains to equip Special Operations 
forces before equipping the remainder of the Army.

The system was modified in 2002 to address ATIRCM reliability, 
producibility, and built-in-test issues. Six ATIRCM systems are being 
manufactured and tested to demonstrate and verify the enhancements. 
ATIRCM is scheduled to begin low-rate initial production in May 2003, 
and CMWS is scheduled to begin low-rate initial production in January 
2004. The program office stated that low-rate production is required to 
maintain a production base. The system's operational testing is planned 
for March 2005. According to the program office, the prime contractor 
indicated that statistical process control is not within its corporate 
philosophy, particularly for a program with such low production rates 
and quantities.

Advanced Wideband Satellite (AWS).

The Advanced Wideband Satellite system is designed to provide improved, 
survivable, jam-resistant, worldwide, secure and general purpose 
communications to support the National Aeronautics and Space 
Administration, DOD and the intelligence community. It will replace the 
current Milstar satellite system and supplement the AEHF satellite 
system, reviewed elsewhere in this report. It will be the cornerstone 
of a DOD architecture that includes the multiple satellite systems.

[See PDF for image]

[End of figure]

[See PDF for image]

[End of figure]

AWS/TSAT Program:

Technology Maturity:

Of the five AWS/TSAT key space segment technologies, one is mature 
while the other four are scheduled to reach maturity by January 2006, 
more than 2 years after development start. Three of the four immature 
technologies have a backup technology available in case of development 
difficulties. However, use of these technologies would degrade system 
overall performance. The Single Access Laser Communications technology 
has no backup and, according to program officials, any delay in 
maturing this technology would result in a slip in the expected launch 
date.

Other Program Issues:

The program plans a development cycle that is, according to DOD 
documentation, aggressive. The satellite development cycle is planned 
to be 75 months: 27 months for technology development; 15 months for 
product development; and 33 months for satellite build, test and 
launch. This period of time is substantially shorter than the 
development cycle for the AEHF satellite (118 months vs. 75 months), 
though the AWS/TSAT system is expected to provide a transformational 
leap in satellite communications capability.

The program is managed under the new National Security Space 
Acquisition process, which makes no clear distinction between the end 
of technology development and the start of product development. 
Therefore, the AWS/TSAT acquisition strategy may allow the system's 
technology development and product development to be conducted 
concurrently prior to the production decision. DOD's acquisition system 
policy states that one of the entrance criteria for the system 
development and demonstration phase is technology maturity. The AWS/
TSAT acquisition strategy does not ensure that technology maturity will 
be achieved prior to the start of product development consistent with 
best practices.

Program Office Comments:

In commenting on a draft of this assessment, program officials stated 
that the National Security Space Acquisition Policy was developed to 
streamline the decision-making framework and to tailor it for space 
systems, in order to more efficiently field systems that incorporate 
rapidly changing technology advances.

Cooperative Engagement Capability (CEC).

The Navy's CEC is designed to connect radar systems to enhance 
detection and engagement of air targets. Ships and planes equipped with 
their version of CEC hardware and software will share real-time data to 
create composite radar tracks, essentially allowing the battle group to 
see the same radar picture. A CEC-equipped ship will then be able to 
detect and launch missiles against targets its radar cannot see. We 
assessed block 1 of the CEC. The Navy is developing a more advanced 
block 2 CEC.

[See PDF for image]

[End of figure]


CEC Program:

Technology Maturity:

In January 2002, the Office of Naval Research assessed CEC's six 
critical technologies. Five of the technologies assessed as mature were 
incorporated into the shipboard version when it successfully completed 
the operational evaluation in May 2001. The sixth technology, a data 
processor, was not assessed as part of the operational evaluation but 
was determined to be mature.

Design Maturity:

CEC's basic design appears complete, as all of the drawings needed to 
build the shipboard version have been released to manufacturing.

CEC program officials noted that new drawings continue to be released. 
They explained that as commercially available technologies, which 
comprise approximately 60 percent of CEC's hardware, become more 
advanced, portions of the system will need to be and redesigned to 
incorporate those advances.

Production Maturity:

We could not assess the CEC program's production maturity against the 
best practice as data were not available. According to program 
officials, the noncommercially available portions of CEC do not involve 
any critical manufacturing processes. Officials indicated that they do 
not have insight into manufacturing processes for the commercially 
available portions, including whether these processes are critical and 
whether the contractor has them under statistical control.

The program officials and the contractor are confident that a quality 
product can be delivered on time and within cost based on the 
contractor's adherence to industry standards and past performance on 
the low-rate initial production contracts for the shipboard version.

Other Program Issues:

Battle group-level interoperability, integration, and built-in-test 
false alarm rates were identified as areas needing improvement 
following the operational evaluation. Program officials expect a 
solution for the alarm rates to be in place for a follow-on operational 
test and evaluation planned for 2004.

Some solutions for interoperability and integration issues will also be 
assessed in follow-on testing. However, many of these issues are 
expected to be resolved through the introduction of block 2. The plan 
was approved in April 2002. Block 2 is expected to provide cost, 
performance, and functional improvements over the current system, 
though its details are yet to be defined. Among the anticipated 
characteristics of block 2 is interoperability with legacy combat 
systems.

Program Office Comments:

In commenting on a draft of this assessment, program officials stated 
that a production readiness review conducted in October 2001 found CEC 
production to be mature. They evaluated all areas of production, 
including quality, configuration management, processes and procedures, 
drawings, and testing. They stated that the contractor is delivering 
systems on schedule and within cost. To date, 29 systems over 5 years 
have been successfully delivered, installed, tested, and many have been 
deployed. Following operational testing and evaluation, the Navy found 
CEC to be operationally suitable and effective and the DOD Director for 
Operational Test and Evaluation found CEC demonstrated the highest 
reliability of any system tested so far of comparable complexity. 
According to program officials, CEC's use of commercial off-the-shelf 
components enables the program to select mature cost-effective 
components from industry, instead of manufacturing them in-house. In 
recognition of the above, DOD approved the program for full-rate 
production in April 2002.

CH-47F Improved Cargo Helicopter.

The Army's CH-47F heavy lift helicopter is intended to provide 
transportation for tactical vehicles, artillery, engineer equipment, 
personnel, and logistical support equipment. It is expected to operate 
in both day and night. The purpose of the CH-47F program is to improve 
the performance and extend the useful life of the CH-47. This effort 
includes installing a digitized cockpit, rebuilding the airframe, and 
reducing aircraft vibration through airframe stiffening.

[See PDF for image]

[End of figure]

CH-47F Improved 
Cargo Helicopter: The CH-47F helicopter began low-rate production in 
December 2002, although key production processes were not in control. 
Program officials believe that CH-47F production is low risk because no 
new technology is being inserted into the aircraft, two prototypes have 
been produced, and the production process has been demonstrated during 
the development phase. The CH-47F technologies and design appear 
mature, although a low percentage of engineering drawings were released 
at the design review. Production unit costs have more than doubled due 
to contractor rate increases, increases in system capabilities, and 
initial underestimation of program cost.

[See PDF for image]

[End of figure]

CH-47F Program Technology Maturity:

Although we did not assess technology maturity in detail, the CH-47F is 
a modification of the existing CH-47D helicopter. Program officials 
believe that all critical technologies are mature and have been 
demonstrated prior to integration into the CH-47F development program.

Design Maturity:

The CH-47F design is complete, with 100 percent of the drawings 
released to manufacturing. However, at the design review only 37 
percent of the system's engineering drawings were complete. Since that 
time, the number of drawings completed has increased substantially. The 
majority of the new drawings were instituted to correct wire routing 
and installation on the aircraft, changes program officials believed 
could not be determined until after the first prototype was developed.

Production Maturity:

CH-47F production maturity could not be determined because the program 
does not use statistical process control to ensure that production 
processes are stable. Program officials believe the production is low 
risk because two prototypes have been produced and the production 
processes have been demonstrated during the development phase. The Army 
plans to conduct operational testing in fiscal year 2004 to demonstrate 
its readiness to proceed into full-rate production. Prior to that 
decision, the Army plans to complete a risk assessment for the CH-47F 
to eliminate any production risk that remains.

Other Program Issues:

Both the total cost and the program unit cost for the CH-47F production 
program have more than doubled. This growth triggered a Nunn-McCurdy 
breach (see 10 U.S.C. 2433) in December 2001, requiring a review by the 
Secretary of Defense and a report to Congress. As a result, the 
Secretary of Defense has certified to Congress that the CH-47F is 
essential for national security, there are no alternatives, the new 
cost estimates are reasonable, and the management structure is in place 
to continue to keep costs under control. According to the program 
office, the cost increases were due to (1) prime contractor labor rate 
increases and material cost growth, (2) additional system capabilities 
required by the Army, (3) recapitalization of 36 Special Operations 
aircraft, and (4) initial underestimation of program costs. According 
to the program manager, the Army has fully funded the program's cost 
growth of about $2.5 billion (then-year dollars). This increase in 
program cost necessitated rebaselining the CH-47F program. The Army 
approved the CH-47F acquisition program baseline.

Program Office Comments:

The CH-47F program office generally concurred with this assessment.

RAH-66 Comanche.

The Army's Comanche is a multi-mission helicopter intended to perform 
tactical armed reconnaissance. It is designed to operate in adverse 
weather across a wide spectrum of threat environments and provide 
improved speed, agility, reliability, maintainability, and low 
observability over existing helicopters. It is also expected to lower 
operating costs through the use of integrated diagnostics, a composite 
airframe, and a bearingless rotor system. It will replace the AH-1, 
OH-6, and OH-58 helicopters.

[See PDF for image]

[End of figure]

RAH-66 Comanche: Most of the 
Comanche's critical technologies have demonstrated acceptable levels of 
maturity, and the program appears very close to meeting the best 
practice standard for a stable design. This level of maturity follows 
many years of difficult development. Since the program's first cost 
estimate was originally approved in 1985, the research and development 
cost has almost quadrupled and the time to obtain an initial capability 
has increased from 9 years to over 21 years. The program has recently 
undergone another major restructuring to incorporate an evolutionary 
acquisition approach and reduce concurrency and lower overall risk. 
This restructuring shows promise of being a knowledge-based program 
that matches program resources with user requirements.

[See PDF for image]

[End of figure]

RAH-66 Comanche:

Technology Maturity:

Seven of the Comanche's eight critical technologies are considered 
mature. Only one critical technology, the radar cross-section needed 
for low observability, requires additional development. The Army 
expects that this technology will reach maturity in fiscal year 2005, a 
year before the production decision.

Design Maturity:

The Comanche program has released 73 percent of the engineering 
drawings to manufacturing. The program has improved its ability to 
reach design maturity by rescheduling the design review from July 2002 
to April 2003. The program estimates that it will complete 90 percent 
of the drawings by the design review under the proposed plan, instead 
of the former 59 percent under the previous program.

Critical technologies have not yet been integrated and demonstrated on 
the Comanche airframe. Prior to the proposed program restructure, 
integration of critical technologies was considered high risk, even 
though most of the technologies had reached maturity on other 
platforms. Program officials believe that the restructured program 
reduces integration risks and that the longer development schedule will 
allow for reduced concurrent development and additional integration 
time and facilities, thereby reducing critical risks.

The longer schedule also provides additional time for near-term 
development testing, use of a production representative aircraft for 
initial operational testing, and full qualification testing. 
Additionally, the phasing of development and operational tests was 
revised and expanded to reduce overall program risk.

Other Program Issues:

Continuing cost and schedule issues have led to the most recent 
restructuring of the program. In October 2002, the Office of the 
Secretary of Defense approved the Comanche program to continue under an 
evolutionary acquisition approach. However, because of uncertainties 
with future funding and capabilities, quantities were reduced from 1213 
to 650 aircraft. This reduction in quantities, combined with the 
research and development cost growth, resulted in a unit cost increase 
of approximately 62 percent. Program officials stated that the 
restructuring added a more robust internal review process and balanced 
program requirements with force requirements and program risks. Weight 
issues were addressed through increased engine performance. Initial 
operational capability was moved from December 2008 to September 2009 
to reduce risk and significantly increase the amount of testing 
conducted.

Program Office Comments:

In commenting on a draft of this assessment, program officials 
generally concurred with our assessment. They added that in October 
2002, the Office of the Secretary of Defense approved the Comanche 
program as an evolutionary acquisition approach. The Comanche quantity 
was reduced from 1213 to 650 based on emerging results of the 
Comanche's role in the Objective Force. This reduction in quantities, 
combined with the research and development cost growth, resulted in a 
program acquisition unit cost increase of approximately 62 percent. 
Excluding impacts of the quantity reduction, the average procurement 
unit cost increased 18 percent and the program acquisition unit cost 
increased 23 percent.

EX-171 Extended Range Guided Munition (ERGM).

The Navy's ERGM is a rocket-assisted projectile that is fired from a 
gun aboard ships. It can be guided to land targets at ranges of between 
about 10 and 50 nautical miles to provide fire support for ground 
troops. ERGM is expected to offer increased range and accuracy compared 
to the Navy's current gun range of 13 nautical miles. ERGM requires 
modifications to existing 5-inch guns, a new munitions-handling system 
(magazine), and a new fire control system. We assessed the projectile 
only.

[See PDF for image]

[End of figure]


EX-171 
Extended Range Guided Munition (ERGM): The ERGM program began product 
development with very few of its critical technologies mature according 
to the best practices standards. While significant progress has been 
made in the past 7 years, program officials do not expect to achieve 
maturity on all critical technologies until after the design review. No 
production representative engineering drawings had been released at the 
time of our assessment, and none are projected by the system design 
review. The program office currently expects to release these 1 year 
later. In June 2002, the program conducted a successful test of a 
guided tactical round under realistic launch conditions. This test did 
not evaluate the performance of a new warhead design.



ERGM Program:

Technology Maturity:

Fourteen of ERGM's 20 critical technologies have demonstrated 
technological maturity. The remaining 6 technologies are approaching 
maturity, and program officials expect that all 20 critical 
technologies will be demonstrated in an operational environment by the 
end of 2003, approximately 7 months after the design review. Three of 
the technologies yet to reach maturity are part of the new unitary 
warhead design, and a fourth is related to this change. Program 
officials recently identified the unitary warhead's safe/arm device and 
fuze as a critical technology, after a Navy safety review concluded 
that it needed to be redesigned to meet applicable DOD standards.

The ERGM program began development with only one of its critical 
technologies mature. Having only one critical technology mature at the 
start of product development has caused cost and schedule problems. For 
example, when the program began, none of the components of the rocket 
motor had been integrated into an ERGM representative design. 
Subsequent problems with the performance reliability of the motor 
resulted in cost growth of more than $13 million.

Design Maturity:

None of ERGM's approximately 127 production representative engineering 
drawings have been released to manufacturing. The program office plans 
for all of these drawings to be released in June 2004, about one year 
after the design review. In the meantime, the design review will be 
used to validate the design of the development test rounds. The June 
2004 drawing release, which will reflect knowledge gained from 8 of 18 
flight tests and some qualification tests, will be used to build the 80 
production representative operational test rounds. Program officials 
pointed out that progress has been made in maturing the design. For 
example, the main elements of the design were validated during the 
guided gunfire test in June 2002.

In January 2002, in order to meet lethality and safety requirements, 
the Navy decided to make a significant change to the warhead design, 
moving from a multiple-submunition design to a single explosive--or 
unitary--warhead. This decision, coupled with the decision to stay 
within planned funding levels for fiscal years 2002 and 2003, stretched 
out program milestones and will delay deployment of ERGM until 2006.

Other Program Issues:

Future program costs are not accurately reflected in the latest program 
cost estimate and the fiscal year 2004/2005 budget request. The cost 
estimate is based on a much lower production quantity than is contained 
in either the approved or the current draft revision of the ERGM 
acquisition program baseline. The budget request does not fully fund 
the 80 operational test rounds currently required.

Two testing issues could affect the program. The Director of 
Operational Test and Evaluation has raised a concern about test range 
restrictions that could limit realistic operational testing. Finally, 
the project manager stated that the availability of a fully capable 
ship to support development testing could be an issue due to funding 
shortfalls for magazine modifications on these ships.

Some of the cost increases and schedule slippages to date may be 
attributed to the fact that the contractor relocated the program in 
1998, resulting in a loss of trained personnel and development inertia.

Program Office Comments:

In commenting on a draft of this assessment, program officials stated 
that although production representative drawings will not be available 
at the design review, the entire ERGM design would be under 
configuration control. Design maturity at that time will be sufficient 
to produce all-up rounds for land and ship-based development testing. 
Based on data obtained from flight and qualification tests in fiscal 
year 2004, minor revisions to the ERGM technical data package may be 
made. Production representative drawings will be finalized by June 
2004. Program officials stated that they are highly encouraged by the 
significant progress in ERGM development activities over the last 18 
months. They further stated that they have a high degree of confidence 
that ERGM will meet all performance requirements, while meeting the 
production cost goals specified in the acquisition program baseline.

Excalibur Artillery Round.

The Army's Excalibur is a family of extended range, precision, 155-mm 
artillery projectiles. It is designed to increase soldier survivability 
by allowing the Future Combat Systems' nonline of sight cannon to fire 
from farther away and defeat threats more quickly, while reducing 
logistic support. It also is intended to be more effective when fired 
at urban targets, through a combination of altered trajectory and 
global positioning system accuracy.

[See PDF for image]

[End of figure]

Excalibur Artillery Round: 
The Excalibur program's three critical technologies are not fully 
mature, even though product development began over 5 years ago. The 
technologies appear to be approaching maturity, and program officials 
project demonstrating technology and design maturity before the design 
review in 2005. Currently, 13 percent of the drawings are at the level 
that could be released to manufacturing. Program officials expect to 
have a stable design by the design review. The program has undergone a 
major restructuring effort. It has encountered a number of challenges 
since development began, including a substantial decrease in planned 
quantities, a relocation of the contractor's plant, limited early 
funding, technical problems, changes in program direction, and a merger 
with another program.



Excalibur Program:

Technology Maturity:

None of the Excalibur's three critical technologies are fully mature 
according to best practice standards. According to program officials, 
all three have been demonstrated in a relevant environment and are 
expected to reach maturity before the design review in March 2005. The 
Excalibur's design and requisite technologies have changed since 
product development was started. The three critical technologies for 
the current design are the guidance control system, the airframe, and 
the warhead. The warhead was not considered a critical technology in 
1997 because the Excalibur design called for a warhead that was under 
production for other munitions. Based on Army direction, the program 
has undertaken development of a different warhead that is currently 
undergoing testing.

Design Maturity:

About 13 percent of the Excalibur's engineering drawings are at a level 
that could be released to manufacturing. The program office plans to 
have all of its drawings complete and released to manufacturing by the 
design review in March 2005. However, program officials could not 
estimate the total number of drawings expected.

Other Program Issues:

The program has gone through many changes since the beginning of 
product development in May 1997. It was almost immediately restructured 
due to limited funding, and it was restructured again in 2001. In 
response to congressional direction, the program was restructured to 
merge with the joint Swedish/U.S. program known as Trajectory 
Correctable Munitions. The merger should help the program deal with 
design challenges, including issues related to its folding fin design. 
Also, in May 2002, the Office of the Secretary of Defense directed the 
program to develop the Excalibur for the Future Combat Systems nonline 
of sight cannon and to field it in fiscal year 2008.

Although program officials have not yet released the new cost and 
schedule estimates, the net effect of these changes has been to 
increase the program's schedule and to substantially decrease planned 
procurement quantities. As a result, the program's overall costs and 
unit costs have dramatically increased.

Program Office Comments:

In commenting on a draft of this assessment, program officials 
generally agreed with the information in this report. However, they 
provided the following clarifying comments.

Concerning the Excalibur design maturity, program officials stated that 
approximately 600 drawings are anticipated at the subsystem level. But 
because the program is still in research and development, no drawings 
have been officially released to manufacturing. The program is 
fabricating hardware in a research and development environment.

F/A-18E/F Super Hornet.

The Navy's F/A-18E/F is a multi-mission tactical aircraft designed to 
meet fighter escort, interdiction, fleet air defense, and close air 
support mission requirements. The program was approved as a major 
modification to earlier F/A-18 aircraft in 1992. It is intended to 
complement and replace the Navy's F/A-18C/D and F-14 aircraft.

[See PDF for image]

[End of figure]

F/A-18E/F Super Hornet: The 
F/A-18E/F went into full-rate production in June 2000. Although the 
program proceeded without obtaining full product knowledge at key 
decision points, it embraced the concepts of attaining design and 
manufacturing knowledge early in development. The program released just 
over half of its engineering drawings by its design review. When low-
rate production began, nearly all of the drawings were released and 
about 75 percent of the manufacturing processes were in control. The 
Navy reduced some program risk because aviation electronics from an 
earlier version of the 
F/A-18 were incorporated into the baseline 
F/A-18E/F. Furthermore, focus was placed on commonality between the F/
A-18 C/D and the 
F/A-18 E/F, which further reduced risk.



F/A-18E/F Program:

Technology Maturity:

We did not assess the technology maturity of the 
F/A-18E/F program because it is already in full-rate production. 
Nevertheless, we did not identify any technical challenges during the 
development of this aircraft in our previous reviews.

Design Maturity:

The F/A-18E/F design appears complete. The program has released 100 
percent of the design drawings to manufacturing. At the time of the 
critical design review in July 1994, 56 percent of the engineering 
drawings were completed and released to manufacturing for aircraft 
structure and systems. According to program officials, they decided to 
proceed despite the low level of completed drawings because the 
knowledge gathered from earlier 
F/A-18C/D models gave them confidence that the design was stable. By 
the time of the low-rate initial production decision, 99 percent of the 
drawings had been released.

Production Maturity:

According to program officials, they currently have 100 percent of 
their critical manufacturing processes under control, according to the 
best practice standard. Therefore, they are no longer tracking 
processes using statistical process control. However, defects are still 
monitored through inspections, failures, and age exploration testing, 
and during maintenance. If production problems are identified, the 
program would resume statistical process control analysis where 
necessary.

Program officials estimate that about 75 percent of key manufacturing 
processes were in control at the low-rate production decision in March 
1997. Program officials stated that they concentrated on maturing their 
manufacturing processes before starting production. As a result of 
these efforts, labor efficiency rates have steadily improved.

Other Program Issues:

The F/A-18E/F will not reach its full potential until after the 
incorporation of several preplanned upgrades--the Active 
Electronically Scanned Array (AESA) radar, the Joint Mounted Helmet 
Cueing System coupled with the AIM-9X missile, and the Advanced 
Targeting Forward Looking Infrared sensor. The level of effort and 
timing to incorporate some of the sensors--the AESA radar and the 
Advanced Targeting sensor--may prove to be a challenge. We have 
assessed the AESA radar elsewhere in this report.

Program Office Comments:

In commenting on a draft of this assessment, program officials stated 
initial schedule delays were due to a procurement reduction of 10 
aircraft in a 1998 Program Objective Memorandum. Since that time, the 
contractor has consistently delivered aircraft ahead of schedule. 
Program officials also noted that the aircraft are demonstrating two to 
three times the quality of the F/A-18C/D and have provided measurable 
improvements to squadron readiness. In addition, all F/A-18E/F 
preplanned upgrades continue to track to their program schedules. The 
Joint Mounted Helmet Cueing System has completed operational 
evaluation, and the system has been incorporated into lot 24 of the 
aircraft (deliveries of which began in September 2001). Deliveries of 
the Advanced Targeting Forward Looking Infrared Sensor production units 
began in April 2002, and the units were deployed in January 2003. 
Finally, program officials stated that the AESA radar program continues 
to execute as planned, and the program has received the first 
engineering and manufacturing development unit.

F/A-22 Raptor.

The Air Force's F/A-22, originally planned to be an air superiority 
fighter, will also have air-to-ground attack capability. It is being 
designed with advanced features, such as stealth characteristics, to 
make it less detectable to adversaries and capable of high speeds for 
long ranges. It also has integrated aviation electronics (avionics) 
designed to greatly improve pilots' awareness of the situation 
surrounding them. It is designed to replace the Air Force's F-15 
aircraft.

[See PDF for image]

[End of figure]

F/A-22 Raptor: Because the F/A-22 
Program Office stopped collecting process control data in 2000, the 
program began production in 2001 with no proof that processes were in 
control, as defined by best practice standards. Technology appears 
mature and the design appears stable; however, problems with the 
vertical tail and the avionics have been discovered recently, which 
require design modifications. Delays in capturing technology, design, 
and production knowledge and these latest problems contributed to cost 
increases and schedule delays. The potential exists for further cost 
increases and schedule delays as a significant amount of the test 
program remains, including operational tests. Also, the latest 
production cost estimate is likely to increase because of several 
factors, and the estimate assumes over $25 billion in offsets from cost 
reduction plans.



F/A-22 Program:

Technology Maturity:

Although we did not assess the F/A-22 key technologies using technology 
readiness levels, the three critical technologies (supercruise, 
stealth, and integrated avionics) appear mature. Two of these 
technologies, integrated avionics and stealth, were late to mature. It 
was not until September 2000, or over 9 years into product development, 
that the integrated avionics reached maturity. During development, the 
integrated avionics was a source of schedule delays and cost growth. 
Since 1997, avionics software development and flight-testing have been 
delayed, and the cost of avionics development has increased by over 
$980 million. Moreover, the Air Force did not complete an evaluation of 
stealth technology on a full-scale version of the aircraft until 
several years into product development.

Design Maturity:

The basic design of the F/A-22 is essentially complete, as engineering 
drawings are complete. However, design changes have been necessary as a 
result of flight tests and structural tests. For example, problems with 
excessive movement of the vertical tails and avionics failures in 
flight tests were discovered, and they will require costly design 
modifications. The Air Force still has to complete a significant amount 
of development testing and operational testing. Until initial 
operational testing is completed as planned in June 2004, the 
possibility of additional design changes remains.

Design knowledge for the F/A-22 was built slowly. Only 26 percent of 
the total drawings were released at the 1995 design review. The program 
released 90 percent of the drawings over 3 years later, after the first 
two development aircraft had been delivered. Late drawing release 
contributed to parts shortages and work performed out of sequence 
during assembly, which drove up costs and contributed to delaying 
flight tests by 83 months.

Production Maturity:

The program office stopped collecting process control information in 
November 2000. The contractor estimated that nearly half of the key 
processes had reached a marginal level of control, but not up to best 
practice standards. In September 2001, the Air Force awarded a contract 
for 10 aircraft to begin F/A-22 production.

Other Program Issues:

In September 2001, the Air Force acknowledged an estimated production 
cost increase of $5.4 billion (then-year dollars) over the 
congressional cost limit. We believe conditions exist that makes it 
likely production costs will increase even further. In addition, the 
Air Force is counting on over $25 billion in cost reduction plans to 
offset estimated cost growth and enable the program to meet the 
production cost estimate. If these cost reduction initiatives are not 
achieved as planned, production costs could increase. Further, the 
contractor has yet to demonstrate it can efficiently build the 
development aircraft, and estimates of the cost to build the production 
aircraft continue to increase.

In December 2002, DOD estimated development costs would increase by 
$876 million and that the funding necessary to cover this cost increase 
would be transferred from production funding. Avionics problems 
discovered in flight-testing are the primary contributor to a six-month 
extension to the development program.

Program Office Comments:

In commenting on a draft of this assessment, program officials stated 
that the report implies that had the F/A-22 deferred product 
development until engineering and testing were accomplished, at a level 
providing higher product knowledge, substantial cost increases and 
schedule delays would have been prevented. The issues cited as examples 
do not pose a substantial risk to either cost or schedule and have 
either been fixed through minor design change or are anticipated to be 
resolved without major impact to continued testing and production. A 
program of this nature is expected to have both design and 
technological maturities to overcome and there will be some element of 
risk throughout its development and into the production process.

Joint Air-to-Surface Standoff Missile (JASSM).

JASSM is a joint Air Force and Navy program designed to attack surface 
targets outside of the range of area defenses. JASSM will be delivered 
by a variety of aircraft, including the F-16 C/D, the B-52H, the F/A-
18E/F, the B-2, and the B-1B. The system includes the missile, 
software, and software interfaces with the host aircraft and mission 
planning system.

[See PDF for image]

[End of figure]


Joint 
Air-to-Surface Standoff Missile (JASSM): The JASSM program entered 
production in December 2001 without ensuring that production processes 
were in control, according to best practice standards. However, program 
officials indicated that they have demonstrated the production 
processes and that they sample statistical data at the subsystem level. 
The program ensured that the technology was mature and that the design 
was stable at critical points in development, closely tracking best 
practice standards. Redesign remains one area of concern because recent 
test failures have led to the delay of operational tests. The program 
has identified fixes to the problems, and a retrofit plan is in 
progress. The contractor's ability to attain a higher production rate 
is another area of concern.

[See PDF for image]

[End of figure]

JASSM:

Technology Maturity:

The JASSM program used existing technologies and so its level of 
technology maturity is high. Although none of the subsystems involve 
new technologies, three critical technologies are new applications of 
existing technologies. These three technologies are the global 
positioning system anti-spoofing receiver module, the low observable 
technology, and the composite materials. The program office reports 
these technologies to be mature.

Design Maturity:

The contractor has released 100 percent of the drawings to 
manufacturing. The two remaining concerns are the software for the 
missile and the status of integration with aircraft, although program 
officials believe the risks are low.

Recent failures in development and operational tests have led to the 
delay of the remaining JASSM operational tests. During an operational 
test on October 10, 2002, the missile flew its planned route and 
penetrated the target, but it failed to detonate. According to program 
officials, this failure occurred because the requested test methodology 
was experimental and exceeded original design requirements for the 
fuze. On October 24, 2002, during the last of 11 developmental tests, 
the missile went out of control and crashed at the test site. According 
to program officials, this failure was due to a failed actuator. 
Program officials believe they have identified the problems in both 
cases and have a retrofit plan. Retrofits will be tested in spring 
2003. However, if additional problems occur, they will have to be 
corrected while JASSM is in production, which may require additional 
retrofitting of missiles already produced.

Production Maturity:

Program officials do not collect production process control data at the 
system level. However, they stated that all production processes had 
been demonstrated and that statistical data is collected at the 
subsystem level and is sampled as required. Program officials indicated 
that the contractor will produce at the rates required for the first 
production lot and 76 missiles will be delivered. A contract for the 
second lot, 100 missiles, has been signed. Production concerns 
remaining include achieving full-rate production capacity and expanding 
facilities to support full-rate production plus anticipated foreign 
military sales. Program officials believe that none of the 
manufacturing processes that affect critical system characteristics are 
problematic, although there are key production processes that have cost 
implications, such as the bonding for the low observable materials and 
the painting/coating application.

Program Office Comments:

In commenting on a draft of this assessment, program officials stated 
that JASSM has established a new benchmark for missile development by 
ensuring weapon system design maturity and production capability were 
demonstrated during development prior to entering low-rate initial 
production. JASSM's acquisition strategy incorporated existing 
technology to reduce program risk and speed up delivery of the weapon 
to the warfighter. The officials further stated that JASSM's 
development cycle is 33 percent faster than comparable weapon systems, 
with production unit prices 50 percent less than weapon systems with 
less capability. The contractor was contracted to produce 82 all-up 
production prove-out test rounds during development on the production 
line prior to low-rate initial production missile delivery. Program 
officials noted that establishment of production representative 
hardware during development was key to the contractor's ability to 
prove out all production processes. The contractor has a capitalization 
plan to meet full-rate production quantities.

Joint Common Missile.

The Joint Common Missile is an air-launched and potentially a ground-
launched missile designed to target tanks; light armored vehicles; 
missile launchers; command, control, and communications vehicles; 
bunkers; and buildings. It is designed to provide line of sight and 
beyond line-of-sight capabilities. It can be employed in a fire-and-
forget mode--providing maximum survivability--or a precision attack 
mode, providing the greatest accuracy. The Joint Common Missile will be 
a joint Army and Navy program with USMC participation.

[See PDF for image]

[End of figure]

[See PDF for image]

[End of figure]


Joint Common Missile System:

Technology Maturity:

None of the Joint Common Missile's three critical technologies have 
demonstrated full maturity. These critical technologies include a 
multi-mode seeker for increased countermeasure resistance, a boost-
sustain propulsion for increased standoff range, and a multi-purpose 
warhead for increased lethality capability. Program officials noted 
that many of the components of these technologies are currently in 
production on other missile systems, but that they have not been fully 
integrated. While backup technologies exist for each of the critical 
technologies, substituting any of them would result in degraded 
performance or increased costs.

Design Maturity:

Program officials project that full integration of the subsystems into 
the Joint Common Missile will be mature one year after the system 
design review, which is scheduled for July 2004.

Other Program Issues:

The current cost estimates are from the fiscal year 2004 President's 
budget. This cost estimate will be updated at the conclusion of the 
Army's formal estimating process. The formal estimating process began 
in January 2003 for presentation at the milestone decision review in 
September 2003. According to program officials the Army's acquisition 
objective is 54,400 missiles and the Navy's acquisition objective is 
23,000. Program officials also indicated that the modular design will 
reduce life-cycle costs, including demilitarization, and will enable 
continuous technology insertion to ensure improvements against 
advancing threats.

Program Office Comments:

In commenting on a draft of this assessment, program officials stated 
that they plan to demonstrate the technological maturity required by 
DOD acquisition system policy before beginning the development phase in 
September 2003. Program officials further stated that the technological 
maturity projected represents a major achievement in the technology's 
demonstrated readiness in a relevant environment and provides the 
critical technologies the maturity necessary to accomplish system 
integration of demonstrated subsystems, thereby reducing program risk. 
Prototype testing of a multi-mode seeker (tower and captive flight), a 
multi-purpose warhead (heavy armor and building structures), and a 
rocket motor (high maximum to minimum thrust profiles over operational 
temperatures) is currently being conducted with results to be available 
in sufficient time to support the milestone decision to begin the 
development phase.

Joint Primary Aircraft Training System (JPATS).

JPATS is a joint acquisition by the Air Force and the Navy to replace 
the aging primary trainer aircraft fleet. JPATS is a variant of the 
Beech Pilatus PC-9 commercial aircraft, but it has been modified 
significantly to incorporate military unique requirements. The JPATS 
program includes the aircraft; the ground-based training system 
(simulators, course materials), and an integrated training management 
system. We assessed the aircraft.

[See PDF for image]

[End of figure].

Joint 
Primary Aircraft Training System (JPATS): The JPATS aircraft entered 
full production in December 2001 without ensuring that the 
manufacturing processes were mature. The aircraft entered limited 
production in 1995 before achieving design stability. DOD considered 
the aircraft a mature commercial product that did not require extensive 
product development. However, program officials underestimated the 
number of design changes needed to accommodate the military unique 
requirements. The design has subsequently changed about 70 percent from 
the commercial baseline. The JPATS initial operating capability 
occurred in 2002, 2 years later than originally planned.

[See PDF for image]

[End of figure]

JPATS Program:

Technology Maturity:

Although we did not assess the JPATS aircraft key technologies, the 
aircraft is a derivative of a commercial aircraft and the technologies 
appear mature.

Design Maturity:

The basic design of the aircraft is currently complete. However, the 
military unique design was only about 5 percent complete shortly after 
the program was approved to enter limited production in 1995. The 
design has changed about 70 percent from its commercial baseline. 
Testing has revealed tangible examples of design immaturity. Several 
subsystems, including the engine, the UHF radio, and the environmental 
control system, have required extensive modification or redesign. These 
and other problems have delayed both aircraft testing and the 
production decision.

In November 2001, operational testers concluded that JPATS was 
operationally effective but not operationally suitable. They cited 
concerns about the aircraft's reliability, availability, and 
maintainability. They also reported that the full JPATS had not yet 
been tested due to uncorrected deficiencies in the aircraft and the 
immaturity of the software-intensive training information management 
system. The contractor is incorporating changes to the aircraft as a 
result of operational test issues. Operational testers expressed 
concern that some changes may adversely impact other critical 
subsystems. Despite these issues, the Air Force proceeded into full-
rate production the following month.

Production Maturity:

Production maturity remains at issue because information about the 
contractor's manufacturing process controls is not available. The Air 
Force did not require this information because the aircraft was 
considered a commercial derivative.

Other factors could affect production maturity. In 2002, two key 
modifications--the environmental control system and the UHF radio--
began to be incorporated on the aircraft. The program office has also 
identified additional retrofit requirements and is evaluating a 
replacement for the collision warning system. The rework associated 
with these changes may affect aircraft production efficiencies.

Program Office Comments:

In commenting on a draft of this assessment, program officials 
disagreed with our analysis of production maturity. They stated that 
statistical process control is not the only determinant of maturity. 
The production line was certified by the International Organization for 
Standardization in 1994 and by the Federal Aviation Administration in 
1999, and is currently producing aircraft according to these 
guidelines. Assembly labor hours per aircraft are on a 78 percent 
learning curve, and they have decreased 65 percent since the first 
operational aircraft was delivered. The production line rate increased 
to five aircraft per month by the end of 2002, and remains there still, 
even as design changes are incorporated into the production line. After 
initial production difficulties, over the past year the contractor has 
been delivering aircraft ahead of schedule while incorporating 
engineering changes to increase the suitability of the system. Program 
officials also stated that the cycle time should be reduced by 6 months 
because the JPATS program was unable to award a contract or proceed 
with contract performance pending the disposition of several bid 
protests.

GAO Comments:

Our prior work has shown that leading commercial firms rely on 
statistical control data as the best indicator of production readiness. 
Despite its commercial origins, the JPATS program entered limited and 
full production without this information. Subsequent testing has 
uncovered numerous problems that require modification and retrofit. 
Although the aircraft has been production certified by the Federal 
Aviation Administration, its regulations merely require the contractor 
to maintain a generic quality control system and do not provide 
assurance that the components can be built within cost and on schedule. 
We used DOD official documents to determine acquisition cycle time.

F-35 Joint Strike Fighter (JSF).

The JSF program goals are to develop and field a family of stealthy, 
strike fighter aircraft for the Navy, Air Force, Marine Corps, and U.S. 
allies, with maximum commonality to minimize life-cycle costs. The 
carrier suitable version will complement the Navy F/A-18 E/F. The Air 
Force version will primarily be an air-to-ground replacement for the 
F-16 and the A-10 and complement the F/A-22. The short take-off and 
vertical landing version will replace the Marine Corps F/A-18 and AV-
8B. Significant foreign military purchases are expected.

[See PDF for image]

[End of figure]

F-35 Joint Strike 
Fighter (JSF): The JSF program entered the development phase without 
demonstrating that its eight critical technologies had reached maturity 
according to best practice standards. Two technologies, propulsion and 
critical fabrication techniques, were very close to maturity. DOD 
conducted an independent review in 2001 and concluded that the 
technology maturity was sufficient to proceed into product development. 
The JSF program no longer focuses on the previous 8 technology areas, 
instead it uses a different method of integration and risk management 
that currently tracks 23 program level risks. We were unable to assess 
the new risk areas, but program data indicates that the majority are 
moderate risk. The program expects to have 80 to 90 percent of its 
critical build-to-packages completed by the final design review in 
2005.

[See PDF for image]

[End of figure]

JSF Program:

Technology Maturity:

During its concept development phase, the Joint Strike Fighter had 
eight critical technologies: short take-off vertical landing/
integrated flight propulsion control, prognostic and health management, 
integrated support systems, subsystems technology, integrated core 
processor, radar, mission systems integration, and manufacturing. We 
reported in May 2000 and again in October 2001 that low levels of 
maturity in these technologies could increase the likelihood of program 
cost and schedule growth.

The program experienced cost growth and schedule concerns during the 
concept demonstration phase, prior to starting product development in 
October 2001. This included manufacturing delays for hardware used on 
the propulsion system for the Marine Corps version. To reduce cost and 
schedule delays, the program eliminated planned risk-reduction efforts 
and delayed other technology demonstrations until after product 
development began.

An independent review performed by DOD in 2001, using a different 
method than technology readiness levels, concluded that the overall 
technology maturity of the JSF program was sufficient to enter into 
product development. Today, the program no longer monitors the eight 
specific technologies from the previous phase. Instead, the program is 
using Lockheed Martin's Key System Development Integration approach to 
monitor overall technology development and design integration. Further, 
the program tracks 23 program level risk areas and has assessed 19 as 
moderate and 2 as high. Five of eight critical technologies from the 
concept development phase are contained within elements of these 
program level risks. We have not evaluated the current JSF technique 
for assessing risks.

Design Maturity:

The program has committed time and funding to the system development 
and demonstration phase that should improve its chances for success. 
Specifically, the new program structure will now include additional 
test aircraft, increased software on the aircraft, and a greater number 
of flight test hours. Program documents indicate that the 1996 
estimated cost and schedule for JSF's development phase have increased 
by 56 percent and 40 percent, respectively, due to changes in program 
scope. Meetings were held for the JSF preliminary design review in late 
March 2003. We were unable to review the results of those meetings 
prior to the release of this report, but program office data indicates 
the discovery of higher risk levels for the propulsion system and 
overall aircraft weight.

Other Program Issues:

Due to the highly complex nature of the JSF design, the Director, 
Operational Test and Evaluation, expects numerous test challenges for 
the program. These challenges include the integration of highly 
advanced sensors with the avionics systems, vertical thrust capability 
for the Marine Corps version, and performance and maintenance 
requirements of the low observable capabilities. The program has 
received authority for its low-rate production quantity to reach 15 
percent--427 aircraft--of the total production run.

Program Office Comments:

In commenting on a draft of this assessment, program officials stated 
that, prior to the start of the development phase, JSF's key 
technologies had gone through an extensive series of tests and 
demonstrations, culminating in four experimental aircraft proving 
flight capabilities for each service variant in over 200 hours of 
flight. An independent DOD review concluded that JSF had demonstrated 
sufficient technical maturity for low risk entry into the development 
phase. For this phase, the program officials stated that JSF has 
adapted the contractor's risk mitigation approach. Risk mitigation 
assessments in February 2003 indicated that most program level risks 
were rated moderate using the contractor's approach. Cost and schedule 
planning for the development phase has evolved as the services iterated 
system operational requirements with life cycle cost. The JSF air 
system preliminary design review is scheduled in March 2003, and the 
first of three critical design reviews is to occur in April 2004. 
Finally, program officials stated that the program is being executed in 
accordance with its cost, schedule, and technical baselines.

Joint Standoff Weapon (JSOW).

JSOW is a joint Air Force and Navy guided bomb to attack targets from 
outside of the range of most enemy air defenses. There are three JSOW 
variants that use a common air vehicle. Two variants (JSOW A and B) 
carry submunitions to attack soft targets or armored vehicles. The 
unitary variant (JSOW C) uses a seeker, autonomous targeting 
acquisition software, and a single warhead to attack targets. We 
assessed the unitary variant and the common air vehicle.

[See PDF for image]

[End of figure]


JSOW Program:

Technology Maturity:

The JSOW unitary's technology appears mature. The program office 
identified the imaging infrared seeker with the autonomous acquisition 
software as the only critical technology for the system. The seeker was 
not mature at the start of development, but it did demonstrate maturity 
in October 2001--over three-fourths through development--when it was 
flown aboard an aircraft in a captive flight test. Program officials 
stated that in three free-flight tests, the seeker's performance 
substantially exceeded requirements.

Design Maturity:

The JSOW unitary variant's basic design appears complete. At the system 
design review in May 2002, the program office had completed 99 percent 
of the drawings. The Navy included nine developmental tests in its 
development program--three sled tests with the warhead, three free 
flights with the seeker, and three combined warhead/seeker tests. The 
Navy has completed two of the warhead sled tests and the seeker free-
flight tests.

Production Maturity:

JSOW production maturity could not be determined because the contractor 
does not use statistical process controls to ensure that production 
processes are stable and units are produced with few, if any, defects. 
Rather, the contractor uses a process of post-production inspection to 
control production quality. The contractor collects this postproduction 
data on a factorywide basis that includes JSOW production but is not 
specific to it.

According to the program office, the contractor delivered end items in 
the past that included manufacturing defects. The program office 
attributes these defects at least partially to suppliers and to 
reorganization and relocation of the prime contractor to Tucson, 
Arizona. To mitigate the risk of further manufacturing problems, the 
Navy has instituted a series of reviews of major suppliers. The Navy 
will conduct an additional production readiness review after the low-
rate production is approved. Program officials report that the 
contractor is meeting its revised production schedule and that the 
scrap and rework rates remain low.

Program Office Comments:

In commenting on a draft of this assessment, program officials stated 
that the contractor has completed 17 consecutive months of on-schedule 
deliveries, increasing the inventory to over 850 combat ready assets. 
In addition, program officials noted that the Air Force has upgraded 
its JSOW inventory to mission ready as a result of a successful 
resolution of remaining manufacturing, navigation, and vibration 
tolerance issues. The JSOW unitary continues development and its 
performance is being monitored by the program office.

National Polar-orbiting Operational Environmental Satellite System 
(NPOESS).

The NPOESS is a joint National Oceanic and Atmospheric Administration 
(NOAA), DOD, and National Aeronautics and Space Administration 
satellite program to monitor the weather and environment. Current NOAA 
and DOD satellites will be merged into a single national system 
(NPOESS), with projected savings of at least $1.3 billion. The program 
consists of five segments: space; command, control, and communications; 
interface data processing; launch; and field terminals.

[See PDF for image]

[End of figure]

National Polar-orbiting Operational 
Environmental Satellite System (NPOESS): The NPOESS program entered 
product development in August 2002 with most of its technologies 
mature. The program also completed a significant portion of the 
engineering drawings well in advance of the design review; however, the 
total number has yet to be determined. Over 5 years ago, program 
officials considered the program to have several high-risk areas. Since 
then, officials have implemented several efforts, which are expected to 
reduce all program areas to low risk by the first NPOESS launch, 
currently scheduled for the 2008-2009 time frame. Perhaps the most 
significant step taken to reduce risk was to put the pacing space 
sensor technologies into full development in advance of the satellite 
system itself.

[See PDF for image]

[End of figure]

NPOESS Program:

Technology Maturity:

The NPOESS spacecraft and the sensors under development consist of 14 
key technologies; twelve were mature at the start of development in 
August 2002.

In 1997, the program office determined that the space segment had high 
cost and technical risks and that the interface data processing segment 
and overall system integration effort had high cost, schedule, and 
technical risks.

To reduce the risk to the data processing segment, two contractors 
selected for program definition and risk reduction each conducted four 
ground-based demonstrations of the data processing hardware and 
software components. Therefore, the program office expects the data 
processing segment to be relatively mature before product development.

Program officials indicated that they achieved maturity by 
concentrating on the early development of key individual sensors. The 
acquisition strategy focused on maturing key sensor technologies using 
individual development contracts structured to demonstrate the maturity 
of each sensor through a component-level design review prior to the 
system-level design review. The two technologies that are not mature 
are needed for two key sensors--the cross-track infrared sounder and 
the conical microwave imager/sounder. However, program officials 
project that those two technologies will be mature by the system design 
review in 2005.

Design Maturity:

Although the total number of engineering drawings has yet to be 
determined, program officials indicated that at least 52 percent of the 
6,829 currently identified drawings were completed and released to 
manufacturing by the end of January 2003. Program officials further 
project that all of the currently identifiable drawings will be 
complete by the system design review in 2005.

The program is taking advantage of a unique opportunity to demonstrate 
design maturity. The NPOESS Preparatory Project, a planned 
demonstration satellite, is to be launched in 2006, about 2 to 3 years 
before the first NPOESS satellite launch. The demonstration satellite 
is scheduled to carry four critical sensors--the visible/infrared 
imager radiometer suite, the cross-tracked infrared sounder, the 
advanced technology microwave sounder and the ozone mapper/profiler 
suite. This satellite will provide the program office and the data 
processing centers with an early opportunity to work with the sensors, 
ground control, and data processing systems, thus allowing lessons 
learned to be incorporated into the NPOESS satellites.

Program Office Comments:

The NPOESS integrated program office concurred with this assessment.

Patriot Advanced Capability 3 (PAC-3) Program.

The Army's Patriot system is a long-range, high-medium altitude air and 
missile defense system. PAC-3 is designed to enhance the Patriot's 
ability to detect and identify missiles and other targets, increase 
system computer capabilities and the number of missiles in each 
launcher, improve communications, and incorporate a new hit-to-kill 
missile. The PAC-3 system has two primary components, the fire unit and 
the missile. We assessed both components.

[See PDF for image]

[End of figure]

Patriot Advanced Capability 3 (PAC-3) Program: The PAC-3 program 
currently has only about one-fourth of its critical production 
processes under statistical control using best practice standards. 
Continuing problems with producing and testing the missiles are 
partially explained by the absence of process control and partially a 
consequence of maturing PAC-3's design late in development. Technical 
and design challenges disrupted the early part of product development, 
causing cost and schedule increases and delays in attaining production 
knowledge. PAC-3's basic design is now complete and the technology 
appears mature. However, the contractor must increase production 
earlier than planned because DOD decided to accelerate deliveries. This 
decision may present new production challenges because the contractor 
must find and train additional personnel.



Patriot PAC-3 Program:

Technology Maturity:

Although we did not assess the PAC-3 technologies using technology 
readiness levels, the system's critical technologies appear mature. 
However, a key technology, the Ka band seeker, was particularly late to 
mature. The seeker did not mature until 1999, close to the low-rate 
production decision. Problems experienced during development increased 
the seeker's cost by 76 percent and delayed the contractor in attaining 
design and production knowledge.

Design Maturity:

PAC-3's basic design is complete, with 100 percent of the drawings 
released to manufacturing. Only 21 percent of the drawings were 
complete when the program held its design review, which led to a number 
of problems. For example, the contractor attributed a $101 million cost 
increase to first-time manufacturing problems, such as some subsystems 
not fitting together properly and some not passing ground or 
environmental tests. These problems were a major contributor to a 2-
year schedule delay. To reduce missile costs, the contractor has 
identified several major design changes, which will be incorporated 
into the design in 2004.

Production Maturity:

The program has 23 percent of the key manufacturing processes used to 
assemble the missile and the seeker under control. Production maturity 
has deteriorated from the 35 percent that was in control at the October 
1999 low-rate production decision. A switch in the manufacturing 
facilities may have played a role. According to program officials, the 
program entered production before process control was emphasized to the 
contractor. The contractor is still having difficulties building the 
missile. For example, each seeker still needs to be reworked about 
three times on average before it passes quality inspections. Program 
officials have added quality tests of components, which have improved 
the situation, but the contractor has not yet demonstrated that these 
tests will eliminate the need for seeker rework in the future.

Other Program Issues:

The Army conducted four operational tests in 2002; none were completely 
successful. The PAC-3 system defeated half of the targets in flight-
testing. System performance was adversely affected by PAC-3 missile 
reliability and launch failures. According to program officials, there 
were several anomalies caused by manufacturing practices, software, and 
test hardware. However, they believe there are no systemic issues and 
the anomalies have been corrected. A flight test to validate these 
corrections is scheduled for the spring of 2003.

The program has adopted an evolutionary acquisition approach, with 
production decisions every 2 to 3 years. In October 2002, DOD decided 
to buy 208 missiles covering the next 2 years. DOD plans to accelerate 
the production rate immediately by adding a second manufacturing shift 
and test equipment. Because production was not expected to be 
accelerated to this level this early in production, the contractor must 
expeditiously find and train qualified personnel. The accelerated plan 
requires additional funding of $239 million for fiscal years 2003 and 
2004.

Program Office Comments:

In commenting on a draft of this assessment, program officials stated 
that they believe production processes are in control. Program 
officials stated that they have meticulously and methodically examined 
every critical process from a labor and inspection standpoint to help 
ensure a consistent and quality product. Despite the less than fully 
successful operational tests, they also believe that they have the most 
successful development flight test program in the history of missile 
development. They provided technical comments, which were incorporated 
as appropriate.

Space Based Infrared System (SBIRS) High.

SBIRS High will consist of a constellation of four satellites plus one 
spare, two sensors on a non-SBIRS satellite, and associated ground 
stations. SBIRS High is to provide missile warning and missile defense 
information and will be used to support the technical intelligence and 
battlespace characterization missions. The first launch of SBIRS High 
is scheduled for fiscal year 2007.

[See PDF for image]

[End of figure]

Space Based 
Infrared System (SBIRS) High: The SBIRS High program's critical 
technologies have demonstrated acceptable levels of maturity. This 
level of maturity follows many years of difficult development. The 
level of design stability is unknown since the contractor was unable to 
provide information on the total number of releasable drawings at 
specific milestones. Similarly, production maturity could not be 
determined because the contractor does not collect statistical control 
data. The SBIRS High program is building the first two satellites using 
research and development funding with a first launch expected in fiscal 
year 2007. The program also recently underwent a major restructuring to 
reduce program risk.



SBIRS High Program:

Technology Maturity:

The SBIRS High program's three critical technologies--the infrared 
sensor, thermal management, and the on-board processor--are now mature. 
Program officials indicated that the hardware was built and tested in a 
thermal vacuum chamber under expected flight conditions. When the 
program began product development in 1996, none of its critical 
technologies were mature, according to best practice standards.

Design Maturity:

Program officials do not know how many total drawings are expected for 
SBIRS High, and thus do not track the number of releasable drawings. As 
a result, we could not assess design stability relative to best 
practices. Program officials did state that the current number of 
releasable drawings is 2,342, about twice the number at the time of the 
design review. This means that at most, no more than half of the 
drawings could have been releasable at the design review. Design 
stability has been an issue for this program. During development, the 
satellite was redesigned to maintain key performance parameters. 
Redesign efforts resulted in a 6-month slip to the spacecraft and 
increased the requirement for ground processing.

On the other hand, the two sensors that will be aboard non-SBIRS 
satellites are considered stable with subsystem qualification nearing 
completion, and integration and delivery of the flight payload are 
expected within the year. The first of these sensors is scheduled for 
delivery in May 2003--three months behind schedule. This delay is 
attributed to problems with radio waves emitted by the sensor's 
electronics that interfere with the host satellite. Despite these 
integration difficulties, data shows that the sensors will perform much 
better than expected.

Production Maturity:

We could not assess the SBIRS High production maturity relative to best 
practice standards because the contractor does not use statistical 
process control to ensure that production processes are stable.

Other Program Issues:

The total unit cost of the SBIRS High program rose more than 25 percent 
in 1 year. The notification to Congress of the Nunn-McCurdy breach (see 
10 U.S.C. 2433) occurred on December 31, 2001, requiring a review by 
the Secretary of Defense and a report to Congress. As a result, DOD 
certified to Congress in May 2002 that the SBIRS High program is 
essential for national security, there are no alternatives that provide 
equal or greater capability at less cost, cost estimates are 
reasonable, and the management structure is in place to continue to 
keep costs under control.

Program Office Comments:

Program officials generally concurred with our assessment and provided 
technical comments, which we have incorporated where appropriate. 
Program officials added that the fiscal year 2004 budget fully funds 
their restructured program and directs the satellite procurement to 
begin in fiscal year 2006.

Theater High Altitude Area Defense (THAAD).

THAAD is an element of the terminal defense segment of the Ballistic 
Missile Defense System. Its mission is to defend against short and 
medium range ballistic missiles. THAAD's ability to intercept inside 
and outside the atmosphere makes effective countermeasures more 
difficult and allows multiple intercept opportunities. The system 
includes missiles, launchers, radars, command and control/battle 
management (C2/BM), and THAAD support equipment.


[See PDF for image]

[End of figure]

Theater 
High Altitude Area Defense (THAAD): Most of THAAD's critical 
technologies have demonstrated acceptable levels of maturity and the 
program appears close to meeting the best practice standard for a 
stable design. The program's launcher and radar have essentially 
attained technological maturity, but the missile and the command and 
control/battle management components are somewhat less mature. This 
level of maturity follows many years of difficult development. It 
appears that the THAAD program has mostly recovered from initial 
problems driven by an early fielding requirement and poor quality 
control. The current THAAD acquisition strategy shows a much greater 
emphasis on attaining knowledge. The program expects to reach 
technological maturity and design stability by February 2004.

[See PDF for image]

[End of figure]

THAAD Program:

Technology Maturity:

THAAD program officials assessed 47 technologies in four major 
elements--command and control/battle management; missile interceptor; 
launcher; and radar. Of the four elements, the radar is currently the 
most mature, followed by the launcher, command and control/battle 
management, and the missile. The program has made progress on 
technology maturity since it began development despite early failures 
in intercept attempts. Early flight-test failures were caused by a 
combination of the compressed test schedule and quality control 
problems. The program was restructured twice, before the first 
successful intercept occurred in 1999. The research and development 
cost grew from $4.4 to $10.5 billion prior to the program's transfer to 
the Missile Defense Agency, partially as a result of these problems.

The current program strategy appears geared to obtaining the necessary 
knowledge by providing more time for maturing the technology before 
flight tests and placing greater emphasis on risk reduction efforts. 
This strategy includes utilizing technology readiness levels to assess 
technological maturity.

Design Maturity:

The program has released about 82 percent of total drawings. Program 
officials expect to release about 91 percent of the drawings by the 
system-level design review in February 2004. The Missile Defense Agency 
is redesigning the missile to be more reliable and testable, with 
significantly fewer parts than the previous version. The first flight 
test of the redesigned missile is not scheduled to occur until at least 
6 months after the system design review. Depending on the outcome, 
flight tests could require more design changes and delay achieving 
design stability.

Other Program Issues:

THAAD was recently transferred from the Army to the Missile Defense 
Agency, which has restructured and modified the contract to a block 
upgrade approach. Therefore, limited information is currently available 
on the total projected costs of this program.

In response to the prior program setbacks, the THAAD project office is 
accelerating some risk reduction activities, and it has planned a 
series of flight tests that (1) tests the missile in a less stressing 
intercept environment outside the atmosphere, (2) tests the missile in 
the more stressing flight environment inside the atmosphere, and (3) 
prepares the system for initial operational test and evaluation.

Program Office Comments:

In commenting on a draft of this assessment, program officials stated 
that, to ensure the highest probability of success in flight-testing, a 
substantial amount of ground testing is being conducted in the next 
year and a half. This testing includes exhaustive engineering and 
qualification level testing on all flight components. Program officials 
further stated that the extensive design, fabrication, and test 
preparation activity has been very successful to date, and the program 
remains healthy with a slightly ahead-of-schedule and under-cost 
status.

Tactical Tomahawk Missile.

The Navy's Tactical Tomahawk (block IV) is a major upgrade to the 
Tomahawk Land Attack Missile (block III). The Tactical Tomahawk missile 
will provide ships and submarines with enhanced capability to attack 
targets on land. New features include improved antijamming global 
positioning system, in-flight retargeting, and ability to transmit 
battle damage imagery. The system includes the missile, the weapon 
control system, and the mission planning system. We assessed only the 
missile.

[See PDF for image]

[End of figure]


Tactical Tomahawk Missile: 
The Tactical Tomahawk missile entered low-rate production without 
ensuring that production processes were in control. Program officials 
indicated that they plan to collect production process control data 
over the next year, prior to award of the full-rate production contract 
in fiscal year 2004. At that time, program officials expect over 80 
percent of the low-rate production missiles to be in various stages of 
assembly. The technology and design have reached acceptable levels of 
maturity. While engineering drawings have improved to 96 percent, the 
program only had about half of its drawings released at the design 
review. Program plans call for a full-rate production decision in May 
2004.

[See PDF for image]

[End of figure]

Tactical Tomahawk Program:

Technology Maturity:

We did not assess the technology readiness levels of the key 
technologies for the Tactical Tomahawk missile. At the time of our 
review, critical technologies were mature. According to the program 
office, the critical technologies for the key subsystems--antijam 
global positioning system, digital scene matching area correlator, and 
cruise engine--were modified derivatives from other programs or 
upgrades to existing Tomahawk subsystems and consequently already 
mature. To date, subsystem and the majority of missile-level 
qualification testing has been completed successfully.

Design Maturity:

The basic design of the Tactical Tomahawk missile is essentially 
complete. The critical design review occurred in June 2000. At that 
time, approximately 47 percent of the drawings had been released to 
manufacturing. In October 2002, at the first low-rate initial 
production award, 723 of 750 total drawings, or about 96 percent, had 
been released.

Production Maturity:

Officials plan to collect statistical control data at the start of the 
manufacturing process but do not expect to have meaningful statistical 
data until sometime in 2004. Manufacture of the Tactical Tomahawk 
missile is scheduled to begin at the subcontractor's facility in 2003 
and missile assembly in 2004. Although two low-rate production 
contracts have been awarded, program officials stated that data 
regarding manufacturing process controls currently is very limited. 
Program officials told us that it is too soon to know what percentage 
of critical manufacturing processes will be under statistical control 
when the full-rate production contract is awarded in mid-2004, but that 
they plan to start collecting production process control data over the 
next year.

Other Program Issues:

The Tactical Tomahawk missile successfully completed its first 
developmental flight test in August 2002, and the first low-rate 
production contract for 25 units was awarded in October 2002. A second 
and final low-rate production contract was awarded in mid-January 2003 
for 167 units. Program officials stated that total quantities have 
increased to 2,396.

Program Office Comments:

In commenting on a draft of this assessment, program officials stated 
that two development test flights, conducted prior to low-rate 
production awards, demonstrated that the Tactical Tomahawk missile 
design met or exceeded technical and key performance parameters. They 
also noted that, due to the stability of the design and successful 
completion of all component and flight qualification testing, the 
Navy's operational test agency issued a favorable operational 
assessment, stating that the Tactical Tomahawk missile is potentially 
suitable and potentially operationally effective.

V-22 Osprey.

The V-22 Osprey is a tilt-rotor, vertical takeoff and landing aircraft 
designed to meet the amphibious/vertical assault needs of the Marine 
Corps, long-range missions of Special Operations forces, and combat 
search and rescue needs of the Navy. The V-22 will replace the CH-46E 
and the CH-53A/D in the Marine Corps; the H-53 and H-60 will augment 
the C-130 in the Air Force and the Special Operations Command; and 
supplement the H-60 in the Navy. We assessed the block A version.

[See PDF for image]

[End of figure]

V-22 Osprey: The V-22 program plans to 
enter full-rate production without ensuring that the manufacturing 
processes are mature. Redesign of the aircraft's hydraulic and electric 
system, and software changes have been made to address safety, 
reliability, maintainability, and logistics supportability. These 
design changes and others are undergoing developmental testing to ready 
the aircraft for an operational test and evaluation test period in late 
2004 through early 2005 to determine if the V-22 is operationally 
suitable and effective. The design changes, however, have not been 
incorporated into the low-rate production aircraft currently being 
produced. The value of contract modifications needed to address the 
cost of these design changes is not yet known. Also, parts shortages 
and quality issues are currently effecting low-rate production costs. 
Some key performance requirements have been eliminated.

[See PDF for image]

[End of figure]

V-22 Program:

Technology Maturity:

Although we did not specifically assess the V-22's technology maturity, 
the program office believes key technologies to be mature. An 
operational test report, dated November 2000, determined that the V-22 
was not operationally suitable because of poor reliability, 
maintainability, availability, human factors, and interoperability 
problems. Immature technology, in part, contributed to this assessment.

Design Maturity:

As a result of a crash in December 2000, the V-22 has undergone several 
design changes. Specifically, the aircraft's hydraulic and electrical 
lines were redesigned to improve safety, reliability, maintainability, 
and logistics supportability. The V-22 flight control system software 
was also redesigned. The program office estimates that redesign of the 
V-22 resulted in 1,755 additional drawings, increasing the total number 
of drawings to 7,490. To date, all of these drawings are complete.

The success of these design changes will be determined as the aircraft 
undergoes additional developmental testing through 2005. Testing will 
address many issues, including high rate of descent, handling 
qualities, austere environment operations, and ship operations. The 
operational assessment of these characteristics will not occur until 
late 2004 or early 2005. Recent decisions to defer some V-22 
operational requirements previously considered critical until later 
blocks will void the need for some design changes in the block A.

Production Maturity:

Neither V-22 contractor collects statistical process control data on 
its critical manufacturing processes. A recent program management 
assessment rated V-22 production as cautionary. Part shortages and 
quality problems caused inefficiencies in shop and assembly operations, 
as well as scrap, rework, repair, and schedule delays.

Other Program Issues:

Low-rate production of the V-22 continues. V-22s are being fabricated 
and partially assembled, but not delivered until the first set of 
upgrades--referred to as block A--needed to bring the V-22 to a safe 
operational and suitable configuration are approved and incorporated 
into production aircraft. Delivery of block A aircraft is expected to 
start in the fourth quarter of fiscal year 2003. However, the cost of 
contract modifications needed to reconfigure already produced aircraft 
and aircraft still on the assembly line to the bock A configuration has 
not been negotiated.

Program Office Comments:

In commenting on draft of this assessment program officials stated that 
they have restructured the program to gather more technical knowledge 
through a more rigorous "event-driven" flight test program. Program 
officials strongly disagreed that the program plans to enter full-rate 
production without ensuring that manufacturing process are mature. V-
22s are currently being manufactured at a minimum sustaining rate (11 
aircraft per year). A May 20TH defense acquisition board review is 
scheduled to consider increasing this rate. Manufacturing processes and 
tooling are in place and being continually analyzed and improved. Both 
companies utilize statistical process control techniques and numerous 
metrics to assess program performance. They do not use the process 
capability index, the only metric that GAO uses as a basis for their 
assessment. Program officials are also undertaking an affordability 
review to reduce the aircraft unit cost to $58 million by 2010. High 
unit costs are driven by the current low production quantities and will 
remain the norm until production quantities increase.

GAO Comments:

Our prior work has shown that leading commercial firms rely on 
statistical control data, specifically, the process capability index, 
as the best indicator of production readiness. The V-22 program entered 
low-rate production without this information and has experienced 
production quality problems.

Wideband Gapfiller Satellite (WGS) Communications System.

The Wideband Gapfiller Satellite system is a joint Air Force and Army 
program intended to provide communications to the U.S. warfighters, 
allies, and Coalition Partners during all levels of conflict short of 
nuclear war. It is the next generation wideband component in the DOD's 
future Military Satellite Communications architecture.

[See PDF for image]

[End of figure]

[See PDF for image]

[End of figure]

WGS Program:

The WGS program's two critical technologies--the digital channelizer 
and the phased array antenna--are mature. Most of these technologies 
are commercial derivatives. For this reason, many of the satellite 
technologies selected were already at high levels of maturity. In fact, 
the program is leveraging commercial technology and practices by 
modifying commercial satellites to better support unique military 
requirements.

Design Maturity:

The WGS design is essentially complete, as the program has released 
approximately 95 percent of the expected drawings.

Production Maturity:

The contractor has six of its eight key manufacturing processes under 
control, according to the best practice standards. Program officials 
indicated that they are bringing the remaining processes under 
statistical control.

Program Office Comments:

In commenting on a draft of this assessment, program officials stated 
that while critical technology areas being applied to WGS are fairly 
mature, the manufacturing of the systems using these technologies is 
relatively new for the contractor. Risk of production problems was to 
be reduced due to other commercial satellite system developments and 
production ahead of WGS in the development and production schedule. 
However, due to the drastic loss of commercial satellite orders, only 
one commercial satellite with similar technologies as WGS is now 
leading WGS in the manufacturing schedule. Recently identified problems 
found on the "leader" program will impact the WGS manufacturing 
schedule, and a first launch schedule delay of 4 to 6 months can be 
expected due to time needed to resolve the "leader" program 
manufacturing problems. Satellites four and five have been directed by 
DOD to be launched in fiscal year 2009 and fiscal year 2010, 
respectively. These dates are outside the allowable dates of the WGS 
contract option clauses and will require renegotiation to finalize 
their cost. The cost is expected to increase to compensate for loss of 
learning curve from over a 3-year break in production, parts 
obsolescence, and inflation.

[End of section]

Appendix II: Methodology:

In conducting our work, we evaluated performance and risk data from 
each of the programs included in this report. We summarized our 
assessments of each individual program in two components--a system 
profile and a product knowledge assessment. We did not validate or 
verify the data provided by DOD. However, we took several steps to 
address data quality. Specifically, we reviewed the data and performed 
various quality checks, which revealed some discrepancies in the data. 
We discussed these discrepancies with program officials and adjusted 
the data accordingly.

System Profile Assessment:

In the past 3 years, DOD revised its policies governing weapon system 
acquisitions and changed the terminology used for major acquisition 
events. In order to make DOD's acquisition terminology more consistent 
across the 26 program assessments, we standardized the terminology for 
key program events. In the individual program assessments, program 
start refers to the initiation of a program; DOD usually refers to 
program start as milestone I or milestone A, which begins the concept 
and technology development phase. Similarly, development start refers 
to the commitment to product development that coincides with either 
milestone II or milestone B, which begins DOD's system development and 
demonstration phase. The production decision generally refers to the 
decision to enter the production and deployment phase, typically with 
low-rate initial production. Initial capability refers to the initial 
operational capability, sometimes also called first unit equipped or 
required asset availability.

The funding request information presented refers to the President's 
fiscal year 2004 budget request, except where noted. The program cost 
comparisons are the latest estimates provided by the individual 
programs. The quantities listed refer to total quantities, including 
both procurement and development quantities.

To assess the cost, schedule, and quantity changes of each program, we 
reviewed DOD's selected acquisition reports or obtained data directly 
from the program offices. In general, we compared the latest available 
selected acquisition report information with a baseline for each 
program. For systems that have started product development--those that 
are beyond milestone II or B--we compared the latest available Selected 
Acquisition Report to the development estimate from the first Selected 
Acquisition Report issued after the program was approved to enter 
development. For systems that have not yet started product development, 
we compared the latest available data to the planning estimate issued 
after milestone I or A. For systems not included in selected 
acquisition reports, we attempted to obtain comparable baseline and 
current data from the individual program offices.

All cost information is presented in base year 2003 dollars, unless 
otherwise noted, using Office of the Secretary of Defense approved 
deflators to eliminate the effects of inflation. We have depicted only 
the programs' main elements of acquisition cost--research and 
development, and procurement, however the total program costs displayed 
also include military construction and acquisition operation and 
maintenance costs. Because of rounding and these additional costs, in 
some situations the total cost may not match the exact sum of the 
research and development and procurement costs. The program unit costs 
are calculated by dividing the total program cost by the total 
quantities planned. These costs are often referred to as program 
acquisition unit costs.

The schedule assessment is based on acquisition cycle time, defined as 
the number of months between the program start, usually milestone I or 
A, and the achievement of initial operational capability or an 
equivalent fielding date.

The intent of these comparisons is to provide an aggregate or overall 
picture of a program's history. These assessments represent the sum 
total of the federal government's actions on a program, not just those 
of the program manager and the contractor. DOD does a number of 
detailed analyses of changes that attempt to link specific changes with 
triggering events or causes. Our analysis does not attempt to make such 
detailed distinctions.

Product Knowledge Assessment:

To assess the product development knowledge of each program at key 
points in development, we submitted a data collection instrument to 
each program office. The results are graphically depicted in each two-
page assessment. The methodology used to generate each graph is 
discussed at the beginning of appendix I. We also reviewed pertinent 
program documentation, such as the operational requirements document, 
the acquisition program baseline, test reports, and major program 
reviews.

To assess technology maturity, we asked program officials to apply a 
tool, referred to as technology readiness levels, for our analysis. The 
National Aeronautics and Space Administration originally developed 
technology readiness levels, and the Army and Air Force Science and 
Technology research organizations use them to determine when 
technologies are ready to be handed off from science and technology 
managers to product developers. Technology readiness levels are 
measured on a scale of one to nine, beginning with paper studies of a 
technology's feasibility and culminating with a technology fully 
integrated into a completed product. Our best practices work has shown 
that a technology readiness level of 7--demonstration of a technology 
in an operational environment--is the level of technology maturity that 
constitutes a low risk for starting a product development program.

In most cases, we did not validate the program offices' selection of 
critical technologies or the determination of the demonstrated level of 
maturity. We sought to clarify the technology readiness levels in those 
cases where information existed that raised concerns. If we were to 
conduct a detailed review, we might adjust the critical technologies 
assessed, the readiness level demonstrated or both. It was not always 
possible to reconstruct the technological maturity of a weapon system 
at key decision points after the passage of many years.

To assess design maturity, we asked program officials to provide the 
percentage of engineering drawings completed or projected for 
completion by the design review, the production decision, and as of our 
current assessment. Completed engineering drawings were defined as the 
number of drawings released or deemed releasable to manufacturing that 
can be considered the "build to" drawings.

To assess production maturity, we asked program officials to identify 
the number of critical manufacturing processes and, where available, to 
quantify the extent of statistical control achieved for those 
processes. We used a standard called the Process Capability Index, 
which is a process performance measurement that quantifies how closely 
a process is running to its specification limits.[Footnote 2] The index 
can be translated into an expected product defect rate and we have 
previously found it to be a best practice. We sought other data, such 
as scrap and rework trends in those cases where quantifiable 
statistical control data was unavailable.

Although the knowledge points provide excellent indicators of potential 
risks, by themselves, they do not cover all elements of risk that a 
program encounters during development, such as funding instability. Our 
detailed reviews on individual systems normally provide for a fuller 
treatment of risk elements.

[End of section]

Appendix III: GAO Contact and Acknowledgments:

GAO Contact:

Paul Francis (202) 512-2811:

Acknowledgments:

David B. Best, James A. Elgas, and James L. Morrison made key 
contributions to this report and were largely supported by GAO's 
Acquisition and Sourcing Management staff. The following staff were 
responsible for individual programs.

Staff: Michael Hazard; System: Advanced Amphibious Assault Vehicle 
(AAAV).

Staff: Steven Martinez; System: Advanced Extremely High Frequency 
(AEHF) Communications Satellite.

Staff: David Hubble; System: Advanced Wideband Satellite/
Transformational Communications Satellite (AWS/TSAT).

Staff: Gaines Hensley; System: AN/APG-79 Active Electronically Scanned 
Array (AESA) Radar.

Staff: Marvin Bonner; System: AIM-9X Short-Range Air-to-Air Missile.

Staff: Thomas Gordon; System: Airborne Laser (ABL).

Staff: Dana Solomon/Carrie Wilson/Danny Owens; System: Advanced Threat 
Infrared Countermeasures/Common Missile Warning System (ATIRCM/CMWS).

Staff: Johanna Ayers; System: Cooperative Engagement Capability (CEC).

Staff: Leon Gill; System: CH-47F Improved Cargo Helicopter.

Staff: Wendy Smythe; System: RAH-66 Comanche.

Staff: Marti Dey/Ron Schwenn; System: EX-171 Extended Range Guided 
Munition (ERGM).

Staff: Larry Gaston; System: Excalibur Artillery Round.

Staff: Cheryl Andrew; System: F/A-18E/F Super Hornet.

Staff: Donald Springman; System: F-22 Raptor.

Staff: Beverly Breen/Lynn Lavalle; System: Joint Air-to-Surface 
Standoff Missile (JASSM).

Staff: Danny Owens; System: Joint Common Missile.

Staff: Rae Ann Sapp/Art Cobb; System: Joint Primary Aircraft Training 
System (JPATS).

Staff: Brian Mullins/Ron Schwenn; System: F-35 Joint Strike Fighter 
(JSF).

Staff: Carol Mebane; System: Joint Standoff Weapon (JSOW).

Staff: Bruce Thomas; System: National Polar-orbiting Operational 
Environmental Satellite System (NPOESS).

Staff: Matthew Lea; System: Patriot Advanced Capability 3 Program 
(PAC-3).

Staff: Tana Davis; System: Tactical Tomahawk Missile.

Staff: William Lipscomb/Tana Davis; System: Theater High Altitude Area 
Defense (THAAD).

Staff: Maricela Cherveny/Nancy Rothlisberger; System: Space Based 
Infrared Satellite-High (SBIRS-High).

Staff: Jerry Clark; System: V-22 Osprey.

Staff: Art Gallegos/Tony Beckham; System: Wideband Gapfiller Satellite 
Communication System.



(120185):

:

FOOTNOTES

[1] U.S. General Accounting Office, Best Practices: Capturing Design 
and Manufacturing Knowledge Early Improves Acquisition Outcomes, 
GAO-02-701 (Washington, D.C.: July 15, 2002).

[2] Process Capability Index provides assurance that production 
processes are under 
100 percent statistical control. A high index value equates to fewer 
defects per part based on statistical process control data. The general 
rule of thumb used by the manufacturing industry states that if the 
index value for a process is less than 1.33, then the process is not 
capable of producing a part with acceptable consistency.

GAO's Mission:

The General Accounting Office, the investigative arm of Congress, 
exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony:

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics.

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to daily E-mail alert for newly 
released products" under the GAO Reports heading.

Order by Mail or Phone:

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to:

U.S. General Accounting Office

441 G Street NW,

Room LM Washington,

D.C. 20548:

To order by Phone: 	

	Voice: (202) 512-6000:

	TDD: (202) 512-2537:

	Fax: (202) 512-6061:

To Report Fraud, Waste, and Abuse in Federal Programs:

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470:

Public Affairs:

Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.

General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.

20548: