Weapon Systems Annual Assessment: Limited Use of Knowledge-Based Practices Continues to Undercut DOD's Investments
Fast Facts
In our 17th annual assessment, we reviewed DOD's 82 largest complex programs (worth $1.69 trillion). Our review comes during a time of significant change at DOD. DOD has transferred oversight responsibilities for many programs to the military services that manage them. At the department-level, it is now focusing on developing weapons capabilities faster. We will continue to monitor these changes in oversight.
Meanwhile, we found that costs continue to increase for many of these programs—even those that started after major acquisition reforms were adopted in 2010. This is troubling because these reforms were supposed to limit cost growth.
Arleigh Burke-class Destroyer
A military ship in the water.
Highlights
This special report is GAO’s 17th annual assessment of the Department of Defense’s (DOD) $1.69 trillion portfolio of 82 major weapon systems acquisition programs. This report (1) examines changes in the portfolio since last year, (2) evaluates DOD’s actions in areas related to recent, selected acquisition reforms, and (3) offers a quick look at cost and schedule performance of 51 individual weapon programs based on DOD documentation and questionnaire responses from their respective program offices.
This report provides observations on:
1. cost and schedule performance and contract awards for DOD’s 2018 portfolio of 82 programs that provide acquisition reports to Congress and
2. knowledge that 51 selected programs attained at key points in the acquisition process.
Further, GAO presents individual assessments of 51 programs.
For years, GAO has highlighted the importance of applying knowledge-based acquisition practices as a way to improve DOD’s program outcomes. When programs enter development with insufficient knowledge, negative effects often cascade throughout the acquisition cycle. The figure below shows how three acquisition phases—technology development, system development, and production—align with three key points for demonstrating knowledge.
Department of Defense (DOD) Acquisition Process
DOD’s 2018 portfolio of major weapon programs has grown in cost by $8 billion, but contains four fewer systems than last year. GAO’s analysis suggests that one of the primary drivers of this cost growth is that, since 2012, the average age of programs has increased—indicative of DOD decisions to introduce new capabilities through additions to existing programs rather than by starting new programs. On average, programs in the current portfolio are about 4 months older than last year and nearly 3 years older than in 2012. (See figure.)
DOD's Portfolio Increased in Cost and Average Program Age
Portfolio-wide cost growth has occurred in an environment where awards are often made without full and open competition. Specifically, GAO found that DOD did not compete 67 percent of 183 major contracts currently reported for its 82 major programs. GAO also observed that DOD awarded 47 percent of these 183 contracts to five corporations and entities connected with them. (See figure.)
DOD Programs Competed One-Third of Currently Reported Major Contracts with Nearly Half of Awards Concentrated within Five Companies
This year’s assessment comes at a time of significant change to how DOD manages and oversees its major weapons programs. At the direction of Congress, DOD restructured the Office of the Secretary of Defense to focus more on capability development and less on program oversight, which now resides primarily with the three military departments. Amid these changes, this year GAO found that cost performance has slipped. Further, unlike prior years, programs initiated after major acquisition reforms were implemented in 2010 are now showing cost growth.
As we have reported in previous assessments, DOD programs continue not to fully implement knowledge-based acquisition practices. GAO observed that most of the 45 current programs proceeded into system development, through critical design reviews, and into production without completing the key knowledge-based practices associated with each of these three points. (See table.)
DOD Major Defense Acquisition Programs Continue Not to Fully Implement Key Knowledge-Based Acquisition Practices
| Practices associated with the three key knowledge points (KP) | Programs that completed the KP before this assessment period | Programs that completed the KP during this assessment period |
| KP 1 practices | 38 programs | Four programs |
| Demonstrate all critical technologies are very close to final form, fit, and function within a relevant environment | ◐ | ◐ |
| Demonstrate all critical technologies are in form, fit, and, function within a realistic environment | ○ | ○ |
| Completed preliminary design review before system development start | ◐ | ○ |
| KP 2 practices | 33 programs | Two programs |
| Release at least 90 percent of design drawings to manufacturing | ○ | ● |
| Test a system-level integrated prototype | ○ | ○ |
| KP 3 practices | 15 programs | Three programs |
| Demonstrate critical manufacturing processes are in statistical control | ○ | ○ |
| Demonstrate critical processes on a pilot production line | ◐ | ● |
| Test a production-representative prototype in its intended environment | ○ | ● |
Programs completing each best practice: ● 75 - 100 percent; ◐ 50 - 74 percent; ◯ 0 - 49 percent
Source: GAO analysis of Department of Defense data. │GAO-19-336SP
This lack of knowledge and the effects it can have throughout a program’s acquisition life cycle can increase the risk of undesirable cost and schedule outcomes. In 2018, GAO conducted an exploratory statistical analysis of 15 programs in production to evaluate their completion of the eight key knowledge-based acquisition practices. In 2019, GAO expanded this analysis to include 17 programs. Over the past two years, GAO found that the major DOD acquisition programs that completed one or more of three specific practices had significantly lower cost and schedule growth than those that did not. These three practices were (1) demonstration that all critical technologies were very close to final form, fit, and function, within a relevant environment, before starting development; (2) completion of a preliminary design review prior to starting development; and (3) release of at least 90 percent of design drawings by critical design review.
For more information, contact Shelby S. Oakley at (202) 512-4841 or oakleys@gao.gov.