Leading Practices: Agency Acquisition Policies Could Better Implement Key Product Development Principles
Fast Facts
Together, the Departments of Defense and Homeland Security and NASA invest hundreds of billions of dollars each year to develop and deliver a wide range of systems, from stealth jets to lunar rovers.
Leading companies rely on certain principles to ensure product development success. But we found that these federal agencies aren't generally required to meet such principles. For example, DOD's policies don't require acquisition staff to consider dropping less urgent requirements to stay on schedule—a key development principle.
We recommended that the agencies create or update guidance to reflect key principles of product development.
Highlights
Why This Matters
Each year, the Departments of Defense (DOD) and Homeland Security (DHS) and the National Aeronautics and Space Administration (NASA) together invest hundreds of billions of dollars to buy stealth jets, cutters and ships, and lunar rovers, among other things, all with complex software. However, GAO’s annual reviews of these agencies’ major acquisitions find they often take longer and spend more money than planned to deliver capabilities to users.
Key Takeaways
Leading companies take a disciplined approach to develop innovative products that satisfy their customers’ needs, and to deliver them to market on time and within planned costs. The 13 leading companies GAO interviewed perform similar activities when developing new products, such as iterative design in hardware and software development. These activities in the development process align with the four key principles that help project teams deliver innovative products to market quickly and efficiently (see figure). GAO found that the department-wide acquisition policies of DOD, DHS, and NASA implement some key product development principles. But, they have yet to fully implement others. This gap limits agencies from ensuring a consistent approach to developing and delivering products with speed and efficiency.
Leading Companies Use Four Key Principles for Product Development
For example, leading companies focus on designing a minimum marketable product—one with the minimum capabilities needed for customers to recognize value. Leading companies also prioritize a project’s schedule: they release the features most critical to the customer and will off-ramp non-critical product features—an industry term for removing them from the current release—as necessary, in order to maintain schedule. Leading companies have mechanisms to solicit and implement feedback from customers early and often throughout development to ensure the product is relevant to customer needs, among other things.
Primary DOD, DHS, and NASA acquisition policies incorporate many aspects of the four key principles, to varying degrees. However, agencies miss opportunities for positive outcomes by not addressing some sub-principles in their policies.
- DOD’s policies do not require all programs to consider off-ramping non-critical capabilities in order to achieve schedule, hindering programs’ best chance of maintaining time frames.
- DHS’s policies do not require all programs to utilize modern design tools during hardware and software development, limiting consistent opportunities for programs to successfully improve revisions to the design.
- NASA’s policies do not include mechanisms for programs to obtain and utilize product feedback from stakeholders or end users—such as astronauts using spacecraft or the science community benefiting from NASA projects—in order to identify challenges or new features to include in subsequent projects.
GAO previously found that other factors beyond policies can affect agency outcomes, including structural differences between government and private industry. However, GAO’s prior work also demonstrates that key principles from private industry can be thoughtfully applied to government acquisition to improve outcomes, even with the different cultures and incentives.
How GAO Did This Study
This report examines principles that guide leading companies’ product development efforts and the extent to which primary, department-wide DOD, DHS, and NASA acquisition policies reflect the companies’ key principles and result in similar outcomes. GAO identified the 13 leading product development companies based on rankings in well-recognized lists; interviewed company representatives; analyzed department-wide acquisition policies from DOD, DHS, and NASA; and interviewed agency officials. The report is the first product in a planned body of work. In future work, GAO will explore how government agencies can apply some of the key principles outlined in this report.
Recommendations
GAO is making nine recommendations to DOD, DHS, and NASA to update acquisition policies to fully implement key principles of product development. All three agencies concurred with our recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Defense | The Secretary of Defense should ensure that the Under Secretary of Defense for Acquisition and Sustainment update DOD acquisition policies to fully implement the following principle throughout development: attaining a sound business case (Recommendation 1). |
DOD concurred with our recommendation. In August 2022, the Under Secretary of Defense for Acquisition and Sustainment noted it would give full consideration to the further application of key product development principles when it next formally updates its overarching acquisition policy and the other individual pathways and functional acquisition policies. In July 2024, officials from the Under Secretary's office indicated that they have drafted revisions to certain policies, including DOD Instruction 5000.85 Major Capability Acquisition and DOD Instruction 5000.80 Middle Tier of Acquisition. As of August 2024, the officials said they expect to issue the revised policies by January 2025. They stated that other relevant policy updates are also in progress and will be issued subsequently, though the officials did not provide a specific timeline. DOD has yet to provide GAO with draft copies of the revised policies to determine the extent to which the updates address this recommendation. We will continue to track DOD's progress in this area.
|
| Department of Defense | The Secretary of Defense should ensure that the Under Secretary of Defense for Acquisition and Sustainment update DOD acquisition policies to fully implement the following principle throughout development: applying iterative design approaches (Recommendation 2). |
DOD concurred with our recommendation. In August 2022, the Under Secretary of Defense for Acquisition and Sustainment noted it would give full consideration to the further application of key product development principles when it next formally updates its overarching acquisition policy and the other individual pathways and functional acquisition policies. In July 2024, officials from the Under Secretary's office indicated that they have drafted revisions to certain policies, including DOD Instruction 5000.85 Major Capability Acquisition and DOD Instruction 5000.80 Middle Tier of Acquisition. As of August 2024, the officials said they expect to issue the revised policies by January 2025. They stated that other relevant policy updates are also in progress and will be issued subsequently, though the officials did not provide a specific timeline. DOD has yet to provide GAO with draft copies of the revised policies to determine the extent to which the updates address this recommendation. We will continue to track DOD's progress in this area.
|
| Department of Defense | The Secretary of Defense should ensure that the Under Secretary of Defense for Acquisition and Sustainment update DOD acquisition policies to fully implement the following principle throughout development: off-ramping capabilities when needed to maintain schedule (Recommendation 3). |
DOD concurred with our recommendation. In August 2022, the Under Secretary of Defense for Acquisition and Sustainment noted it would give full consideration to the further application of key product development principles when it next formally updates its overarching acquisition policy and the other individual pathways and functional acquisition policies. In July 2024, officials from the Under Secretary's office indicated that they have drafted revisions to certain policies, including DOD Instruction 5000.85 Major Capability Acquisition and DOD Instruction 5000.80 Middle Tier of Acquisition. As of August 2024, the officials said they expect to issue the revised policies by January 2025. They stated that other relevant policy updates are also in progress and will be issued subsequently, though the officials did not provide a specific timeline. DOD has yet to provide GAO with draft copies of the revised policies to determine the extent to which the updates address this recommendation. We will continue to track DOD's progress in this area.
|
| Department of Defense | The Secretary of Defense should ensure that the Under Secretary of Defense for Acquisition and Sustainment update DOD acquisition policies to fully implement the following principle throughout development: incorporating feedback from users of initial capabilities (Recommendation 4). |
DOD concurred with our recommendation. In August 2022, the Under Secretary of Defense for Acquisition and Sustainment noted it would give full consideration to the further application of key product development principles when it next formally updates its overarching acquisition policy and the other individual pathways and functional acquisition policies. In July 2024, officials from the Under Secretary's office indicated that they have drafted revisions to certain policies, including DOD Instruction 5000.85 Major Capability Acquisition and DOD Instruction 5000.80 Middle Tier of Acquisition. As of August 2024, the officials said they expect to issue the revised policies by January 2025. They stated that other relevant policy updates are also in progress and will be issued subsequently, though the officials did not provide a specific timeline. DOD has yet to provide GAO with draft copies of the revised policies to determine the extent to which the updates address this recommendation. We will continue to track DOD's progress in this area.
|
| Department of Homeland Security | The Secretary of Homeland Security should ensure that the DHS Undersecretary for Management update DHS acquisition policies to fully implement the following principle throughout development: attaining a sound business case (Recommendation 5). |
DHS concurred with our recommendation. We found that DHS policies did not implement the leading sub-principle of terminating product development promptly if the product no longer has a sound business case. In January 2023, DHS updated Instruction 102-01-001 Acquisition Management to include guidance for communicating and documenting the decision to cancel or terminate an acquisition program because of lack of funding, extreme schedule breaches, changes in mission need, or inability to meet the performance requirements, for example. The additional guidance clarifies when to consider a program for termination and should help DHS avoid continuing to invest in acquisition programs that no longer have a sound business case.
|
| Department of Homeland Security | The Secretary of Homeland Security should ensure that the DHS Undersecretary for Management update DHS acquisition policies to fully implement the following principle throughout development: applying iterative design approaches (Recommendation 6). |
DHS concurred with our recommendation. This recommendation stemmed from our report findings that DHS policies did not fully implement leading sub-principles to (1) use modern, digital design tools capable of integrating development of hardware and software and (2) use iterative design and testing processes to generate a minimum viable product that can be continuously updated and improved after delivery. In January 2023, DHS updated Instruction 102-01-001 Acquisition Management to include guidance for program managers to use modern engineering design tools. In April 2024, DHS incorporated changes to the instruction to encourage program managers to implement iterative development methodology, as appropriate, and to structure acquisition programs to provide iterative capability that delivers a minimum viable product to the user as soon as is practicable. In addition, DHS updated its Lexicon Guidebook - Terms and Definitions 102-01-003-01 to define iterative development consistent with terminology provided in our report. The updated guidance should help DHS program managers execute their acquisition programs to reduce the need for costly design changes later in the life cycle and deliver the most critical capabilities users need sooner.
|
| Department of Homeland Security | The Secretary of Homeland Security should ensure that the DHS Undersecretary for Management update DHS acquisition policies to fully implement the following principle throughout development: off-ramping capabilities when needed to maintain schedule. (Recommendation 7). |
DHS concurred with our recommendation. In January 2023, DHS updated Instruction 102-01-001 Acquisition Management to provide for identifying trade-offs to maintain affordability and considering trade-offs to implement acquisition programs within the approved cost, schedule, and performance parameters of the acquisition program baseline. This additional guidance should help DHS ensure acquisition programs provide users with needed capabilities within cost and schedule goals.
|
| National Aeronautics and Space Administration | The NASA Administrator should ensure that the NASA Office of the Chief Engineer update NASA acquisition policies to fully implement the following principle throughout development: applying iterative design approaches (Recommendation 8). |
NASA concurred with our recommendation. In September 2023, NASA reiterated its earlier position that a complete, iterative design approach is currently documented in its policy. NASA referred to the same policies we reviewed for our report, as well as the NASA Systems Modeling Handbook for Systems Engineering, which NASA released more recently, in November 2022. The handbook shows how system modeling using the Systems Modeling Language can be integrated into NASA's systems engineering processes, which applies to systems that include hardware and software. The release of this handbook partially addressed the recommendation by partially implementing a sub-principle of iterative design approaches--using modern design tools during both hardware and software development. In February 2024, NASA updated its procedural requirements, NPR 7123.1D, NASA Systems Engineering Processes and Requirements, to clarify that the NASA Systems Modeling Handbook for Systems Engineering is a resource for implementing the required technical processes in NASA's programs and projects. This update connects the handbook to required policy and therefore fully implements the sub-principle. In addition, NASA's policies have yet to be updated to fully implement the leading sub-principle of using iterative design and testing to identify a minimum marketable product that can be followed by successive updates. In April 2024, NASA revalidated its procedural requirements, NPR 7120.7A, NASA Information Technology Program and Project Management Requirements. As part of this effort, NASA added language directing projects to use minimally viable products (MVP) and minimally marketable products (MMP). However, the new language states that the MVP and MMP are to be used in incremental development efforts, an investment that is portioned into discrete increments to develop capabilities that the larger investment will deliver. This definition is distinct from iterative development, which involves ongoing user input through an iterative design effort that refines requirements and identifies an MVP to be delivered to the user. Further, NPR 7120.7A explicitly states that it applies to IT service lines and IT projects that do not include IT incorporated within space flight, space technology, or aeronautics research projects. NASA space flight projects--to include components beyond IT--are the major acquisitions that are the subject of this recommendation. We stand behind our recommendation that NASA should fully implement in policy the leading sub-principle of iterative design and testing as a way to structure its major projects to develop and deliver relevant capabilities faster. We will continue to monitor NASA's progress in this area.
|
| National Aeronautics and Space Administration | The NASA Administrator should ensure that the NASA Office of the Chief Engineer update NASA acquisition policies to fully implement the following principle throughout development: incorporating feedback from users of initial capabilities (Recommendation 9). |
NASA concurred with our recommendation. In September 2023, NASA reiterated its earlier position that that its policies require feedback to address challenges and inform designs for future programs and projects. NASA referred to policies we previously reviewed for our report but did not provide evidence of updates that fully implement the leading principle of incorporating feedback from users of initial capabilities. While NASA's policies require programs to examine and incorporate lessons learned into the planning process, they do not require programs to incorporate feedback from users, in particular, to inform improvements or changes in capabilities for the next system design. In July 2024, NASA indicated it has conducted an in-depth review of our report and will coordinate internally to identify specific policies in place that meet the intent of our recommendation. We think this approach is potentially misguided and instead encourage NASA to revise the relevant policy documents that we included in our analysis--and discussed with senior officials from NASA's Office of the Chief Engineer, the Aeronautics Research Mission Directorate, the Science Mission Directorate, and others multiple times throughout the course of our review--that underpinned this report recommendation. We will continue to track NASA's progress in this area.
|