Artificial Intelligence: Agencies Have Begun Implementation but Need to Complete Key Requirements
GAO-24-105980
Published: Dec 12, 2023. Publicly Released: Dec 12, 2023.
Fast Facts
Artificial intelligence is rapidly changing the world and could improve government operations. For example, federal agencies can use AI to analyze drone photos and large datasets. But safeguards are needed to manage AI risks.
Federal law and guidance have several requirements for agencies implementing AI, but they haven't all been met. For example, there's no government-wide guidance on how agencies should acquire and use AI. Without such guidance, agencies can't consistently manage AI. And until all requirements are met, agencies can't effectively address AI risks and benefits.
Our 35 recommendations address these issues and more.
Highlights
What GAO Found
Twenty of 23 agencies reported about 1,200 current and planned artificial intelligence (AI) use cases—specific challenges or opportunities that AI may solve. Three agencies reported not having uses for AI. Agency reported uses included analyzing data from cameras and radar to identify border activities, analyzing photographs from drones, and targeting of scientific specimens for planetary rovers. The National Aeronautics and Space Administration (NASA) and the Department of Commerce (Commerce) reported the highest number of AI use cases (see figure).
Agency Reported AI Use Cases in Fiscal Year 2022
Most of the reported AI use cases were in the planning phase and not yet in production (i.e., currently used) (see figure). In about 200 instances, agencies reported that they were currently using AI.
Agency Reported Artificial Intelligence (AI) Use Case Lifecycle Stage, as of Fiscal Year 2022
a“Other” includes life cycle stage responses not included in the 2021 Guidance for Creating Agency Inventories of Artificial Intelligence Use Cases, such as “research and design” and “exploratory.”
GAO's analysis of agencies' inventories of use cases identified instances of incomplete and inaccurate data. Specifically, five agencies provided comprehensive information for each of their reported use cases while the other 15 had instances of incomplete and inaccurate data. For example, some inventories did not include required data elements, such as the AI life cycle stage or an indication of whether an AI use case was releasable or not. In addition, two inventories included AI uses that were later determined by the agencies to not be AI. Without accurate inventories, the government's management of its use of AI will be hindered by incomplete and inaccurate data.
Federal agencies have taken initial steps to comply with AI requirements in executive orders and federal law; however, more work remains to fully implement these.
- Commerce and the General Services Administration (GSA) fully implemented selected requirements. Specifically, Commerce created a plan to develop AI technical standards and GSA established the AI Center of Excellence.
- The Offices of Management and Budget (OMB) and Personnel Management (OPM) did not fully implement selected requirements. OMB has not yet developed guidance for the acquisition and use of AI. OPM has not yet established or updated an occupational category for those employees performing AI work.
- The Office of Science and Technology Policy (OSTP) did not communicate its designation of which agencies were expected to fulfill specific AI requirements.
- Ten of 23 agencies implemented all AI requirements specific to their agencies, 12 implemented some but not all, and one was exempt from the requirements. These requirements included preparing an inventory on the use of AI, planning for inventory updates, and planning for AI regulatory authorities.
Addressing these requirements will improve agency identification, development, implementation, and oversight of AI.
Why GAO Did This Study
While there are varying definitions of AI, they generally refer to computing systems that “learn” how to improve their performance. AI has the potential to rapidly change the world and holds substantial promise for improving government operations. However, AI poses risks that can negatively impact individuals, groups, organizations, communities, and society.
The President's fiscal year 2023 budget request included $1.8 billion for nondefense research and development investment in AI. In addition, some agencies are using AI operationally to identify information security threats and facilitate the review of large datasets, among other uses. Given the rapid growth in capabilities and widespread adoption of AI, the federal government should have safeguards to manage AI's complexities, risks, and societal consequences.
In this report, GAO reviewed the implementation of AI at major federal agencies. This report examines (1) federal agency reported current and planned uses of AI, (2) the extent to which federal agencies' AI reporting was comprehensive and accurate, and (3) the extent to which federal agencies have complied with selected federal policy and guidance on AI.
This review focused on agencies with government-wide roles in AI implementation (including OMB and OPM) as well as agencies with individual responsibilities for AI implementation (including 23 of the 24 Chief Financial Officers (CFO) Act of 1990 agencies). The Department of Defense was excluded because GAO had issued recent AI reports on that department.
GAO reviewed the 23 civilian CFO Act agencies' AI inventories and analyzed reported use cases and supporting data. GAO also compared agency reported data to the requirements for developing an AI inventory outlined in federal guidance.
In addition, GAO identified requirements from executive orders, OMB guidance, and a law regarding the implementation of AI. GAO then assessed agencies' implementation of these requirements. GAO also interviewed relevant officials.
Recommendations
GAO is making 35 recommendations to 19 agencies, including OMB, to fully implement federal AI requirements. Specifically, GAO is recommending that:
- Fifteen agencies update their AI use case inventories to include required information and take steps to ensure the data aligns with guidance.
- OMB, OSTP, and OPM implement AI requirements with government-wide implications, such as issuing guidance and establishing or updating an occupational series with AI-related positions.
- Twelve agencies fully implement AI requirements in federal law, policy, and guidance, such as developing a plan for how the agency intends to conduct annual inventory updates; and describing and planning for regulatory authorities on AI.
Of the 19 agencies, ten agencies agreed with their recommendations; three agencies partially agreed with one or more recommendations; four agencies neither agreed nor disagreed; and one agency did not agree with its recommendation. OMB agreed with one recommendation but disagreed with another because it had taken recent action. GAO maintains that the recommendations in the report are warranted.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Office of Management and Budget | The Director of OMB should ensure that the agency issues guidance to federal agencies in accordance with federal law, that is to (a) inform the agencies' policy development related to the acquisition and use of technologies enabled by AI, (b) include identifying responsible AI officials (RAIO), (c) recommend approaches to remove barriers for AI use, (d) identify best practices for addressing discriminatory impact on the basis of any classification protected under federal nondiscrimination laws, and (e) provide a template for agency plans that includes the required contents. (Recommendation 1) |
OMB agreed with the recommendation. As of May 2024, OMB has partially addressed the requirement. Specifically, while OMB has met most of this requirement, the agency has not provided agencies with a template for their plans, as required by the AI in Government Act of 2020. We will continue to monitor the implementation of this recommendation.
|
| Office of Management and Budget | The Director of OMB should ensure that the agency develops and posts a public roadmap for the agency's policy guidance to better support AI use, and, where appropriate, include a schedule for engaging with the public and timelines for finalizing relevant policy guidance, consistent with EO 13960. (Recommendation 2) |
OMB disagreed with this recommendation as it appeared in a draft version of this report and stated it believed that this action was superseded by the publication of EO 14110. OMB further noted that the EO contains a list of actions for OMB, alongside deadlines for finalizing policy guidance. EO 14110 was issued in October 2023, after we provided OMB a copy of a draft of this report. We updated our report to show that the issuance of the new EO would implement our recommendation. In July 2024, we assessed EO 14110 and determined that it fully addressed the recommendation.
|
| Office of Science and Technology Policy |
Priority Rec.
The Director of the Office of Science and Technology Policy should communicate a list of federal agencies that are required to implement the Regulation of AI Applications memorandum requirements (M-21-06) to inform agencies of their status as implementing agencies with regulatory authorities over AI. (Recommendation 3) |
OSTP did not agree with the recommendation in our report. As of July 2024, we have not received any updates on OSTP efforts to address this recommendation. When we confirm what actions the agency has taken to address this recommendation, we will provide updated information.
|
| Office of Personnel Management | The Director of OPM should ensure that the agency (a) establishes or updates and improves an existing occupational series with AI-related positions; (b) establishes an estimated number of AI-related positions, by federal agency; and, based on the estimate, (c) prepares a 2-year and 5-year forecast of the number of federal employees in these positions, in accordance with federal law. (Recommendation 4) |
OPM partially concurred with the recommendation. As of May 2024, OPM has partially addressed the requirement. Specifically, while OPM has estimated the number of federal AI-related positions and prepared a forecast of the number of federal employees in the position. It has not established nor updated the occupational series with AI related positions. We will continue to monitor the implementation of this recommendation.
|
| Office of Personnel Management | The Director of OPM should ensure that the agency creates an inventory of federal rotational programs and determines how these programs can be used to expand the number of federal employees with AI expertise, consistent with EO 13960. (Recommendation 5) |
OPM concurred with the recommendation. At the time of our report, OPM stated that it had created an inventory after a June 2023 data call. In February 2024, OPM published a memorandum that included the federal inventory of rotational programs. In addition, the memorandum included recommendations on how to expand the rotational programs. In July 2024, we assessed the memorandum and determined that the agency has fully implemented this recommendation.
|
| Office of Personnel Management | The Director of OPM should ensure that the agency issues a report with recommendations for how the programs in the inventory can be used to expand the number of federal employees with AI expertise and shares it with the interagency coordination bodies identified by the Chief Information Officers Council, consistent with EO 13960. (Recommendation 6) |
OPM concurred with the recommendation. At the time of our report, OPM stated that it was finalizing a report on how programs in the inventory can be used to expand the number of employees with AI expertise. In February 2024, OPM published a memorandum that provided recommendations to expand rotational programs to employees with AI expertise. In July 2024, we assessed the memorandum and determined that the agency has fully implemented this recommendation.
|
| Office of Personnel Management | The Director of OPM should ensure that the agency develops a plan to either achieve consistency with EO 13960 section 5 for each AI application or retires AI applications found to be developed or used in a manner that is not consistent with the order. (Recommendation 7) |
OPM concurred with the recommendation. As of July 2024, we have not received any updates on agency efforts to address this recommendation. When we confirm what actions the agency has taken to address this recommendation, we will provide updated information.
|
| Department of Agriculture | The Secretary of Agriculture should ensure that the department (a) reviews the department's authorities related to applications of AI, and (b) develops and submits to OMB plans to achieve consistency with the Regulation of AI Applications memorandum (M-21-06). (Recommendation 8) |
Agriculture agreed with the recommendation and in a March 2024 update, the agency stated that it had plans to address this recommendation. Specifically, Agriculture stated that it would review authorities related to applications of AI and submit a plan to OMB to achieve consistency with M-21-06. These actions are expected to be completed by 12/1/2024. We will continue to monitor the implementation of this recommendation.
|
| Department of Agriculture | The Secretary of Agriculture should ensure that the department updates its AI use case inventory to include all the required information, at minimum, and takes steps to ensure that the data in the inventory aligns with provided instructions. (Recommendation 9) |
Agriculture agreed with the recommendation and in a March 2024 update, the agency stated there were efforts underway to address the recommendation. Specifically, Agriculture stated it has reviewed the process for developing its AI Inventory and has begun to implement changes to address the recommendations. In addition, the agency plans to update its AI use case inventory as specified by OMB guidance. These actions are expected to be completed by 12/1/2024. We will continue to monitor the implementation of this recommendation.
|
| Department of Commerce | The Secretary of Commerce should ensure that the department develops a plan to either achieve consistency with EO 13960 section 5 for each AI application or retires AI applications found to be developed or used in a manner that is not consistent with the order. (Recommendation 10) |
Commerce agreed with the recommendation and in a July 2024 update, the agency stated that it had plans to address this recommendation. Specifically, Commerce stated that it would review its AI inventory and ensure consistency with M-21-06. In addition, Commerce officials stated they would submit a corresponding plan to the agency chief AI officer. These actions are expected to be completed by 12/30/2024. We will continue to monitor the implementation of this recommendation.
|
| Department of Commerce | The Secretary of Commerce should ensure that the department updates its AI use case inventory to include all the required information, at minimum, and takes steps to ensure that the data in the inventory aligns with provided instructions. (Recommendation 11) |
Commerce agreed with the recommendation and in a July 2024 update, the agency stated that it had plans to address this recommendation. Specifically, Commerce stated it will review and update its AI inventory, as required. These actions are expected to be completed by 9/30/2024. We will continue to monitor the implementation of this recommendation.
|
| Department of Education | The Secretary of Education should ensure that the department develops a plan to either achieve consistency with EO 13960 section 5 for each AI application or retires AI applications found to be developed or used in a manner that is not consistent with the order. (Recommendation 12) |
Education concurred with the recommendation to the extent it included a reference to EO 13960 as amended by EO 14110. In a May 2024 update, the agency stated there were efforts underway to address the recommendation. Specifically, Education stated it was developing guidance to address oversight of AI technologies and use cases that will address the requirements of both EO 13960 and new EO 14110. An expected completion date was not provided. We will continue to monitor the implementation of this recommendation.
|
| Department of Energy | The Secretary of Energy should ensure that the department updates its AI use case inventory to include all the required information, at minimum, and takes steps to ensure that the data in the inventory aligns with provided instructions. (Recommendation 13) |
Energy concurred with the recommendation. In March 2024, Energy stated the department will update its use case inventory to include all required information consistent with OMB's revised instructions. The estimated completion date is 12/31/2024. We will continue to monitor the implementation of this recommendation.
|
| Department of Health and Human Services | The Secretary of Health and Human Services should ensure that the department develops a plan to either achieve consistency with EO 13960 section 5 for each AI application or retires AI applications found to be developed or used in a manner that is not consistent with the order. (Recommendation 14) |
HHS concurred with the recommendation. As of July 2024, we have not received any updates on HHS efforts to address this recommendation. When we confirm what actions the agency has taken to address this recommendation, we will provide updated information.
|
| Department of Health and Human Services | The Secretary of Health and Human Services should ensure that the department updates its AI use case inventory to include all the required information, at minimum, and takes steps to ensure that the data in the inventory aligns with provided instructions. (Recommendation 15) |
HHS concurred with the recommendation. As of July 2024, we have not received any updates on HHS efforts to address this recommendation. When we confirm what actions the agency has taken to address this recommendation, we will provide updated information.
|
| Department of Homeland Security | The Secretary of Homeland Security should ensure that the department develops a plan to either achieve consistency with EO 13960 section 5 for each AI application or retires AI applications found to be developed or used in a manner that is not consistent with the order. (Recommendation 16) |
DHS concurred with the recommendation. In a June 2024 update, DHS stated it created an AI Policy Working Group which is working to update or revise existing policies, procedures, and processes for AI and estimated that it would be completed by January 31, 2025. We will continue to monitor the implementation of this recommendation.
|
| Department of Homeland Security | The Secretary of Homeland Security should ensure that the department (a) reviews the department's authorities related to applications of AI and (b) develops and submits to OMB plans to achieve consistency with the Regulation of AI Applications memorandum (M-21-06). (Recommendation 17) |
DHS concurred with the recommendation. In a June 2024 update, DHS stated its OCIO and PLCY offices are working to address this recommendation. These actions are expected to be completed by 12/31/2024. We will continue to monitor the implementation of this recommendation.
|
| Department of Homeland Security | The Secretary of Homeland Security should ensure that the department updates its AI use case inventory to include all the required information, at minimum, and takes steps to ensure that the data in the inventory aligns with provided instructions. (Recommendation 18) |
DHS concurred with the recommendation. In a March 2024 update, DHS provided its 2023 AI use case inventory in response to this recommendation; However, we assessed the inventory and determined that gaps remained. We will continue to monitor the implementation of this recommendation.
|
| Department of the Interior | The Secretary of the Interior should ensure that the department develops a plan to either achieve consistency with EO 13960 section 5 for each AI application or retires AI applications found to be developed or used in a manner that is not consistent with the order. (Recommendation 19) |
Interior concurred with the recommendation and in a May 2024 update, the agency stated that it had plans to address this recommendation. Specifically, Interior stated that it would develop a process to ensure ongoing consistency with EO 13960 section 5 for AI applications or retire those applications not in compliance. In addition, the anticipated process would address the monitoring controls and the retirement of non-AI compliant applications, if applicable. Interior expects to complete these actions by January 15, 2025. We will continue to monitor the implementation of this recommendation.
|
| Department of the Interior | The Secretary of the Interior should ensure that the department (a) reviews the agency's authorities related to applications of AI and (b) develops and submits to OMB plans to achieve consistency with the Regulation of AI Applications memorandum (M-21-06). (Recommendation 20) |
Interior concurred with the recommendation and in a May 2024 update, the agency stated that it had plans to address this recommendation. Specifically, Interior stated that it would review and confirm its authorities related to applications of AI. Interior also planned to identify the authorities that are most relevant to the agency's implementation and usage of AI and develop and submit a plan to OMB to achieve consistency with M-21-06. Interior expects to complete these actions by December 1, 2024. We will continue to monitor the implementation of this recommendation.
|
| Department of the Interior | The Secretary of the Interior should ensure that the department updates its AI use case inventory to include all the required information, at minimum, and takes steps to ensure that the data in the inventory aligns with provided instructions. (Recommendation 21) |
Interior concurred with the recommendation and in a May 2024 update, the agency stated that it had plans to address this recommendation. Specifically, Interior stated that it would establish an AI use case policy and AI use case data collection process to improve the collection of required information. Interior expects to complete these actions by December 31, 2024. We will continue to monitor the implementation of this recommendation.
|
| Department of Labor | The Secretary of Labor should ensure that the department updates its AI use case inventory to include all the required information, at minimum, and takes steps to ensure that the data in the inventory aligns with provided instructions. (Recommendation 22) |
Labor did not state whether the department agreed or disagreed with our recommendations. As of July 2024, we have not received any updates on agency efforts to address this recommendation. When we confirm what actions the agency has taken to address this recommendation, we will provide updated information.
|
| Department of State | The Secretary of State should ensure that the department updates its AI use case inventory to include all the required information, at minimum, and takes steps to ensure that the data in the inventory aligns with provided instructions. (Recommendation 23) |
State did not state whether the department agreed or disagreed with our recommendation. In an August 2024 update, the agency stated has plans to address the recommendation. Specifically, State Department stated it intends to update its AI use case inventory in alignment with OMBs finalized guidance for 2024 Agency Artificial Intelligence Reporting per EO 14110. An expected completion date was not provided. We will continue to monitor the implementation of this recommendation.
|
| Department of Transportation | The Secretary of Transportation should ensure that the department (a) reviews the department's authorities related to applications of AI and (b) develops and submits to OMB plans to achieve consistency with the Regulation of AI Applications memorandum (M-21-06). (Recommendation 24) |
Transportation concurred with the recommendation. In May 2024, Transportation provided evidence that showed its authorities related to applications of AI were reviewed and that its plans to achieve consistency with the Regulation of AI Applications memorandum (M-21-06) were submitted to OMB in April 2024. We assessed the documentation provided and determined that this recommendation has been addressed. As a result, the agency has taken steps to increase its transparency with the public and proactively prepare for the complexities and risks that may arise as they strive to implement AI.
|
| Department of Transportation | The Secretary of Transportation should ensure that the department updates its AI use case inventory to include all the required information, at minimum, and takes steps to ensure that the data in the inventory aligns with provided instructions. (Recommendation 25) |
Transportation concurred with the recommendation. In May 2024, DOT provided its most recent AI inventory. However, that inventory was developed using the instructions from a prior year and will need to be updated. We will continue to monitor the implementation of this recommendation.
|
| Department of the Treasury | The Secretary of the Treasury should ensure that the department develops a plan to either achieve consistency with EO 13960 section 5 for each AI application or retires AI applications found to be developed or used in a manner that is not consistent with the order. (Recommendation 26) |
Treasury did not state whether the department agreed or disagreed with our recommendation. In a June 2024 update, the agency stated that it has taken steps to address this recommendation and has plans to publish a public version of the agency AI implementation plan 180 days from the release of M-24-10. When we confirm what actions the agency has taken to address this recommendation, we will provide updated information.
|
| Department of the Treasury | The Secretary of the Treasury should ensure that the department updates its AI use case inventory to include all the required information, at minimum, and takes steps to ensure that the data in the inventory aligns with provided instructions. (Recommendation 27) |
Treasury did not state whether the department agreed or disagreed with our recommendation. In a June 2024 update, the agency stated that once OMBs final guidance on use case reporting is published, it will ensure its inventory satisfies such guidance. An estimated date of completion was not provided. We will continue to monitor the implementation of this recommendation.
|
| Department of Veterans Affairs | The Secretary of Veterans Affairs should ensure that the department updates its AI use case inventory to include all the required information, at minimum, and takes steps to ensure that the data in the inventory aligns with provided instructions. (Recommendation 28) |
VA concurred with the recommendation. In a June 2024 update, the agency provided its AI use case inventory. We assessed the agency's 2023 AI inventory and determined that the inventory does not include all required information. We will continue to monitor the implementation of this recommendation.
|
| Environmental Protection Agency | The Administrator of the Environmental Protection Agency should ensure that the agency fully completes and approves its plan to either achieve consistency with EO 13960 section 5 for each AI application or retires AI applications found to be developed or used in a manner that is not consistent with the order. (Recommendation 29) |
EPA agreed with the recommendation. In a June 2024 update, EPA stated that implemented its new governance plan to ensure compliance with EO13960 during the first quarter of FY2024. However, documentation was not provided. When we confirm what actions the agency has taken to address this recommendation, we will provide updated information.
|
| Environmental Protection Agency | The Administrator of the Environmental Protection Agency should ensure that the agency updates its AI use case inventory to include all the required information, at minimum, and takes steps to ensure that the data in the inventory aligns with provided instructions. (Recommendation 30) |
EPA agreed with the recommendation. In a June 2024 update, EPA stated has updated its AI inventory, as required. However, the agency did not provide an inventory for review. When we confirm what actions the agency has taken to address this recommendation, we will provide updated information.
|
| General Services Administration | The Administrator of General Services should ensure that the agency develops a plan to either achieve consistency with EO 13960 section 5 for each AI application or retires AI applications found to be developed or used in a manner that is not consistent with the order. (Recommendation 31) |
GSA agreed with the recommendations. In January 2024, GSA provided a letter that lays out the actions the agency plans to take to address the recommendation. GSA expects to finalize these actions by September 30, 2024. When we confirm what actions the agency has taken to address this recommendation, we will provide updated information
|
| General Services Administration | The Administrator of General Services should ensure that the agency updates its AI use case inventory to include all the required information, at minimum, and takes steps to ensure that the data in the inventory aligns with provided instructions. (Recommendation 32) |
GSA agreed with the recommendations. In January 2024, GSA provided a letter that lays out the actions the agency plans to take to address the recommendation. GSA expects to finalize these actions by November 29, 2024. When we confirm what actions the agency has taken to address this recommendation, we will provide updated information.
|
| National Aeronautics and Space Administration | The Administrator of the National Aeronautics and Space Administration should ensure that the agency updates and approves the agency's plan to achieve consistency with EO 13960 section 5 for each AI application, to include retiring AI applications found to be developed or used in a manner that is not consistent with the order. (Recommendation 33) |
NASA concurred with our recommendation. In May 2024, NASA provided a letter that lays out the actions the agency plans to take to address the recommendation. NASA expects to finalize these actions by December 2024. When we confirm what actions the agency has taken to address this recommendation, we will provide updated information.
|
| National Aeronautics and Space Administration | The Administrator of the National Aeronautics and Space Administration should ensure that the agency updates its AI use case inventory to include all the required information, at minimum, and takes steps to ensure that the data in the inventory aligns with provided instructions. (Recommendation 34) |
NASA partially concurred with the recommendation. In May 2024, NASA provided a letter that lays out the actions the agency plans to take to address the recommendation. NASA expects to finalize these actions by the end of the first quarter of FY2025 (December 2024). When we confirm what actions the agency has taken to address this recommendation, we will provide updated information.
|
| U.S. Agency for International Development | The Administrator of the U.S. Agency for International Development should ensure that the agency updates its AI use case inventory to include all the required information, at minimum, and takes steps to ensure that the data in the inventory aligns with provided instructions. (Recommendation 35) |
USAID did not state whether the agency agreed or disagreed with our recommendation. In the agency's May 2024 response, USAID provided a summary of taken and planned actions to address the recommendation. The agency expects to complete these actions by January 31, 2025, or one month after the agency submits its 2024 AI use case inventory to OMB. When we confirm what actions, the agency has taken to address this recommendation, we will provide updated information.
|
Full Report
GAO Contacts
Topics
Artificial intelligenceChief information officersCompliance oversightFederal agenciesInventoryPersonnel managementResearch and developmentScience and technologyTechnical standardsData integrity