Federal Research: Additional Actions Needed to Improve Public Access to Research Results

The Agency for Healthcare Research and Quality concurred with this recommendation. According to an AHRQ update as of March 2022, the Data Management Plan (DMP) Policy that AHRQ released on May 26, 2020 was the first step toward making AHRQ funded research data available for public access. AHRQ stated that all Notices of Funding Opportunity Announcements and Requests for Proposals had been updated to include the DMP requirements, and that the DMP has become part of the terms and conditions of award, as appropriate, for grants and contracts. AHRQ's DMP policy directs researchers to describe their plan for the long term preservation of data generated from AHRQ funding and lists specific items that should be included in a DMP, such as how the data will be made discoverable, among other things. Additionally, AHRQ stated that it continues to participate in an interagency open science working group that is aimed at developing guidance and learning best practices to make federally funded research data available for public access. While we believe these efforts are steps to make AHRQ funded data more readily findable and accessible, in order for a member of the public to access AHRQ funded data, a request would need to be submitted to an agency contact person. In February 2023, an AHRQ official stated that, in response to the August 25, 2022, Office of Science and Technology Policy memorandum entitled "Ensuring Free, Immediate, and Equitable Access to Federally Funded Research," AHRQ submitted a draft updated public access plan on February 21, 2023. After receiving input on the revised public access plan, AHRQ will update its agency public access policies. In April 2024, an AHRQ official provided an update stating that AHRQ is in the process of updating its public access policies according to the updated public access plan. Guided by its annual budget appropriation, the official stated AHRQ is continuing its efforts to consider feasible options, including a web-based point of access, for making data publicly available. AHRQ plans to finalize its updated policies by the end of 2024. Once we obtain additional information on steps AHRQ has taken to implement this recommendation, we will provide an update.

The Department of Homeland Security (DHS) concurred with this recommendation and noted that it was in the process of establishing a portal on its website to increase public access to agency-funded research. In January 2022, DHS provided information demonstrating that a DHS-funded publications repository, hosted by NIH, has been linked to the DHS-funded data repository and is publicly available. According to DHS, metadata collected during posting of articles to the publications repository, including unique digital object identifiers, will be used to connect articles to the data in the data repository. As of April 2022, DHS said it continued to work toward making some additional improvements to this linkage, including reviewing the releasability of certain datasets. In March 2023, DHS provided an update stating that it created a website link on the main public facing DHS webpage that directs to an official DHS landing page that has descriptions and links to the various repositories for publicly funded research and associated data sets. A DHS official said this effort was complete as of December 2022. An examination of the website did not allow us to readily find datasets associated with DHS-funded research, and we asked for a more detailed explanation how to identify and find datasets. In April 2024, DHS provided an update stating that it has entered into agreements with a nongovernmental entity, Figshare, and the National Institute of Health to use PubMed Central, as a means of making DHS-funded research data publicly available. We asked how these agreements would ensure DHS-funded research data are readily findable and accessible to the public and are awaiting additional information.

The Department of Veterans Affairs concurred with this recommendation but indicated in its comments on the report that it had already taken steps to implement it. As discussed in the report, the department's efforts to ensure research data availability pertained to a portion of the agency's federally-funded research data. In a June 11, 2020 response, VA officials outlined planned actions and timelines to address this recommendation. Specifically, VA officials stated they would discuss with agency leadership and other potential VA partners how to make a broader set of VA-funded data findable and accessible. They also said they would continue submitting a funding request to the VHA IT Committee for Operations and Maintenance to meet requirements for enabling data to be findable and accessible. In addition, they said they would work through the Veterans Informatics and Computing Infrastructure division to explore opportunities and methods for making data from other types of research publicly accessible. Subsequently, in December 2020, VA officials provided an update stating that the COVID-19 pandemic impacted these plans but that they still intend to pursue previously identified plans, along with new efforts such as increasing the use of VA Data Commons to allow access to clinical and genetic resources data. In February 2022, VA officials provided an update stating that its public access activities would be under the purview of a subcommittee it created for data governance, among other topics. They added that VA is still pursuing the creation of a VA Data Commons, which would include the ability to provide controlled access to research data for non-VA individuals. Furthermore, VA officials stated that it has plans to convene cognizant VA offices in late winter or early spring 2022 to develop a strategy to obtain funding to meet requirements for its proposed actions. In June 2022, VA provided an additional update in which it stated that its subcommittee had held initial meetings regarding its scope and role to advance enterprise approaches to managing ORD-funded research data. VA stated that the subcommittee would develop a written plan to outline further actions to implement the recommendation. In August 2023, VA officials met with GAO staff to provide a presentation on VA's efforts to implement this recommendation. The officials provided information on different pathways through which certain areas of VA-funded research data are made publicly available. Nevertheless, some gaps remain in the availability of VA-funded research data, and the officials said VA continues to pursue the effort to create VA Data Commons. In February 2024, VA officials said they created a publicly available clearinghouse including information on over 100 datasets, as well as summary statistics on its Million Veteran Program on the database of genotypes and phenotypes and established an internal secure site for its data. They also said they conducted a business and cost analysis for how to make data for research funded by one of its offices, the Office of Research and Development, publicly available. In April 2024, VA officials provided another update with details on its cost analysis for making agency-funded data publicly available and said that other VA information technology requirements had a higher priority. Despite these actions, gaps remain in the public availability of VA-funded research data. VA officials said they will provide any new information on initiatives related to making VA-funded research results publicly available.

The Department of Defense concurred with this recommendation stating data management plan (DMP) requirements would be in place by September 2023. In September 2022, DOD provided an update stating that it issued guidance in August 2021 requiring DMPs from grantees. DOD provided documentation of this guidance in May 2023, which states that a data management plan is required either as part of the application package prior to award or 30 days after award at the discretion of component program managers. DOD's September 2022 update also indicated that it had proposed DMP requirements for contractors into the DFARS. DOD provided a further update in June 2023 stating that the proposed changes to DFARS remain under review. In August 2024, DOD said they have been tracking this recommendation but do not yet have any new updates to share. We will provide an update once DOD provides additional information demonstrating the completion of these actions.

The Department of Homeland Security (DHS) concurred with this recommendation, stating that it would develop a departmentwide management directive for research and development data as well as data management plan guidance and a template to document requirements. In a February 2021 update, DHS officials provided draft versions of its department-wide management directive for research data; a draft data management plan guidance document and template; and user guides relating to handling research data resulting from DHS funding. As of April 2022, DHS stated that it continues to work toward finalizing the directive. As of March 2023, DHS provided an estimated completion timeframe of late October 2023 for the directive. When we confirm the directive has been finalized, we will provide updated information. In April 2024, DHS officials said the department has changed its approach, noting that it has stood up an Office of the Chief Data Officer and has started to take steps to address this recommendation. Specifically, they said its Science and Technology (S&T) Directorate was moving forward on a new policy and procedure that would establish data management plan requirements. In July 2024, DHS finalized its policy and procedure for data management plan requirements. However, the policy and requirements only apply to S&T-funded researchers and not those funded by other parts of DHS. As such, we are awaiting further information on steps taken to establish data management plan requirements for all of DHS.

The Department of Defense concurred with this recommendation. According to DOD officials, in May 2020, DOD research organizations performed a self-assessment of their compliance with public access requirements. The self-assessment found that DOD medical organizations were not sending articles stemming from DOD-funded research to DOD's Defense Technical Information Center (DTIC) per DOD policy. While articles that were jointly funded with the National Institutes of Health (NIH) were being sent to NIH's repository, PubMed Central, articles that were not jointly funded with NIH were not being made publicly available. To address this, DOD is taking steps to coordinate with NIH to ensure DOD-funded articles are made available through DOD's public facing interface, PubDefense, by developing a web service to download DOD-funded materials from PMC and also discussing with NIH whether non-NIH-funded documents may be deposited in PMC and then downloaded to DTIC's repository. Additionally, DOD officials stated the self-assessment led to an increase in submissions to DTIC and started conversations with DTIC on the best way to submit documents and DMPs to DTIC in an automated way. In a September 2022 update, DOD stated that it had established a methodology to determine noncompliance and identify the number of journal articles and DMPs generated but not sent to the DOD public access repository. In February 2023, DOD issued a memo directing compilation of an annual inventory of published journal articles, manuscripts, and supporting digital data to ensure researcher compliance. According to a DOD update, the initial FY22 inventory was due in June 2023. In October 2023, DOD provided an update stating that it expects to complete the analysis of the FY22 inventory in the second quarter of FY24. In August 2024, DOD said they have been tracking this recommendation but do not yet have any new updates to share, such as the results of their analysis of the FY 22 inventory. We will update the status of this recommendation when DOD provides additional information on these actions.

The Department of Transportation concurred with this recommendation. According to its response to our report, the Department will build upon existing compliance mechanisms to ensure researcher compliance with its public access plan and associated requirements. As part of this process, the Department reported that it plans to update its public access plan, and initially expected to complete these actions by December 31, 2020. According to May and September 2024 updates, the Department is still developing its updated public access policy. In developing the policy, the Department stated its intent to strengthen the requirements to ensure compliance by requiring each Department funding agency to ensure submission of all research deliverables to the National Transportation Library prior to closing the funding agreement. To ensure the funding agencies are properly carrying out their responsibilities, periodic audits of funding agency compliance will be conducted, and results posted to a Department dashboard showing the level of compliance. Follow up work with the funding agency to ensure training and future compliance could be conducted as deemed necessary to improve individual funding agency/office compliance levels. When we confirm what further actions the Department has taken to finalize its policy and implement this recommendation we will provide additional information.

The Department of Commerce agreed with this recommendation and the National Institute of Standards and Technology (NIST) identified several planned steps to implement it. Specifically, in January 2021, NIST deployed a system developed to identify NIST-authored papers that have been published, and ensure that all NIST authors are in compliance with the requirement that papers are made freely available within 12 months of publication. Previously, NIST also completed a review of awardees' compliance with data management plan requirements, and took steps (e.g., developing a template and checklist) to address issues NIST identified. Additionally, NIST took steps to evaluate an option to ensure compliance with public access requirements by NIST awardees (i.e., extramural researchers). NIST determined that the option it evaluated would not be practical as of April 2021. As of August 2023, NIST stated that it was not yet able to report awardees' compliance with the terms and conditions in their grants because the agency's Grants Management Information System (GMIS) does not track that. According to NIST's update, it is transitioning from GMIS to a different system (eRA Commons). NIST anticipates that the new system will allow monitoring of extramural compliance and the agency should be able to implement compliance tracking. According to a September 2024 update, NIST's transition to the new system has been shifted from October 2024 to October of 2026. In the interim, NIST is looking for ways to determine awardees' compliance with terms and conditions of their agreements, according to the agency's update. We will further update the recommendation status when NIST provides additional information on steps taken to ensure compliance with public access requirements by NIST's extramural researchers.