Aviation Safety: Additional FAA Efforts Could Enhance Safety Risk Management
GAO-12-898
Published: Sep 12, 2012. Publicly Released: Sep 12, 2012.
Skip to Highlights
Highlights
What GAO Found
The Federal Aviation Administration (FAA) and its business lines and offices are in different stages of their implementation of Safety Management Systems (SMS). FAA finalized its agency-wide implementation plan in April 2012, and the Air Traffic Organization (ATO) has completed its SMS implementation, but other FAA SMS efforts are in the early stages. FAA business lines, such as the Aviation Safety Organization (AVS) and the Office of Airports (ARP), have SMS guidance and plans largely in place and have begun to integrate related practices into their operations, but many implementation tasks remain incomplete, and officials and experts project that full SMS implementation could take many years.
There are a number of key practices that can help agencies plan for and efficiently implement new projects, including large scale transformations such as FAA's SMS implementation, and FAA has many in place. For example, FAA has support from top leadership and a clear project mission. However, FAA has only partially addressed other key practices such as developing a project plan to track SMS implementation, and FAA has not addressed performance-related practices such as establishing SMS performance measures or links between employees' performance standards and SMS.
Several challenges remain that may affect FAA's ability to effectively implement SMS. FAA is taking steps to address some challenges and stakeholder concerns, but challenges related to data sharing and data quality; capacity to conduct SMS-based analyses and oversight; and standardization of policies and procedures could negatively affect FAA's efforts to implement SMS in a timely and efficient manner. Further, FAA officials stated that SMS implementation will require some skills that agency employees do not have, but FAA has not yet assessed the skills of its workforce to identify specific gaps in employee expertise. In addition, while existing federal law protects any data collected for SMS, any data airports collect could be subject to state-specific Freedom of Information Act laws, a gap that could create a disincentive for airports to fully participate in SMS implementation.
Why GAO Did This Study
The nation's aviation system is one of the safest in the world, but with air travel projected to increase over the next 20 years, efforts to ensure the continued safety of aviation are increasingly important. The FAA is seeking to further enhance safety by shifting to a data-driven, risk-based safety oversight approach--referred to as SMS. SMS implementation is required for FAA and several of its business lines and the agency is taking steps to require industry implementation.
As requested, this report addresses (1) the status of FAA's implementation of SMS, (2) the extent to which FAA's SMS efforts have been consistent with key practices for successful planning and implementation of a new program, and (3) challenges FAA faces in implementing SMS. To address these issues, GAO reviewed FAA SMS documents, compared FAA efforts to key practices, and interviewed agency and industry officials.
Recommendations
GAO recommends that FAA develop systems to: track SMS implementation, evaluate employee performance as it relates to SMS, and assess whether SMS meets its goals and objectives; conduct a workforce analysis for SMS; and consider strategies to address airports' data concerns. The Department of Transportation agreed to consider the recommendations and provided clarifying information about SMS, which GAO incorporated.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Transportation | To enhance the effectiveness of efforts to implement SMS and maximize the positive impact of SMS implementation on aviation safety, and to better evaluate the effectiveness of the agency's efforts to implement SMS, the Secretary of Transportation should direct the FAA Administrator to develop a system to assess whether SMS meets its goals and objectives by identifying and collecting related data on performance measures. |
The nation's aviation system is one of the safest in the world, but with air travel projected to increase over the next 20 years, efforts to ensure the continued safety of aviation are increasingly important. The FAA is seeking to further enhance safety by shifting to a data-driven, risk-based safety oversight approach- referred to as Safety Management System (SMS). SMS implementation is required for FAA and several of its business lines and the agency is taking steps to require industry implementation. In 2012, GAO reported that the agency-wide project plan for SMS implementation is a single page of high-level milestones. FAA has plans to develop a system to monitor and track the progress of activities needed to implement SMS, but FAA does not currently have a system for tracking agency-wide SMS implementation, a key practice particularly important during the initial planning phase of project implementation. Therefore, GAO recommended that the Administrator develop a system to assess whether SMS meets its goals and objectives by identifying and collecting related data on performance measures. In 2016, GAO confirmed that FAA developed a system to manage and measure safety performance in the aerospace system known as Safety Performance Management (SPM). According to agency documents, SPM includes steps to identify significant safety issues, perform safety risk management, and develop risk control and monitoring plans, among other steps. This system identifies safety issues and collects data to monitor and measure progress to determine if safety management goals and objectives are being met. FAA officials stated that this system has already been put into use by the agency, and they will continue to refine it to ensure that it works for the agency. As a result, FAA now has a system to identify and monitor safety issues agency-wide.
|
| Department of Transportation | To enhance the effectiveness of efforts to implement SMS and maximize the positive impact of SMS implementation on aviation safety, and to align strategic goals with employee efforts, the Secretary of Transportation should direct the FAA Administrator to develop a system to evaluate employees' performance as it relates to SMS. |
The nation's aviation system is one of the safest in the world, but with air travel projected to increase over the next 20 years, efforts to ensure the continued safety of aviation are increasingly important. The FAA is seeking to further enhance safety by shifting to a data-driven, risk-based safety oversight approach- referred to as Safety Management System (SMS). SMS implementation is required for FAA and several of its business lines and the agency is taking steps to require industry implementation. In 2012, GAO reported that FAA does not consistently evaluate employees' performance on SMS-related tasks, which is a key practice for implementing a new program. We have previously reported that effective performance management systems create a clear linkage between individual performance and organizational success. FAA's organizational mission and goal, and that of SMS, is to improve safety, yet FAA does not have a system for assessing the extent to which staff are effectively supporting SMS. Therefore, GAO recommended that the Administrator develop a system to evaluate employees' performance as it relates to SMS. In 2016, GAO confirmed that FAA had incorporated SMS into its employee Organizational Success Increase (OSI) measures. SMS goals are incorporated into employee performance in an indirect way, and are not explicitly identified by tasks or activities. For example, FAA established and met an organizational success effort for SMS, as indicated in FAA's fiscal year 2014 year-end evaluation. In this case, FAA staff demonstrated using a risk-based decision making approach to develop risk mitigation strategies for a safety issue. The SMS measure was also included and met in the fiscal year 2015 OSI and continues to be included in 2016 measures. As a result, FAA now has a system to evaluate employees' performance as it relates to safety management issues.
|
| Department of Transportation | To enhance the effectiveness of efforts to implement SMS and maximize the positive impact of SMS implementation on aviation safety, and to better manage implementation, the Secretary of Transportation should direct the FAA Administrator to develop a system to track and report on SMS implementation across business lines. |
In 2012, GAO reported that Federal Aviation Administration's (FAA) implementation of its new Safety Management System (SMS) approach was consistent with many, but not all key practices for successful project implementation. There are a number of key practices and implementation steps that can help agencies successfully plan for and implement new projects, including large scale transformative ones such as FAA's implementation of SMS. As GAO has previously reported, addressing these practices can help an agency improve its efficiency, effectiveness, and accountability. At the time of GAO's audit, FAA had many of these key factors in place, such as established support from top leadership and a clear project mission. However, FAA had only partially addressed other key practices such as developing a project plan to track SMS implementation. FAA officials stated that they had plans to develop a system to monitor and track the progress of activities needed to implement SMS, but FAA did not have a system for tracking agency-wide SMS implementation, a key practice particularly important during the initial planning phase of project implementation. Therefore, GAO recommended that the Secretary of Transportation direct the FAA Administrator to develop a system to track and report on SMS implementation across business lines. In 2015, GAO confirmed that FAA issued a policy (Order 8000.369A) requiring the development of a system for tracking and reporting on SMS implementation within and across business lines. To implement this policy, FAA utilized the FAA SMS Committee, which is comprised of managers from each of the four business lines and one staff office currently implementing SMS. The Committee meets twice a month and reports to the FAA SMS Executive Council. The FAA SMS Committee currently monitors SMS implementation and provides reports to the FAA SMS Executive Council regarding the status of activities identified in the FAA SMS Implementation Plan. Meeting notes are maintained by the Office of Accident Investigation and Prevention, FAA's lead office for SMS. As a result, FAA has implemented another key practice enhancing the effectiveness of the agency's efforts to implement SMS and help to maximize the positive impact of SMS implementation on aviation safety.
|
| Department of Transportation | To enhance the effectiveness of efforts to implement SMS and maximize the positive impact of SMS implementation on aviation safety, and to better leverage existing resources and facilitate SMS implementation, the Secretary of Transportation should direct the FAA Administrator to conduct a workforce analysis to inventory existing employee skills and abilities and develop strategies for addressing any SMS-related gaps identified. |
The nation's aviation system is one of the safest in the world, but with air travel projected to increase over the next 20 years, efforts to ensure the continued safety of aviation are increasingly important. The Federal Aviation Administration (FAA) is seeking to further enhance safety by shifting to a data-driven, risk-based safety oversight approach--referred to as Safety Management System (SMS). SMS implementation is required for FAA and several of its business lines and the agency is taking steps to require industry implementation. In 2012, GAO reported that SMS implementation across FAA requires some skills that agency employees do not have, yet FAA had not formally assessed the skills of its workforce to identify any gaps in the expertise required to implement SMS or determine how to fill those gaps. Stakeholders and experts also questioned whether FAA currently has the resources and capacity needed to fully implement SMS. For example, experts noted that FAA may not have the requisite engineers and other staff to participate in safety risk management efforts, or FAA inspectors to oversee individual airport and air carrier SMS programs. Despite these concerns, FAA has not yet conducted a strategic workforce assessment to accurately determine the skills and staffing levels it needs to manage SMS. Although FAA's SMS implementation plan recommends that business lines create such staffing analyses, none have done so. Nor has FAA conducted an agency-wide workforce assessment for SMS. GAO's internal control standards state that agencies should ensure that skill needs are continually assessed to ensure workforces have the skills necessary to help the agency meet its goals. Therefore, GAO recommended that the Administrator conduct a workforce analysis to inventory existing employee skills and abilities and develop strategies for addressing any SMS-related gaps identified. In 2016, GAO confirmed that FAA developed and administered a competency survey to assess the skills of employees performing aviation safety risk and data analysis to baseline the current competency levels of the workforce. In the Project's Closeout Report, FAA identified gaps in the skills needed to implement SMS and made recommendations to address these gaps, such as creating a job task analysis for their safety analytics workforce, covering both existing analysts and future hires. As a result, FAA now knows whether its staff has sufficient skills or competencies to implement SMS and is taking steps to address the identified gaps.
|
| Department of Transportation | To enhance the effectiveness of efforts to implement SMS and maximize the positive impact of SMS implementation on aviation safety, and to maximize the positive impact of SMS implementation on aviation safety, the Secretary of Transportation should direct the FAA Administrator to consider strategies to address airports' concerns that may negatively affect data collection and data sharing, including asking Congress to provide additional protections for SMS data collected by public entities. |
The nation's aviation system is one of the safest in the world, but with air travel projected to increase over the next 20 years, efforts to ensure the continued safety of aviation are increasingly important. The Federal Aviation Administration (FAA) is seeking to further enhance safety by shifting to a data-driven, risk-based safety oversight approach?referred to as Safety Management System (SMS). SMS implementation is required for FAA and several of its business lines and the agency is taking steps to require industry implementation. In 2012, GAO reported that FAA's implementation of its new SMS approach was consistent with many, but not all key practices for successful project implementation. Several challenges remain that may affect FAA's ability to effectively implement SMS, including challenges related to data sharing. Airport and air carrier representatives are concerned that the public disclosure of safety data could result in negative publicity or legal liability in the event of an incident or accident. While existing federal law protects any data collected for SMS, any data airports collect and any data air carriers share with airports could be subject to state-specific Freedom of Information Act (FOIA) laws, a gap that could create a disincentive for airports to fully participate in SMS implementation. FAA officials said that data protection and legal liability are two of the major concerns throughout the aviation industry that could hinder the implementation of SMS, but airports could find ways to limit the public release of air carrier safety information. In FAA's official response to comments on two Notices of Proposed Rulemaking, FAA stated that airport officials are best situated to understand how to comply with state laws. Nonetheless, GAO found consensus among NTSB and many aviation stakeholders that FAA should seek congressional action regarding the protection of airport data from state FOIA laws. To enhance the effectiveness of efforts to implement SMS, GAO recommended that the Secretary of Transportation direct the FAA Administrator to consider strategies to address airports' concerns that may negatively affect data collection and data sharing, including asking Congress to provide additional protections for SMS data collected by public entities. In 2015, GAO confirmed that FAA held discussions with representatives from airports and airport associations to review specific SMS data that should be considered for protection. The FAA Office of Airports received comments from these stakeholders on data collection, protection, and sharing. After consideration and study of the issue, FAA concluded that it will not request a legislative proposal for additional protections for SMS data. By considering this information, FAA is in a better position to understand stakeholder concerns regarding SMS data collection, protection and sharing.
|
Full Report
GAO Contacts
Gerald L. Dillingham
Topics
AircraftAircraft accidentsAircraft industryCommercial aviationFederal regulationsProgram evaluationRisk managementSafety regulationStandardsStrategic planningSystems analysis