This is the accessible text file for GAO report number GAO-14-516 entitled 'Aviation Safety: Additional Oversight Planning by FAA Could Enhance Safety Risk Management' which was released on June 25, 2014. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to the Subcommittee on Aviation, Committee on Transportation and Infrastructure, House of Representatives: June 2014: Aviation Safety: Additional Oversight Planning by FAA Could Enhance Safety Risk Management: GAO-14-516: GAO Highlights: Highlights of GAO-14-516, a report to the Subcommittee on Aviation, Committee on Transportation and Infrastructure, House of Representatives. Why GAO Did This Study: The U.S. aviation system is one of the safest in the world, reflecting the work of FAA, industry, and others to continually improve safety. To further enhance safety, in 2005, FAA began adopting a proactive, data-driven, risk-based approach to managing safety, referred to as SMS, and has proposed rules that would require SMS implementation for certain segments of the aviation industry. GAO was asked to review SMS implementation in the aviation industry. This report addresses (1) the status of SMS implementation at FAA and in the aviation industry; (2) key challenges that FAA and industry face in implementing SMS; and (3) actions aviation stakeholders believe FAA could take to improve SMS implementation. GAO reviewed FAA documents and interviewed FAA officials. GAO also interviewed representatives from 20 selected aviation stakeholders, including commercial air carriers, certificated airports, repair stations, and design and manufacturing firms. Because the stakeholders were non- statistically selected based on their size, SMS implementation, and the industry segment represented, their views cannot be generalized to the industry or any industry segment. What GAO Found: The Federal Aviation Administration's (FAA) Air Traffic Organization completed Safety Management System (SMS) implementation in 2010, and five other FAA organizations are implementing it now. SMS is an approach to collect and analyze safety data to identify hazards, manage risks, and take corrective action before an accident occurs. FAA's implementation activities include developing internal SMS guidance and procedures and using them to, among other things, identify hazards in the aviation system and provide oversight of the aviation industry. For example, FAA's Flight Standards Service is developing an SMS-based oversight system for the commercial air carriers it oversees. Although SMS is not yet required for commercial air carriers, airports, or any other industry segment, some are voluntarily implementing SMS as part of several FAA pilot projects. Of the 83 commercial air carriers, 77 are in the process of implementing SMS. FAA anticipates publishing a final rule in September 2014 requiring commercial air carriers to implement SMS. To a lesser extent, other industry segments are voluntarily implementing SMS. For example, according to FAA, 9 of the nation's largest airports are implementing SMS. FAA issued a proposed rule for airport SMS implementation, but development of a final rule has been delayed, and FAA has not yet determined if it will propose rules for other industry segments. Stakeholders and FAA officials speculated that the other industry segments may be waiting to implement SMS until FAA issues additional guidance or a final rule. According to FAA officials, completing the rulemaking processes for commercial air carriers and airports has been a primary challenge to industry SMS implementation. Officials stated that one reason for delay has been difficulty in developing the benefit-cost analyses required for significant regulatory action. However, FAA is revisiting these analyses through the ongoing rulemaking process. Uncertainty about FAA plans for SMS oversight was among the key challenges for aviation industry SMS implementation. Although some inspector training has been provided, representatives from 9 of the 20 stakeholders GAO interviewed cited concerns that FAA inspectors may not be adequately trained to oversee industry SMS activities, and 6 expressed concerns that inspectors throughout FAA may not consistently interpret SMS regulations. However, FAA has not completed plans for its SMS oversight activities, including inspector training, and officials stated that they would not do so until the final rule is published. Without adequate planning of oversight and training of inspectors, FAA could find itself unprepared to meet its oversight responsibilities when final SMS rules are published. Twelve of the 20 aviation stakeholders GAO spoke with identified additional FAA actions that could improve their SMS implementation efforts. For example, 4 stakeholders stated that providing SMS training to their employees was a challenge, and 2 suggested that FAA could assist by providing them access to FAA's SMS training. FAA indicated that it is considering industry stakeholder training needs and provided training through the pilot projects. Fourteen stakeholders were pleased with FAA's collaboration and communication, but 6 of them stated that this effort could be broadened. FAA updates its SMS website information, and FAA's most significant industry SMS forum is focusing more on SMS implementation. What GAO Recommends: GAO recommends that FAA develop a plan for overseeing industry SMS implementation that includes providing guidance and training for FAA inspectors by the time final rules are published. GAO provided DOT with a draft of this report for comment. DOT provided technical corrections which were incorporated as appropriate. View [hyperlink, http://www.gao.gov/products/GAO-14-516]. For more information, contact Gerald L. Dillingham, Ph.D., at (202) 512-2834 or dillinghamg@gao.gov. Contents: Letter: Background: FAA's Internal Implementation of SMS Continues, While Aviation Industry Segments Are in Various Stages of Voluntary Implementation: FAA and Stakeholders Cited Rulemaking, Data Issues, and Stakeholder Uncertainty as Key Challenges to Implementing SMS: Stakeholders Identified Additional Actions FAA Could Take to Improve SMS Effectiveness and Address Implementation Challenges: Conclusions: Recommendation for Executive Action: Agency Comments: Appendix I: Objectives, Scope, and Methodology: Appendix II: GAO Contact and Staff Acknowledgments: Related GAO Products: Tables: Table 1: Flight Standards' Safety Management System Implementation Levels for Commercial Air Carriers: Table 2: Changes in Stakeholder and FAA Inspector Roles under a Safety Management System: Table 3: Industry and International Interviewees: Table 4: Aviation Industry Stakeholder Interviewees: Figures: Figure 1: The Four Components of Safety Management Systems: Figure 2: Selected FAA Organizations Implementing a Safety Management System and Oversight Responsibilities, as of June 2014: Figure 3: Rulemaking Timeline for the Commercial Air Carrier Safety Management System Requirement, as of June 2014: Figure 4: Rulemaking Timeline for the Certificated Airport Safety Management System Requirement, as of June 2014: Abbreviations: ANPRM: advance notice of proposed rulemaking: ARC: aviation rulemaking committee: CFR: Code of Federal Regulations: DOT: Department of Transportation: FAA: Federal Aviation Administration: FOIA: Freedom of Information Act: ICAO: International Civil Aviation Organization: NTSB: National Transportation Safety Board: NextGen: Next Generation Air Transportation System: NPRM: notice of proposed rulemaking: SMS: safety management system: SNPRM: supplemental notice of proposed rulemaking: SSP: state safety program: [End of section] United States Government Accountability Office: GAO: 441 G St. N.W. Washington, DC 20548: June 25, 2014: The Honorable Frank A. LoBiondo: Chairman: The Honorable Rick Larsen: Ranking Member: Subcommittee on Aviation: Committee on Transportation and Infrastructure: House of Representatives: The U.S. aviation system is one of the safest in the world with an extremely low accident rate. This record reflects the efforts of the Federal Aviation Administration (FAA), air carriers, airports, aviation manufacturers, the National Transportation Safety Board (NTSB), and others to continually improve aviation safety. Focus on safety continues to be important, particularly in light of projected air traffic growth over the next 20 years. In keeping with this anticipated growth, FAA is seeking to further enhance safety by shifting to a proactive, data-driven, risk-based approach to safety oversight referred to as a safety management system (SMS). SMS is a formalized process that involves collecting and analyzing data on aviation operations to identify emerging safety problems, determining risk severity, and mitigating that risk to an acceptable level. SMS is becoming the standard throughout the global aviation industry. The International Civil Aviation Organization (ICAO)[Footnote 1] requires SMS for the management of safety risk in air operations, maintenance organizations, air traffic services, and airports as well as for certain flight-training operations and organizations that design and manufacture aircraft. Further, ICAO has published safety management requirements for its member countries that mandate that civil aviation authorities--such as FAA--establish an SMS.[Footnote 2] FAA's internal SMS approach is designed to meet ICAO requirements. ICAO also directs civil aviation authorities to require that certain types of aviation service providers implement SMS. The adoption of SMS is important because the low accident rate increases the importance of analyzing incident and other data for root causes and indicators of potential hazards. According to FAA, SMS can also address organizational and management issues that have been identified as contributing to past accidents. To this end, FAA is (1) engaged in implementing SMS internally across selected organizations within the agency, (2) has undertaken SMS pilot projects for certain segments of the aviation industry, and (3) has developed proposed regulations that would require commercial air carriers and certificated airports to implement SMS.[Footnote 3] In 2012, we reported on FAA's internal implementation of SMS.[Footnote 4] For this report, you asked us to focus on the status of SMS implementation in the aviation industry. We examined the status of SMS implementation in four industry segments--commercial air carriers, certificated airports, design and manufacturing firms, and repair stations. We also examined the current status of FAA's internal SMS implementation efforts and FAA's development of SMS regulations. Specifically, this report addresses the following questions: 1. What is the status of SMS implementation within FAA and for key segments of the aviation industry, including certificated airports, commercial air carriers, repair stations, and design and manufacturing firms? 2. What are the key challenges FAA and the aviation industry face in implementing SMS? 3. What additional actions do aviation stakeholders believe FAA could take to improve SMS implementation and potential effectiveness? To assess the status of SMS implementation within FAA and four key segments of the aviation industry, we reviewed FAA orders, advisory circulars, and pilot program reports as well as FAA guidance for internal SMS implementation and for implementation in the aviation industry. We also interviewed FAA officials and industry trade group representatives. To determine the key challenges FAA and the aviation industry face in implementing SMS, we interviewed FAA officials and representatives from 20 selected aviation stakeholders--commercial air carriers, certificated airports, repair stations, and design and manufacturing firms--that participated in SMS pilot projects. These 20 stakeholders, 5 from each industry segment, were non-statistically selected based on size, SMS implementation, and industry sub-segment represented. (Results of these interviews cannot be projected to all companies in any of the given industry segments). We also reviewed documents from FAA's rulemaking projects for commercial air carriers and certificated airports. To identify additional actions FAA could take to improve SMS implementation and potential effectiveness, we interviewed the industry stakeholders described above and foreign aviation authorities that are overseeing SMS implementation, including Australia's Civil Aviation Safety Authority, the European Aviation Safety Agency, and Transport Canada. In addition, we interviewed two Canadian airlines and one Canadian airport that have implemented SMS. We also interviewed FAA officials to obtain their perspectives on potential actions. See appendix I for a more detailed description of our scope and methodology. We conducted this performance audit from May 2013 to June 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background: When fully implemented, SMS provides a continuous approach to managing safety risk, which FAA expects will improve aviation safety. SMS is not an additional safety program that is distinct from existing activities that accomplish an entity's safety mission, such as quality management, quality assurance, or similar activities. Rather, SMS provides a set of decision-making processes and procedures to plan, organize, direct, and control business activities in a manner that enhances safety and ensures compliance with regulatory standards. According to FAA, the overarching goal of SMS is to improve safety by helping ensure that the outcomes of any management or system activity incorporate informed, risk-based decision making. SMS consists of four key components: (1) safety policy, (2) safety risk management, (3) safety assurance, and (4) safety promotion. (See figure 1.) Together, these four components are intended to provide a systematic approach to achieving acceptable risk levels. FAA provides its personnel with guidance on the principles underpinning these components in its official orders and other internal FAA guidance. To the industry, FAA currently provides SMS guidance via advisory circulars, an SMS newsletter, focus groups, and a dedicated page for the SMS program office on the FAA website. Figure 1: The Four Components of Safety Management Systems: [Refer to PDF for image: illustration] Safety policy: Establishes senior management's commitment to continually improve safety, and defines the methods, processes and organizational structure needed to meet safety goals: Management commitment and accountability; Key safety personnel; Safety policy statement. Safety promotion: Includes training, communication, and other actions to create a positive safety culture within all levels of the workforce: Training; Communication. Safety risk management: Determines the needs for and adequacy of new or revised risk controls based on an assessment of acceptable risk: System analysis and hazard identification; Risk assessment; Risk mitigation. Safety assurance: Evaluates the continued effectiveness of implemented risk control strategies; supports the identification of new hazards: Performance assessment; Performance monitoring and measurement; Continuous improvement. Source: GAO analysis of FAA information. GAO-14-516. [End of figure] FAA is undertaking the transition to SMS in coordination with the international aviation community, working with ICAO to adopt applicable global standards for safety management. ICAO first mandated SMS worldwide for air traffic service providers in 2001. ICAO later specified that member states should mandate SMS implementation for airports, air carriers, and others by 2009. FAA began SMS implementation in 2005, but FAA officials informed ICAO that the agency and industry would not be able to meet the 2009 deadline. The United States filed a "difference" with ICAO--indicating that it does not yet completely comply with the standard--with the understanding that implementation is under way and that FAA is in the midst of a rulemaking to require SMS for commercial air carriers and certificated airports. There have been other actions within the United States to encourage SMS implementation. For instance, in 2007, NTSB recommended that FAA require all commercial air carriers to establish an SMS [Footnote 5] and in 2011 added SMS for all modes of transportation to its Most Wanted List.[Footnote 6] NTSB identified SMS as one of the most critical changes needed to reduce the number of accidents and save lives. FAA's implementation of SMS will affect how the agency oversees the aviation industry. Historically, FAA oversight of airlines, airports, and other regulated entities has involved oversight of such things as operations and maintenance. Once SMS requirements for industry are in place, FAA will continue this oversight, but will also apply SMS principles to its processes for oversight. Specifically, the agency will provide oversight of the safety management systems of service providers, such as commercial air carriers and certificated airports, to help ensure that they are managing safety within their operations through SMS. Internally, FAA has directed the Air Traffic Organization, Airports Organization, Aviation Safety Organization (Aviation Safety), the Office of Commercial Space Transportation, the Next Generation Air Transportation System (NextGen) Office,[Footnote 7] and the Office of Security and Hazardous Materials Safety to implement SMS. Within Aviation Safety, the Aircraft Certification Service (Aircraft Certification),[Footnote 8] and the Flight Standards Service (Flight Standards),[Footnote 9] among others, are also implementing SMS. Oversight of the aviation industry's implementation of SMS will be conducted by FAA's inspector workforce, some of whom have been involved in SMS pilot programs. The inspector workforce includes approximately 4,000 Flight Standards aviation safety inspectors around the country who oversee certificate holders, including commercial air carriers and repair stations, among others. Approximately 950 aircraft certification engineers and inspectors are responsible for overseeing firms that design and manufacture aircraft, aircraft engines, propellers, and other parts and equipment. The nation's certificated airports are overseen by approximately 40 airport certification inspectors assigned to regional offices. In 2009, FAA issued an Advance Notice of Proposed Rulemaking (ANPRM) to solicit comments on establishing a regulatory framework to require SMS for various sectors of the aviation industry, including commercial air carriers, repair stations, and design and manufacturing firms. [Footnote 10] FAA received public comments on the ANPRM but withdrew it[Footnote 11] following a 2010 congressional mandate for FAA to issue a rule requiring air carriers to implement SMS.[Footnote 12] FAA subsequently issued a Notice of Proposed Rulemaking (NPRM) in November 2010 that would require SMS for commercial air carriers.[Footnote 13] FAA simultaneously developed a proposed rule for certificated airports and issued an NPRM in October 2010 that would also require SMS at those facilities.[Footnote 14] The status of rulemaking efforts for repair stations and design and manufacturing firms are discussed below. FAA's Internal Implementation of SMS Continues, While Aviation Industry Segments Are in Various Stages of Voluntary Implementation: FAA Organizations Continue SMS Implementation: Five FAA organizations are proceeding with SMS implementation. A sixth organization, Air Traffic, completed implementation in 2010. Figure 2 shows the FAA organizations that are implementing SMS as well as industry segments included in this report that these selected organizations oversee. Figure 2: Selected FAA Organizations Implementing a Safety Management System and Oversight Responsibilities, as of June 2014: [Refer to PDF for image: organization chart] FAA Administrator: * Air Traffic; * Airports: - Safety and Standards: oversight responsibility for airports; * Commercial Space Transportation; * Aviation Safety: - Flight Standards Service: oversight responsibility for Air carriers and Repair stations; - Aircraft Certification Service: oversight responsibility for Design and manufacturing firms; * NextGen; * Security and Hazardous Materials Safety. Source: GAO analysis of FAA information. GAO-14-516. [End of figure] The Air Traffic Organization completed SMS implementation in 2010 and now uses SMS-based processes to identify hazards, enact mitigations, and assess the extent to which the mitigations are working.[Footnote 15] Since we last reported on their progress in 2012, the other organizations--Aviation Safety, the Airports Organization, the NextGen Office, and the Office of Commercial Space Transportation--have continued SMS implementation by developing SMS guidance and establishing internal SMS procedures, and an additional organization, the Office of Security and Hazardous Materials Safety has begun SMS implementation. * Aviation Safety--including two of its components, Flight Standards and Aircraft Certification--are developing or updating SMS guidance and plans, according to FAA officials.[Footnote 16] For example, Flight Standards is developing for an SMS-based oversight system for commercial air carriers. * The Airports Organization established procedures for conducting safety risk management for certain airport actions that require FAA approval. * At the time of our last report, the NextGen Office and the Office of Commercial Space Transportation were implementing SMS in line with FAA's strategic goal in its 2009-2013 Flight Plan to implement SMS policy in all appropriate FAA organizations.[Footnote 17] In May 2013, FAA issued a revised order that directed these two organizations to implement SMS. * In addition, an FAA order directed another organization, the Office of Security and Hazardous Materials Safety, to implement SMS.[Footnote 18] That organization is updating its inspections and investigations manual to be more consistent with SMS principles and is coordinating with other FAA organizations' SMS efforts to identify hazards in the transport of hazardous materials. Although Not Currently Required, Some Aviation Stakeholders Are Voluntarily Implementing SMS: Voluntary implementation of SMS varies by industry segment, while FAA continues rulemaking to require SMS implementation by commercial air carriers[Footnote 19] and certificated airports. Beginning in 2007, FAA initiated various pilot projects for commercial air carriers, certificated airports, repair stations, and design and manufacturing firms, among others, to encourage voluntary SMS implementation and to guide rulemaking activities. The pilot projects varied among FAA organizations. The Airports Organization and Aircraft Certification pilot projects were conducted for fixed periods of time, while the Flight Standards pilot project, which began in 2007, is ongoing. Flight Standards officials said that the pilot project for commercial air carriers would end after the final rule is published and that they hope to transition the pilot project for the remaining certificate holders (e.g., repair stations) into a more formal program. Participant selection for the pilots also varied. FAA selected the participants for the Aircraft Certification pilot for design and manufacturing firms, while the Flight Standards and Airports Organization pilot project participants volunteered to participate. The pilot projects are more fully described in forthcoming sections of this report. While most commercial air carriers--the first industry segment for which SMS implementation will be mandatory--are moving forward with implementation, a small proportion of certificated airports, repair stations, and design and manufacturing firms have begun SMS implementation. For example, less than 1 percent of the more than 4,700 repair stations are currently implementing SMS. These three industry segments, according to FAA officials and industry stakeholders, may be waiting for additional guidance or rules, which may not be finalized for a number of years, if the time frames for finalizing the air carrier and airport SMS rules are any indication. Commercial Air Carriers: Seventy-seven of the 83 U.S. commercial air carriers are at varying levels of SMS implementation as FAA nears completion of the final SMS rule. Flight Standards developed four SMS implementation levels to benchmark the progress of commercial air carriers (see table 1 below for additional details on the Flight Standards SMS implementation levels). In general, Flight Standards officials work with commercial air carriers to determine their SMS implementation level. Flight Standards reports the SMS level for each participating stakeholder in its monthly SMS newsletter. According to the March 2014 SMS newsletter, of the 77 commercial air carriers implementing SMS, the vast majority of them (58) were at SMS level one or two, 10 were at level three, and 9 were at the final continuous improvement level. [Footnote 20] Table 1: Flight Standards' Safety Management System Implementation Levels for Commercial Air Carriers: Level: One: planning and organization; Description: Top management commits to providing the resources necessary for full implementation of SMS throughout the organization. The organization completes a gap analysis to determine which elements in the SMS framework are not already being performed. Based on the gap analysis, the organization creates an SMS implementation plan to describe how the organization will close any identified gaps. Level: Two: reactive process; Description: The organization develops and implements basic safety risk management and safety assurance processes. These processes include information acquisition, processing, and analysis as well as the establishment of a risk control and corrective action tracking system. Level: Three: proactive process; Description: The organization has a functioning SMS. It applies safety risk management to the initial design of systems, processes, organizations, and products; the development of operational procedures; and planned changes to operational processes. This application of safety risk management includes analysis of systems and tasks, identification of potential hazards, and development of risk controls. Upon completion of level three, all required components are documented, demonstrated, and operational. Level: Four: continuous improvement; Description: Processes are in place, and their performance and effectiveness have been verified. The complete safety assurance process, including continuous monitoring, and the remaining features of the other safety risk management and safety assurance processes are functioning. A major objective of a successful SMS is to attain and maintain this continuous improvement status for the life of the organization. Source: GAO analysis of FAA information. GAO-14-516. [End of table] The Airline Safety and Federal Aviation Administration Extension Act of 2010[Footnote 21] stipulated that FAA issue a final SMS rule for commercial air carriers. In November of 2010, FAA published an NPRM that would require all commercial air carriers to implement an SMS that meets the requirements of the new regulation and is acceptable to FAA within 3 years of the effective date of the final rule.[Footnote 22] According to FAA, it did not meet the statutory deadline of August 1, 2012, to issue the final rule due to the lengthy internal DOT review process and complexity of the required benefit-cost analysis.[Footnote 23] The DOT Significant Rulemaking Report for June 2014 indicated that FAA anticipates publishing the final rule in September 2014. (See figure 3.) Commercial air carriers that have already begun to implement SMS must ensure that their efforts comply with the requirements of the final rule. Figure 3: Rulemaking Timeline for the Commercial Air Carrier Safety Management System Requirement, as of June 2014: [Refer to PDF for image: timeline] 2010: NPRM published: November 5, 2010. 2011: NPRM comment period closed: March 7, 2011. Scheduled OST review: March 16, 2012. Scheduled then extended. 2012: Actual OST review: April 12, 2012. Scheduled OMB review: April 16, 2012. Scheduled then extended. Scheduled OMB clearance: July 16, 2012. Scheduled then extended. Scheduled final rule publication: July 27, 2012. Scheduled then extended. 2013: Returned to FAA: March 5, 2013. Resubmitted to OST: June 27, 2013. 2014: Submitted for OMB review: May 20, 2014. Projected OMB clearance: August 22, 2014. Projected final rule publication: September 4, 2014. Abbreviations: FAA: Federal Aviation Administration; NPRM: Notice of Proposed Rulemaking; OMB: Office of Management and Budget; OST: Office of the Secretary of Transportation. Source: GAO analysis of DOT information. GAO-14-516. [End of figure] Certificated Airports: While FAA's rulemaking to require SMS at all 545 certificated airports has been delayed, FAA officials estimated that 9 of the nation's largest certificated airports were voluntarily implementing SMS as of April 2014. Between 2008 and 2012, the Airports Organization conducted three voluntary SMS pilot studies to provide airports opportunities to gain knowledge about SMS and provide feedback to FAA in its rulemaking efforts. The pilot studies also allowed certificated airports to share their SMS implementation practices with other airports. Thirty-one certificated airports of various sizes participated in at least one of the pilot studies.[Footnote 24] Twenty-one of the 22 airports in the first pilot study and 1 airport that did not participate in any of the pilot studies received a grant to develop an SMS plan. To further encourage SMS implementation, in August 2013, the Airports Organization published guidance that provides some certificated airports access to federal funds for certain SMS activities, including making SMS management software eligible for funding through the Airport Improvement Program.[Footnote 25] FAA officials indicated that another goal of the guidance was to ensure that airports would be aware of the basic elements required for SMS implementation plans and manuals. FAA began the SMS rulemaking process for certificated airports in July 2008 and in October 2010 published an NPRM to require that all certificated airports develop and maintain an SMS that is approved by FAA. According to FAA officials, the nature of the comments received on the NPRM led them to significantly modify the proposed rule, and provide another period for public comment. As a result, in December 2012, FAA announced that it would amend the next scheduled step in the rulemaking process and issue a supplemental notice of proposed rulemaking (SNPRM), which was originally scheduled for publication in December 2013, rather than publish a final rule. FAA is considering changes in the proposed rule's applicability and to some proposed requirements, including SMS implementation options for various sizes of certificated airports. FAA has not yet issued the SNPRM, but in June 2014 FAA estimated that the SNPRM would be published for public comment in October 2014. With FAA's decision to issue an SNPRM, it is likely that required SMS implementation for certificated airports will not occur for some time. Figure 4 summarizes key dates related to SMS rulemaking for certificated airports, as indicated in the DOT Significant Rulemaking Report for June 2014. Figure 4: Rulemaking Timeline for the Certificated Airport Safety Management System Requirement, as of June 2014: [Refer to PDF for image: timeline] 2010: NPRM published: October 7, 2010. 2011: NPRM comment period: closed: July 5, 2011. 2012: Decision to issue SNPRM announced: December 10, 2012. 2013: Scheduled OST review: July 22, 2013; Scheduled then extended. Scheduled OMB review: August 23, 2013; Scheduled then extended. Actual OST review: November 13, 2013. Scheduled OMB clearance: November 25, 2013; Scheduled then extended. Scheduled SNPRM publication: December 9, 2013; Scheduled then extended. 2014: Scheduled comment period closure: February 10, 2014; Scheduled then extended. Projected OMB review: July 25, 2014. Projected OMB clearance: October 24, 2014. Projected SNPRM publication: October 31, 2014. Projected comment period closure: December 31, 2014. Abbreviations: NPRM: Notice of Proposed Rulemaking; SNPRM: Supplemental Notice of Proposed Rulemaking; OMB: Office of Management and Budget; OST: Office of the Secretary of Transportation. Source: GAO analysis of DOT information. GAO-14-516. [End of figure] Repair Stations: Few repair stations are currently implementing SMS, and FAA has not yet determined if it will conduct rulemaking to require SMS at repair stations. As of March 2014, just 15 repair stations (of more than 4,700) were implementing SMS through the Flight Standards SMS pilot project, according to an FAA newsletter. As it does with air carriers, Flight Standards uses the four SMS implementation levels to benchmark and determine the progress of repair station SMS implementation. [Footnote 26] Thirteen were at SMS level one or two. Our interviews with 5 repair stations that participated in the pilot program suggested that there may be fewer than 14 repair stations currently implementing SMS because representatives we interviewed from 4 of the repair stations indicated that they had either ceased or never actually initiated their SMS efforts. Among the reasons these repair station representatives cited for not moving forward included a lack of FAA support. For example, representatives from one repair station told us that FAA did not respond to their questions on SMS policy changes and did not provide feedback on the gap analysis they performed. FAA officials indicated that there are reasons that more repair stations may want to implement SMS. Flight Standards officials said that once the commercial air carrier SMS rule is finalized, repair stations might want to implement their own SMS so that it could be recognized by the commercial air carriers they may contract with. A senior FAA official noted that because the commercial air carrier SMS rule will include identifying risks posed by repair stations, it would have an impact on repair station operations. Additionally, as other countries complete SMS rulemaking, some repair stations may want to implement SMS for international business reasons. Design and Manufacturing Firms: Several design and manufacturing firms were implementing SMS as of April 2014, while an FAA rulemaking committee studies SMS incorporation into the design and manufacturing environment of approximately 3,000 firms. Aircraft Certification conducted an SMS pilot project for design and manufacturing firms from 2010 to 2012 with 11 participants. The participants were selected to represent the diversity of the industry in terms of size and types of products designed and/or manufactured. According to an FAA official, approximately 6 pilot project participants were continuing SMS implementation as of April 2014. In our discussions with several design and manufacturing firms that participated in the pilot, we found that 2--Honeywell and Boeing--were planning to complete SMS implementation in 2015. Honeywell representatives indicated that they expect to complete SMS implementation by mid-2015. Boeing representatives said their company was implementing an SMS as defined to ICAO standards and also expect to complete SMS implementation by mid-2015. In October 2012, FAA chartered a 2-year aviation rulemaking committee (ARC) to, among other things, study SMS incorporation into the design and manufacturing firm environment. According to a January 2014 FAA plan,[Footnote 27] once the committee has completed its work, FAA will begin a rulemaking process that will result in a change to the current rule for certificating design and manufacturing firms. The plan anticipates the release of an NPRM by January 2016 and publication of a revised final rule by June 2017 to require, among other things, design and manufacturing firms to implement SMS. However, an FAA official stated in April 2014 that FAA had not yet determined when an SMS requirement might be proposed for design and manufacturing firms. FAA and Stakeholders Cited Rulemaking, Data Issues, and Stakeholder Uncertainty as Key Challenges to Implementing SMS: FAA Cited Challenges in Completing SMS Rulemaking: FAA officials stated that a primary challenge in developing SMS rules for commercial air carriers and certificated airports is developing the benefit-cost analyses. FAA officials explained that estimating the expected benefits for SMS rules for air carriers and airports relies on analysis of the economic cost of past accidents that might not have occurred had SMS been in place. However, determining which past accidents might not have occurred based on SMS implementation either by air carriers or airports or both is complex and somewhat subjective, and this determination can be affected by other safety improvements that have occurred in recent years or that are being proposed. For example, if subsequent to a particular accident, some other safety improvements have been implemented that would have reduced its likelihood, only a portion of the value of the damage and harm related to that accident can be included in the current benefit calculation for future SMS since the likelihood of the same accident occurring has already been somewhat mitigated. Additionally, an airport industry trade organization expressed concerns in its public comments on the airport NPRM that the proposed rule incorrectly assumed that certain accidents would be mitigated by an airport SMS because those occurrences happened on airport grounds but in areas controlled by air carriers. The trade organization reviewed 53 of the 89 accidents included in the FAA analysis. In the trade organization's view, 37 of these 53 accidents had little or limited connection to safety actions that an airport might take within its SMS. The likelihood of the accident or incident occurring could therefore be mitigated by the air carrier's SMS implementation. Some accidents might also be avoided based on joint implementation of SMS systems by both the air carriers and airports. In such cases, expected benefits have to be apportioned across the benefit-cost analyses of the two separate SMS systems--that is, the SMS for air carriers and the SMS for airports--otherwise expected benefits would be double counted. In addition, a number of comments on the proposed rules for both the air carrier and airports questioned the estimated costs imposed on aviation stakeholders related to the implementation of the SMS systems. FAA, as part of its rulemaking process, revisited the benefit- cost analysis after reviewing public comments on the proposed rules. Another challenge FAA faces in implementing SMS rules is planning for how data collected for proactive safety efforts will be used. Over the years, we have identified numerous challenges related to FAA data and made a number of recommendations, some of which FAA has addressed, to improve the use of data that FAA collects. One recommendation of particular significance to SMS implementation, however, has yet to be fully implemented. In 2010, we recommended that FAA develop a comprehensive plan that addresses how data fits into FAA's implementation of a proactive approach to safety oversight. We indicated that the plan should fully describe the relevant data challenges, analytical approaches to be used, staffing requirements, and the efforts to address them.[Footnote 28] Because SMS relies on data for maximum benefit, we concluded that a comprehensive data plan by FAA would help expand its capability to improve aviation oversight. DOT concurred with this recommendation, and although FAA has taken actions such as implementing data quality controls consistent with our standards for data quality, it has not yet completed the recommended plan. For example, it has not outlined how it will analyze or use data gathered under SMS, assessed the staffing requirements it will need to conduct the data analysis, and indicated a timeline to complete a plan. GAO believes this recommendation remains valid. Aviation Stakeholders Cited Uncertainty over FAA Oversight, Implementation Costs, Data Protection Issues, and Final Rule Requirements among Their Key Challenges: SMS implementation challenges have been identified during FAA's SMS pilot projects and the rulemaking processes for commercial air carriers and certificated airports. We also identified some of these challenges in our prior SMS work.[Footnote 29] We selected five frequently cited challenges to discuss with aviation stakeholders: data sharing and protection, resource constraints, implementation costs, FAA oversight, and collaboration with other aviation stakeholders. Our discussions with 20 selected aviation stakeholders that participated in at least one FAA SMS pilot project confirmed that these challenges still exist for some stakeholders.[Footnote 30] Based on these discussions, we found that a majority of the selected stakeholders identified uncertainties about FAA's oversight plans (17 stakeholders); uncertainties about SMS implementation costs (14 stakeholders); and data protection (16 stakeholders) as challenges. In addition, we found that uncertainty about the FAA's final rule requirements raised some additional concerns with the stakeholders we interviewed, and half of the stakeholders said that of the four SMS components, the safety risk management is the most challenging to implement because of the time and analyses required to identify and mitigate risks. Uncertainty about FAA's SMS Oversight Plans: Seventeen stakeholders identified their uncertainty about FAA's plans and preparation for oversight of industry's SMS implementation as a challenge, with 12 identifying it as a great or very great challenge. Specifically, they reported a lack of information about FAA's intended oversight plans, training for FAA inspectors responsible for oversight, and the potential for inconsistent regulatory interpretation because FAA has not shared that information with stakeholders. Just as the move to SMS will change how stakeholders manage risk, FAA's oversight approach will also need to undergo some changes. (See table 2.) According to FAA, while its inspectors will continue to check for compliance with safety regulations and retain enforcement authority, oversight will also include reviewing the SMS guidance and records and ensuring that processes for identifying and mitigating risks are being carried out. Table 2: Changes in Stakeholder and FAA Inspector Roles under a Safety Management System: Role: Aviation Stakeholder; Current safety approach: The stakeholder responds to regulatory requirements and builds compliance with regulations into its work procedures and safety programs; Safety management system approach: The stakeholder proactively manages risk, while maintaining compliance with regulatory requirements. The stakeholder looks beyond compliance to identify potential risks that may or may not be covered by regulation, assesses the risks, and if necessary takes actions to mitigate those risks to create a safer operating environment. Role: FAA Inspector; Current safety approach: Inspectors review stakeholders' actions to ensure they comply with applicable regulations; Safety management system approach: Inspectors evaluate if the risk management programs the stakeholder has in place are being utilized. As necessary, they also conduct traditional inspections to check for regulatory compliance. Source: GAO analysis of information provided by FAA and Transport Canada. [End of table] Internal control standards,[Footnote 31] an FAA rulemaking committee, and the ICAO Safety Management Manual all state the need to establish plans to ensure that goals and objectives are met. Additionally, we have previously found that communicating internal control efforts on a timely basis to internal and external stakeholders, such as FAA inspectors and aviation industry stakeholders, helps ensure that effective oversight can take place. A 2009 SMS aviation rulemaking committee recommended that FAA ensure that sufficient planning, policy and guidance, and workforce training are in place prior to SMS implementation to accommodate efficient, timely, objective and consistent oversight. Internal control standards also state that personnel should be provided with the right training and tools to perform their duties, and the ICAO Safety Management Manual notes the significance of training to successful SMS implementation and indicates that priority for training must be given to personnel involved in implementation or oversight of SMS. While acknowledging that its oversight role will be changing, FAA has not yet completed plans for overseeing industry's implementation of SMS, including plans to develop guidance and train inspectors about their SMS oversight functions and responsibilities. Nine of the aviation industry stakeholders and three industry trade groups we interviewed expressed concerns that FAA inspectors will not be prepared to oversee industry implementation of SMS. These concerns related primarily to a lack of SMS training for FAA inspectors. FAA officials told us that they plan to finalize SMS rules before addressing oversight issues. Based on the NPRM, commercial air carriers would have 3 years after the effective date of the final rule to comply with SMS implementation requirements. When the final rule for commercial air carriers becomes effective, FAA inspectors will assume oversight responsibility to ensure that commercial air carriers comply with the final rule. Included with this responsibility is approving commercial air carriers' SMS plans. According to Flight Standards data, as of November 2013, over 2,700 of approximately 3,900 of their inspectors had completed initial training in SMS activities.[Footnote 32] This includes inspectors who oversee commercial air carriers. However, even though the publication of the final SMS rule for air carriers is expected before the end of the year--currently planned for September 2014--Flight Standards has not yet established guidance for inspectors for overseeing commercial air carriers' implementation of SMS or updated its inspector training program to incorporate such guidance. In addition, even though a large number of inspectors have received initial SMS training, stakeholders remain concerned about inspectors' knowledge of SMS as well as the potential for inconsistent interpretation of SMS requirements. At the same time, the 77 commercial air carriers that have been voluntarily implementing SMS will need to review their SMS to ensure compliance with the final rule and may request FAA guidance during that process to determine whether and what modifications need to be made. Given that most commercial air carriers already have started implementing SMS, the demand for inspector guidance or assistance may come quickly. For example, inspectors might be asked to review a commercial air carrier's SMS to ensure that all required components are included. Without taking proactive preparations for oversight before requiring SMS, FAA could find itself with oversight responsibilities for which it is not fully prepared. Consequently, the intended benefits of implementing this proactive approach to managing risk could be limited. For other industry segments, notably airports and design and manufacturing firms, that are likely to be next to face an SMS requirement, FAA has taken initial steps to train inspectors working in those areas. In 2008, the Airports Organization provided 3 days of basic SMS instruction to airport safety inspectors during the organization's annual recurrent training. Officials in the Airports Organization acknowledged that, should FAA finalize an SMS rule for certificated airports, up-to-date training will need to be developed and delivered to the approximately 40 airport safety inspectors, including training in detailed inspection procedures for oversight of an airport's SMS. Likewise, Aircraft Certification plans for some of its staff to review the Flight Standards initial SMS training course to determine what type of SMS training is necessary for some of its estimated 950 aircraft certification engineers and inspectors. In addition to training, six aviation stakeholders we interviewed expressed concerns about potentially inconsistent regulatory and other interpretations by FAA inspectors. This concern was identified by smaller stakeholders we interviewed as well as those that hold multiple certificates overseen by different FAA organizations. For example, one stakeholder we interviewed indicated that the company holds certificates for multiple facilities in multiple FAA districts, and is concerned that inspectors from different districts may have different interpretations of the SMS rules once they are implemented. We found in 2010 that ensuring consistency in regulatory interpretations has been a long-standing issue for FAA. This was also identified in the SMS ARC final report as an issue that FAA should address prior to SMS implementation. In 2012, FAA formed another aviation rulemaking committee to address consistency in regulatory interpretation by Flight Standards and Aircraft Certification. The committee made six recommendations to improve the consistency of regulatory interpretation and improve communication with industry. In July 2013, FAA provided a preliminary implementation plan to address the recommendations and indicated a detailed implementation plan with milestones would be developed, but provided no time frames for when that detailed plan would be released. Similarly, two stakeholders that operate both domestically and overseas expressed concerns about whether FAA's SMS regulations will be consistent with the ICAO framework that the regulations are designed to meet and harmonized with the SMS requirements of other countries. These stakeholders are concerned that potential inconsistencies could pose problems if different standards exist in different countries. For example, one repair station with overseas operations we spoke to said that any SMS they may be required to implement to meet FAA regulations must also comply with the SMS requirements in other countries where this repair station operates. The need for international acceptance of a service provider's SMS was also identified as an issue for FAA to address in the SMS ARC's final report in 2010. Uncertainty about SMS Implementation Costs: Aviation stakeholders we interviewed largely had not conducted cost assessments of their SMS efforts (only 5 of 20 had done so), and the difficulty in assessing these costs could present a challenge to those considering voluntary implementation of SMS. Fourteen of the 20 stakeholders told us that identifying and assessing SMS costs was a challenge. Further complicating cost estimates, three stakeholders indicated that implementation costs are spread throughout the organization and that isolating those specifically related to SMS would be difficult. For example, one stakeholder indicated that because the computer systems it purchased to support SMS also support other operational objectives, it has not attempted to separate out SMS costs. In addition, seven stakeholders said that the absence of a final rule specifying SMS requirements made assessing costs difficult. Though stakeholders we spoke with were generally unable to determine the cost of implementing SMS, they were able to identify some types of related expenditures. We asked each of them about specific costs related to SMS implementation, and they reported incurring the following types of costs: * 14 trained employees on SMS or SMS concepts, * 10 incurred recordkeeping costs, * 10 purchased computer systems, and: * 7 hired new employees. Other costs identified by stakeholders included those related to safety promotion and mitigating hazards identified through SMS implementation. Concerns about Collecting and Sharing SMS Data: Industry stakeholders we spoke with had concerns about sharing and protecting their safety data; 16 of the 20 stakeholders we interviewed identified this as a challenge, with 7 identifying this challenge as great or very great. In March 2010, the SMS ARC final report identified this as an issue and recommended that prior to the promulgation of an SMS rule, protections be put in place to ensure that safety information and proprietary data are protected from disclosure and use for other purposes, such as enforcement actions. The 2012 FAA Modernization and Reform Act (the 2012 Act) included a provision that placed a limitation on the disclosure of safety information under the federal Freedom of Information Act (FOIA) for information gathered for the purposes of developing and implementing an SMS.[Footnote 33] However, information provided by or to publicly owned airports is also subject to state FOIA laws, which are not covered under the protections in the 2012 Act. Commercial air carriers, repair stations, and design and manufacturing firms are privately owned and not directly subject to state FOIA laws. However, any data airports collect and any data shared with airports could, according to FAA officials and industry experts, be subject to state FOIA laws, because most certificated airports in the U.S. are owned by a public entity such as a state, city, port, or other local or regional government body. According to officials in the Airports Organization, a federal legislative resolution to override state laws is not a feasible option. These officials reported that in some locations, airports have begun to work with state legislative bodies to address the issue of data protection and disclosure. Without legal protections to prevent data disclosure, stakeholders told us they feel at risk in a variety of potential scenarios related to SMS implementation. * Litigation for damage (real or perceived) caused by incidents disclosed through SMS--Eight stakeholders we spoke with, as well as the 2010 SMS ARC, indicated that protections are needed not only to prevent the disclosure of safety data through the state FOIA process, but also from disclosure as part of litigation. For example, a certificated airport must meet the regulatory requirements of 14 C.F.R. Part 139. One standard included in Part 139 is that holes in airport pavement must not exceed 3 inches in depth.[Footnote 34] An airport representative we met with indicated that if a hole less than 3 inches deep is noted through a reporting system implemented under SMS, the airport may postpone that repair until a later time as it is within the maximum depth allowed under the regulations and the risk is acceptable. Additionally, it would not likely negatively impact the operations of a large aircraft on the runway. However, it could pose a hazard for smaller general aviation aircraft. If the airport does not make that repair and the hole factors into an accident involving a general aviation aircraft, the extent of the airport's responsibility and legal liability is uncertain and may become the subject of a lawsuit. Because the intent of SMS is to correct hazardous conditions before an accident occurs, stakeholders want protections that address their vulnerability to potential litigation. * Misinterpretation of safety data by a third party--If safety data is released to the public under a state FOIA, the aviation industry is exposed to potential misinterpretation, misuse, and further dissemination of the data by third parties. For example, one stakeholder expressed concern that safety data could be obtained, reviewed, analyzed, or disseminated by a third party without the proper context, including how and why the data were collected and the limitations of their use. Without the proper context, this stakeholder stated that dissemination would lead to misinterpretation, which may then be further disseminated by others such as policy makers or the media. * Enforcement actions by FAA or other regulatory bodies--As we discussed earlier, the safety risk management component of SMS includes systems that allow individuals to identify potential safety hazards, assess the risks arising from those hazards, and develop mitigation plans to reduce or eliminate those risks. ICAO guidance indicates that in an SMS environment, procedures should be in place to ensure information obtained under SMS will not be used for enforcement actions. However, four stakeholders expressed concern that data about potential hazards could be used beyond SMS. For example, a repair station's SMS would address identified hazards and, if necessary, develop a mitigation plan to address those hazards. However, if an identified hazard also constitutes a regulatory violation, the company might be subject to FAA enforcement action. The SMS ARC recommended in 2010 that FAA should establish a policy or regulation that provides limits on enforcement actions applicable to information that is identified or produced by SMS. Currently, FAA foregoes civil penalty actions when violations are promptly disclosed to FAA under a voluntary disclosure program, subject to some limitations. FAA believes that the open sharing of apparent violations and a cooperative approach to solving problems will enhance and promote aviation safety. Current FAA policy covers commercial air carriers, repair stations, and design and manufacturing firms, but not certificated airports. It is unclear if FAA will extend this coverage to information provided as part of SMS efforts not currently covered. In 2012, we also found the protection of data to be a concern[Footnote 35] and recommended that FAA consider strategies to address concerns that may adversely affect data collection and data sharing--essential to realizing the benefits of SMS. FAA began discussions with airports and airport associations in 2013 to determine what type of SMS data should be considered for protection. FAA also received and considered comments from airports and airport organizations about data collection, protection, and sharing in response to the NPRM for airports. We concluded in our September 2012 report that the success of SMS relies heavily on the sharing of safety data and that without appropriate protections, the willingness of the aviation industry to share safety data will likely be jeopardized. Although protections exist for safety data from commercial air carriers, FAA has determined that a similar legislative approach is not feasible for data from certificated airports. However, according to FAA, some airports are working to address data protections under state laws through outreach to state agencies and legislators. Uncertainty about Final Rule Requirements: Four of the stakeholders we spoke with felt that delays in the rulemaking process were detrimental to their implementation of SMS while others were confident that their SMS implementation would meet forthcoming requirements. Representatives from two industry stakeholders we spoke with told us that they did not want to move forward in implementing SMS until rulemaking is complete, largely because they first want to know what the requirements will be. They explained that they were concerned about the time and resources required to rework the SMS should it not meet the final rule requirements. Fourteen of the stakeholders we interviewed were concerned that the SMS requirement would not be scalable and flexible, that is, one which is sufficient to apply to a broad range of organizations from small operators to large ones with multiple facilities and certifications. Some of the concerns were based on the stakeholders' views that FAA tends to fashion one-size-fits-all regulations or regulations that focus on one subset of a stakeholder group. For example, stakeholders cited a concern that FAA will establish regulatory requirements that are achievable for larger operators but may require more significant efforts by small operators. The NPRM for the commercial air carrier SMS requirement states that the proposed regulation is designed to be performance-based. FAA explained that because the regulation would be performance-based, it would allow commercial air carriers to comply through a variety of methods (suggesting flexibility) and to accommodate a variety of business sizes and models (suggesting scalability). The NPRM for the SMS requirement at certificated airports states that FAA envisions SMS as an adaptable and scalable system. FAA explained that an SMS could be developed by an organization to meet its unique operating environment. Accordingly, FAA stated in the NPRM that it would prescribe only the general framework for an SMS. Several stakeholders also expressed concerns about how SMS regulations would affect their current efforts to improve safety. For example, one design and manufacturing firm we spoke with reported that it began using system safety techniques similar to those used in SMS in the 1990s. Although this stakeholder participated in an FAA pilot, the firm has developed an SMS approach in house based on the ICAO model. Fourteen stakeholders reported that prior to implementing SMS they had undertaken their own proactive safety or quality assurance efforts. FAA's SMS ARC recommended in 2010 that SMS regulations allow for the incorporation of existing safety management efforts already in place in order to prevent duplicative safety efforts. FAA indicated that such programs could be used to build an operational SMS. Other stakeholders expressed concern that the content of the final rule may require them to adopt practices that may not improve safety in their company while imposing additional burdens. For example, one stakeholder hoped it would not be required to document every hazard identified and mitigated, stating that such documentation did not necessarily improve safety. Safety Risk Management Challenge: We also asked all 20 industry stakeholders which of the four components of SMS would be the most challenging to implement, and more stakeholders (10) cited safety risk management than any other component. As previously discussed, safety risk management is designed to examine a company's operational functions and operational environment to identify hazards and analyze associated risks. The intent of this process is to focus on the areas of greatest risk from a safety perspective and on mitigation, taking into account such factors as complexity and operational scope. Specifically, stakeholders indicated that conducting a risk analysis can be time consuming. A robust analysis includes identifying a wide range of risks, and determining which would require mitigation--a process that some stakeholders were concerned may keep stakeholder staff away from other duties. A number of stakeholders also noted the difficulty in identifying risks in some cases, including industry changes that may have an impact on risk. Stakeholders Identified Additional Actions FAA Could Take to Improve SMS Effectiveness and Address Implementation Challenges: Although 12 of the 20 aviation stakeholders we interviewed are continuing with their SMS implementation,[Footnote 36] they identified actions FAA could take to improve SMS effectiveness and help address the challenges of implementation. These suggestions target SMS training, guidance, and collaboration and communication. SMS Training for Industry: Although we did not ask specifically about any internal training programs developed by the 20 aviation stakeholders we spoke with, four stakeholders cited some difficulties with obtaining needed SMS training for their staff, including allocating time and resources for the training and finding training that was specific to their segment of the industry. Further, a 2012 study by the Transportation Research Board of FAA's airport pilot studies found that pilot participants identified time restrictions and funding as obstacles to developing an SMS training plan for airport personnel.[Footnote 37] ICAO recommends that civil aviation authorities, like FAA, facilitate the SMS education or training of its stakeholders where feasible or appropriate. To that end, stakeholders suggested an action that FAA could take to mitigate stakeholder difficulties in this area. Specifically, two stakeholders we interviewed felt that allowing their staffs to attend the same training as FAA personnel could be beneficial in addressing differences in regulatory interpretation and for increasing industry's understanding of FAA's definitions and concepts as they relate to SMS. According to FAA officials, they have considered the training needs of industry stakeholders and provided some training through the pilot projects. SMS Guidance: Although, as required by ICAO, FAA has provided, as well as updated, guidance to help stakeholders to implement SMS, some stakeholders saw a need for additional guidance and updates. For example, one stakeholder noted that FAA should create one risk matrix to be used by all organizations. Two stakeholders suggested standardizing SMS definitions across FAA. Another stakeholder suggested that FAA create a single document on the SMS framework. FAA has already responded to these concerns with an April 2012 order designed to establish common language standards for some SMS terminology; the order also provides a single risk matrix for the agency.[Footnote 38] Stakeholders also suggested that FAA develop additional guidance to aid them in documenting work processes, assessing organizational hazards, and estimating SMS costs. FAA plans to disseminate additional SMS guidance for air carriers and airports when final rules for these industry segments are published. The final rule for air carriers is expected to be published in September 2014. Collaboration and Communication: Fourteen of the 20 stakeholder representatives we interviewed were pleased with their collaboration and communication with FAA during the pilot projects; six stakeholders, however, noted that FAA could take additional actions to improve collaboration and communication in line with ICAO's guidance for SMS. This guidance recommends establishing an appropriate communication platform to facilitate SMS implementation, particularly for SMS requirements and guidance material. FAA has used numerous methods to communicate about and encourage collaboration regarding SMS. These methods include the SMS newsletter, SMS focus groups, participation in the Safety Management International Collaboration Group,[Footnote 39] and the various SMS pilot projects. However, some stakeholders suggested that additional opportunities exist for the agency to share lessons learned. For example, one stakeholder was not aware of any FAA communications on its progress regarding the data protection and sharing issue, and three others thought that FAA needed to encourage the industry to share lessons learned through existing forums. Two stakeholders suggested using the Commercial Aviation Safety Team [Footnote 40] as an example of, and the Aviation Safety Information Analysis and Sharing system[Footnote 41] as a potential tool for, improved SMS collaboration between FAA and commercial air carriers. Two stakeholders also recommended broadening current FAA collaboration and communication efforts by opening membership in SMS focus groups and other working groups to additional industry members, such as foreign air carriers; increasing the number of conferences and meetings where information-sharing can take place; and increasing FAA attendance at these forums. The Airports Organization noted that it continually updates its SMS website, and Flight Standards indicated that it is allowing its most significant forum with the aviation industry, InfoShare, to focus more on SMS implementation and management. Conclusions: Maintaining a high level of safety in the U.S. aviation industry is a shared responsibility among FAA, air carriers, airports, and other stakeholders. FAA continues to implement SMS internally, in accordance with ICAO requirements, and industry stakeholders such as commercial airlines, certificated airports and others have begun to voluntarily adopt SMS in advance of regulations requiring its implementation. The successful implementation of SMS, with its proactive, risk-based approach to maintaining aviation safety, could help ensure the continued safety of the U.S. aviation system. However, the lack of a comprehensive plan for using data, which we recommended in 2010, may limit the effectiveness of SMS once it is implemented. Although FAA has taken steps to implement data quality controls that meet our standards, it has not disclosed its plans for using data collected or to ensure that it has the requisite staffing to make use of the data to be collected from commercial air carriers, certificated airports, and others. We continue to believe that the success of SMS relies on data for maximum benefit and that such a data plan will strengthen FAA's capability to improve aviation oversight. In addition, the SMS ARC recommended in March 2010 that, among other things, FAA must ensure sufficient planning, guidance, and workforce training are in place prior to SMS implementation. Without providing the necessary training and guidance to its inspector workforce, FAA may not be adequately prepared to ensure the benefits of SMS as industry sectors are required to implement it. Although FAA continues to encourage voluntary implementation of SMS, it also must be prepared to exercise its oversight functions, once final rules requiring SMS implementation are in place, to ensure that industry is developing and utilizing processes to identify, document, and mitigate safety risks. This includes providing clarification and guidance to air carriers, airports and others as they develop their own SMS's once final rules for each industry segment are in place. Recommendation for Executive Action: To maximize the effectiveness and potential benefits of SMS implementation, we recommend that the Secretary of Transportation direct the FAA Administrator to take the following action: * Develop a plan to provide oversight of industry implementation of SMS, a plan that includes providing guidance and training to the relevant FAA inspectors by the time final SMS rules for industry sectors (commercial air carriers, certificated airports, repair stations, design and manufacturing firms) are published. Agency Comments: We provided DOT with a draft of this report for review and comment. DOT provided technical corrections and clarifications, which we incorporated as appropriate. We are sending copies of this report to the appropriate congressional committees, the Secretary of Transportation, the FAA Administrator, and other interested parties. In addition, the report is available at no charge on the GAO website at [hyperlink, http://www.gao.gov]. If you or any of your staff members have any questions about this report, please contact me on (202) 512-2834 or at dillinghamg@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix II. Signed by: Gerald L. Dillingham, Ph.D. Director, Physical Infrastructure Issues: [End of section] Appendix I: Objectives, Scope, and Methodology: Our objective was to review the implementation of safety management systems (SMS) in the U.S. aviation industry and provide an update on SMS implementation within the Federal Aviation Administration (FAA). To do so, we addressed the following questions: 1. What is the status of SMS implementation within FAA and for key segments of the aviation industry, including certificated airports, commercial air carriers, repair stations, and design and manufacturing firms? 2. What are the key challenges FAA and the aviation industry face in implementing SMS? 3. What additional actions do aviation stakeholders believe FAA could take to improve SMS implementation and potential effectiveness? To assess the status of SMS implementation within FAA and for key segments of the aviation industry, we reviewed FAA orders and advisory circulars on SMS and reports on its SMS pilot projects for commercial air carriers,[Footnote 42] certificated airports,[Footnote 43] repair stations, and design and manufacturing firms and from the SMS rulemaking projects for certificated airports and commercial air carriers. We also reviewed FAA information on industry SMS implementation status. In addition, we interviewed FAA officials and industry trade group representatives. To update the status of FAA's implementation of SMS, we reviewed FAA guidance, implementation plans, and other documents published or revised since our 2012 report and interviewed FAA officials and trade group representatives. To determine the key challenges FAA and the aviation industry face in implementing SMS as well as additional actions FAA and other stakeholders may take to improve the implementation and potential effectiveness of SMS, we interviewed officials from FAA, FAA employee, and industry groups. To obtain an international perspective on the challenges and additional actions, we interviewed representatives from foreign aviation authorities, specifically the European Aviation Safety Agency, Transport Canada, and the Civil Aviation Safety Authority of Australia. In addition, we also interviewed two Canadian airlines and one Canadian airport that have implemented SMS. Table 3 lists industry and international interviewees. Table 3: Industry and International Interviewees: Group: FAA employee organizations; Interviewee: National Air Traffic Controllers Association. Professional Aviation Safety Specialists. Group: Aviation industry groups; Interviewee: Aeronautical Repair Station Association. Airlines 4 America. American Association of Airport Executives. Regional Airline Association. Group: International entities; Interviewee: Civil Aviation Safety Authority (Australia). European Aviation Safety Agency. Transport Canada. Flair Airlines (Canada). WestJet (Canada). Region of Waterloo International Airport (Canada). Source: GAO. GAO-14-516. [End of table] To obtain the industry stakeholder perspective for each of our objectives, we collected and analyzed information through structured interviews of a total of 20 industry stakeholders--5 airlines, 5 airports, 5 repair stations, and 5 design and manufacturing firms-- that participated in FAA's SMS pilot projects[Footnote 44]. We selected the certificated airports based on their hub size, geographic location, and whether we interviewed them for our prior study. We did not include certificated airports that we interviewed from that study. We chose the commercial air carriers based on carrier type (mainline or regional; cargo or charter), the level of SMS implementation they had reached by the time of our interview, and whether we interviewed them for the prior study; two of the air carriers we selected were interviewed for our 2012 study. For the repair stations, we choose five firms based on level of SMS implementation and the aircraft category (e.g., transport, commuter, or acrobatic) to which they provide services. We chose this selection factor as a result of research completed by the Center for Aviation Research, which found that SMS compliance for repair stations could be based on aircraft category.[Footnote 45] For design and manufacturing firms, we chose five firms based on the extent of SMS implementation, recommendations from industry stakeholders, and types of products they design or manufacture, or both. We conducted all stakeholder interviews with a standardized data collection instrument to maintain consistency across the interviews. To ensure that our interview questions were clear and reliable, we conducted two pretests with knowledgeable individuals and refined the interview questions based on those results. Because these 20 stakeholders comprise a non-representative sample, the results from these interviews cannot be projected to the universe of these industry segments. Table 4 lists these selected industry stakeholders. Table 4: Aviation Industry Stakeholder Interviewees: Industry segment: Commercial air carriers; Stakeholder: Miami Air. SkyWest Airlines. Southwest Airlines[A]. United Airlines[B]. United Parcel Service. Industry segment: Certificated airports; Stakeholder: Cheyenne Regional/Jerry Olson Field. Dubuque Regional. Hartsfield-Jackson Atlanta International. Kona International at Keahole. San Antonio International. Industry segment: Design and manufacturing firms; Stakeholder: Airmotive. Boeing Commercial Airplanes. Gulfstream. Honeywell. Jamco America. Industry segment: Repair stations; Stakeholder: Able Engineering. Aircraft Ducting Repair Inc. Bombardier WV. KCI Aviation. Pratt & Whitney (Bridgeport, WV). Source: GAO. GAO-14-516. [A] Airtran merged with Southwest in 2012. [B] Continental Airlines merged with United in 2010. [End of table] We conducted this performance audit from May 2013 to June 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions, based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Appendix II: GAO Contact and Staff Acknowledgments: GAO Contact: Gerald L. Dillingham, Ph.D., (202) 512-2834, or dillinghamg@gao.gov: Staff Acknowledgments: In addition to the contact named above, Heather MacLeod (Assistant Director), Amy Abramowitz, James Geibel, David Hooper, Christopher Jones, Delwen Jones, Brooke Leary, Josh Ormond, Pamela Vines, and Elizabeth Wood made key contributions to this report. [End of section] Related GAO Products: Aviation: Status of DOT's Actions to Address the Future of Aviation Advisory Committee's Recommendations. [hyperlink, http://www.gao.gov/products/GAO-13-657]. Washington, D.C.: July 25, 2013. Aviation Safety: Status of Recommendations to Improve FAA's Certification and Approval Processes. [hyperlink, http://www.gao.gov/products/GAO-14-142T]. Washington, D.C.: October 30, 2013. Department of Transportation: Key Issues and Management Challenges, 2013. [hyperlink, http://www.gao.gov/products/GAO-13-402T]. Washington, D.C.: March 14, 2013. General Aviation Safety: Additional FAA Efforts Could Help Identify and Mitigate Safety Risk. [hyperlink, http://www.gao.gov/products/GAO-13-36]. Washington, D.C.: October 4, 2012. Aviation Safety: Additional FAA Efforts Could Enhance Safety Risk Management. [hyperlink, http://www.gao.gov/products/GAO-12-898]. Washington, D.C.: September 12, 2012. Aviation Safety: FAA Is Taking Steps to Improve Data, but Challenges for Managing Safety Risks Remain. [hyperlink, http://www.gao.gov/products/GAO-12-660T]. Washington, D.C.: April 25, 2012. Aviation Safety: Enhanced Oversight and Improved Availability of Risk- Based Data Could Further Improve Safety. [hyperlink, http://www.gao.gov/products/GAO-12-24]. Washington, D.C.: October 5, 2011. Aviation Safety: Certification and Approval Processes Are Generally Viewed as Working Well, but Better Evaluative Information Needed to Improve Efficiency. [hyperlink, http://www.gao.gov/products/GAO-11-14]. Washington, D.C.: October 7, 2010. Aviation Safety: Improved Data Quality and Analysis Capabilities Are Needed as FAA Plans a Risk-Based Approach to Safety Oversight. [hyperlink, http://www.gao.gov/products/GAO-10-414]. Washington, D.C.: May 6, 2010. [End of section] Footnotes: [1] ICAO is an agency of the United Nations that promotes the safe and orderly development of international civil aviation worldwide. ICAO sets standards and regulations necessary for aviation safety, security, and efficiency and serves as a forum for collaboration among its 191 member states. [2] ICAO refers to SMS as employed by civil aviation authorities as "state safety programs." [3] FAA issues operating certificates under 14 C.F.R. Part 121 to air carriers offering scheduled, commercial air carrier service on airplanes with a capacity of 10 or more passengers or a maximum payload capacity of more than 7,500 pounds. For purposes of this report, we will refer to these as commercial air carriers. FAA issues operating certificates to airports that (1) service unscheduled air carrier aircraft with more than 30 seats; or (2) serve scheduled air carrier operations with more than 9 seats under 14 C.F.R Part 139.1(a)(1) and (2). For purposes of this report we will refer to these as certificated airports. According to the 2013-2017 National Plan of Integrated Airport Systems there are 545 part 139 certificated airports in the United States. FAA issues type certificates to design and manufacturing firms under 14 C.F.R. Part 21 for the design and manufacturing of aircraft, engines, propellers, and other aircraft parts. FAA issues certificates to repair stations under 14 C.F.R. Part 145. [4] GAO, Aviation Safety: Additional FAA Efforts Could Enhance Safety Risk Management, [hyperlink, http://www.gao.gov/products/GAO-12-898] (Washington D.C.: Sept. 12, 2012). [5] NTSB Recommendation A-07-010. [6] NTSB's Most Wanted List highlights safety issues identified from NTSB accident investigations to increase awareness about the issues and recommended safety solutions. [7] The Office of NextGen was originally a part of the Air Traffic Organization. NextGen is a long-term initiative to incrementally modernize and transform the national airspace system. The objective of the NextGen initiative is to transform the current radar-based system to one centered on satellite-based navigation, automated position reporting, and digital communications. [8] FAA's Aircraft Certification Service is responsible for administering safety standards governing the design, production, and airworthiness of civil aeronautical products; overseeing design, production, and airworthiness certification programs to ensure compliance with prescribed safety standards; providing a safety performance management system to ensure continued operational safety of aircraft; and working with aviation authorities, manufacturers, and other stakeholders to help them successfully improve the safety of the international air transportation system. [9] FAA's Flight Standards Service sets standards for certification and provides oversight of airmen, air operators, air agencies, and designees. It is also responsible for accomplishing certification, inspection, surveillance, investigation, and enforcement; setting regulations and standards; and managing the system for registration of civil aircraft and all airmen records. [10] 74 Fed. Reg. 36414 (July 23, 2009). [11] 76 Fed. Reg. 14592 (Mar. 17, 2011). [12] Airline Safety and Federal Aviation Administration Extension Act of 2010, Pub. L. No. 111-216, § 215, 124 Stat 2348, 2366 (The 2010 Act). [13] 75 Fed. Reg. 68224 (Nov. 5, 2010). [14] 75 Fed. Reg. 62008 (Oct. 7, 2010). [15] The Air Traffic Organization began SMS implementation in 2005, much earlier than other FAA organizations. [16] Other components of Aviation Safety are also implementing SMS, but we limited our review to Flight Standards and Aircraft Certification because they are responsible for oversight of commercial air carriers, repair stations, and design and manufacturing firms. [17] FAA, 2009-20013 Flight Plan (Washington, D.C.) [18] FAA Order 8000.369A. National Policy: Safety Management System, May 8, 2013. [19] This report does not address SMS implementation at other types of air carriers, such as commuter airline operators. [20] Air carriers that have entered the final continuous improvement SMS level include American, American Eagle, Compass, Delta, Hawaiian, Miami Air, PSA, United, and US Airways. [21] Pub. L. No. 111-216, § 215, 124 Stat. 2348. 2366 (2010). [22] This NPRM would amend Title 14 of the Code of Federal Regulations (CFR) by adding a new Part 5. [23] We did not evaluate DOT's review process for rulemaking or FAA's benefit-cost analysis because we determined those issues to be outside the scope of our study. Benefit-cost analysis is required under Executive Order 12866 of September 30, 1993 for significant regulatory action. The Order defines significant regulatory action as "any regulatory action that is likely to result in a rule that may: (1) have an annual effect on the economy of $100 million or more or adversely affect in a material way the economy, a sector of the economy, productivity, competition, jobs, the environment, public health or safety, or State, local or Tribal governments or communities; (2) create a serious inconsistency or otherwise interfere with an action taken or planned by another agency; (3) materially alter the budgetary impact of entitlements, grants, user fees or loan programs or the rights and obligations of recipients thereof; or (4) raise novel legal or policy issues arising out of legal mandates, the President's priorities, or the principles set forth in the Executive Order. [24] The first pilot project, which included 22 airports, focused on airport creation of SMS manuals and implementation plans; the second pilot project was a proof of concept and involved 3 airports; and the third pilot focused on SMS implementation at 14 of the airports that had participated in one or both of the first two pilots. [25] Program guidance letter 13-06, Safety Management Systems (SMS). Eligible airports include airports certificated under 14 C.F.R. Part 139 that are also included in the National Plan of Integrated Airport Systems (NPIAS). The NPIAS identifies existing and proposed airports that are significant to national air transportation and thus eligible to receive federal grants under the Airport Improvement Program. [26] FAA does not track SMS implementation by repair stations that do not participate in the pilot project. [27] FAA, Detailed Implementation Plan for the Federal Aviation Administration Modernization and Reform Act of 2012, Pub. L. No. 112- 95, § 312, January 31, 2014. [28] GAO, Aviation Safety: Improved Data Quality and Analysis Capabilities Are Needed as FAA Plans a Risk-Based Approach to Safety Oversight, [hyperlink, http://www.gao.gov/products/GAO-10-414] (Washington, D.C.: May 6, 2010). [29] [hyperlink, http://www.gao.gov/products/GAO-12-898]. [30] We made a non-statistical selection of 20 aviation stakeholders that participated in FAA's SMS pilot projects--five commercial air carriers, five certificated airports, five design and manufacturing firms, and five repair stations. We asked each of the 20 stakeholders to characterize the five selected challenge on the following five point scale; "not a challenge", "slight challenge," moderate challenge," "a great challenge," and "a very great challenge." [31] GAO, Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.: November 1999). [32] According to a Flight Standards official, not all inspectors are required to complete SMS training because their current duties may not include commercial air carrier oversight. [33] Pub. L. No. 112-95 § 310, 126 Stat. 11, 65 (2012). [34] 14 C.F.R. § 139.305(a)(2). [35] [hyperlink, http://www.gao.gov/products/GAO-12-898]. [36] One stakeholder representative indicated that despite being selected for an SMS pilot project, the stakeholder never began SMS implementation because FAA did not follow up with them after an initial meeting. [37] Transportation Research Board, Airport Cooperative Research Program, ACRP Synthesis 37, Lessons Learned from Airport Safety Management Systems Pilot Studies, Washington, D.C., 2012. [38] FAA Order 8040.4A, FAA National Policy: Safety Risk Management, April 30, 2012. [39] The Safety Management International Collaboration Group was founded by FAA, the European Aviation Safety Agency, and Transport Canada Civil Aviation. It is a joint effort among many regulatory authorities to promote a common understanding of safety management principles and requirements and facilitating their implementation across the international aviation community. [40] The Commercial Aviation Safety Team is a joint government- industry effort to reduce the commercial aviation fatality risk in the United States using an integrated, data-driven strategy. According to the team, its work--along with new aircraft, regulations, and other activities--reduced the commercial aviation fatal accident rate by 83 percent from 1998 to 2008 and is an important aspect of FAA's efforts to improve aviation safety by sharing and analyzing data. [41] Initiated in 2007, FAA's Aviation Safety Information Analysis and Sharing system is a collaborative government-industry information sharing and analysis initiative that aids in the monitoring and identification of potential safety issues. The system features data from a wide variety of data sources, from both public (non- confidential) and protected proprietary (confidential) aviation data. [42] These are defined as air carrier operations involving airplanes with a capacity of 10 or more passengers or a maximum payload capacity of more than 7,500 pounds for which FAA has issued operating certificates under 14 C.F.R. Part 121. [43] These are defined as airports that (1) service unscheduled air carrier aircraft with more than 30 seats or (2) serve scheduled air carrier operations with more than 9 seats and for which FAA has issued operating certificates under 14 C.F.R. Part 139.1(a)(1) and (2). [44] One repair station we selected indicated that although FAA selected them to participate in the pilot, it did not begin the pilot project. However, it has implemented safety and quality assurance programs. [45] Damon Lercel, Ph.D., Center for Aviation Safety Research, Parks College of Engineering, Aviation, and Technology, Saint Louis University, SMS Repair Station Applicability Survey: Initial Results, 2013. [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO's actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO's website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Katherine Siggerud, Managing Director, siggerudk@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, DC 20548. [End of document]