This is the accessible text file for GAO report number GAO-12-24 
entitled 'Aviation Safety: Enhanced Oversight and Improved 
Availability of Risk-Based Data Could Further Improve Safety' which 
was released on October 13, 2011. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as 
part of a longer term project to improve GAO products' accessibility. 
Every attempt has been made to maintain the structural and data 
integrity of the original printed product. Accessibility features, 
such as text descriptions of tables, consecutively numbered footnotes 
placed at the end of the file, and the text of agency comment letters, 
are provided but may not exactly duplicate the presentation or format 
of the printed version. The portable document format (PDF) file is an 
exact electronic replica of the printed version. We welcome your 
feedback. Please E-mail your comments regarding the contents or 
accessibility features of this document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Report to Congressional Committees: 

October 2011: 

Aviation Safety: 

Enhanced Oversight and Improved Availability of Risk-Based Data Could 
Further Improve Safety: 

GAO-12-24: 

GAO Highlights: 

Highlights of GAO-12-24, a report to congressional committees. 

Why GAO Did This Study: 

Takeoffs, landings, and movement around the surface areas of airports 
(the terminal area) are critical to the safe and efficient movement of 
air traffic. The nation’s aviation system is arguably the safest in 
the world, but close calls involving aircraft or other vehicles at or 
near airports are common, occurring almost daily. The Federal Aviation 
Administration (FAA) provides oversight of the terminal area and has 
taken action to improve safety, but has been called upon by the 
National Transportation Safety Board (NTSB) and others to take 
additional steps to improve its oversight. 

As requested, this report addresses (1) recent actions FAA has taken 
to improve safety in the terminal area, (2) recent trends in terminal 
area safety and factors contributing to those trends, and (3) any 
additional actions FAA could take to improve safety in the terminal 
area. To address these issues, GAO analyzed data from FAA data; 
reviewed reports and FAA documents; and interviewed federal and 
industry officials. 

What GAO Found: 

Since 2007, FAA has taken several steps to further improve safety at 
and around airports (see figure), including implementing procedural 
and technological changes to improve runway safety, proposing a rule 
that would require airports to establish risk-management plans that 
include the ramp areas where aircraft are serviced, collecting more 
data on safety incidents, and shifting toward risk-based analysis of 
airborne aviation safety information. Several of these initiatives are 
intended to better identify systemic issues in air traffic safety. 

Figure: The Terminal Area: 

[Refer to PDF for image: illustration] 

Included in the illustration: 
Terminal radar; 
Ramp area; 
Air traffic control tower; 
Taxiway; 
Runway. 

Source: GAO. 

[End of figure] 

Rates of reported safety incidents in the terminal area continue to 
increase. FAA met its interim goals toward reducing the total number 
of runway incursions—the unauthorized presence of an airplane, 
vehicle, or person on the runway—in 2009 and 2010, but the overall 
rate of incursions at towered airports has trended steadily upward. In 
fiscal year 2004, there were 11 incursions per million operations at 
these airports; by fiscal year 2010, the rate increased to 18 
incursions per million operations. The rate and number of airborne 
operational errors—errors made by air traffic controllers—have 
increased considerably in recent years, with the rate nearly doubling 
from the second quarter of fiscal 2008 to the same period of 2011. FAA 
has not met its related performance goals. Comprehensive data are not 
available for some safety incidents, including runway overruns or 
incidents in ramp areas. Recent increases in reported runway 
incursions and airborne operational errors can be somewhat attributed 
to several changes in reporting policies and procedures at FAA; 
however, trends may also indicate an increase in the actual occurrence 
of incidents. 

Enhanced oversight and additional information about surface and 
airborne incidents could help improve safety in the terminal area. FAA 
oversight in the terminal area is currently limited to certain types 
of incidents, notably runway incursions and certain airborne 
incidents, and does not include runway overruns or incidents in ramp 
areas. In addition, the agency lacks data collection processes, risk-
based metrics, and assessment frameworks for analyzing other safety 
incidents such as runway overruns, incidents in ramp areas, or a wider 
range of airborne errors. Further, changes to reporting processes and 
procedures make it difficult to assess safety trends, and existing 
data may not be readily available to decision makers, including those 
at the regional and local levels. As a result, FAA may have difficulty 
assessing recent trends in safety incidents, the risks posed to 
aircraft or passengers in the terminal area, and the impact of the 
agency’s efforts to improve safety. 

What GAO Recommends: 

GAO recommends that FAA (1) extend oversight of terminal area safety 
to include runway overruns and ramp areas, (2) develop risk-based 
measures for runway safety incidents, and (3) improve information 
sharing about incidents. 

The Department of Transportation agreed to consider the 
recommendations and provided clarifying information about efforts made 
to improve runway safety, which GAO incorporated. 

View [hyperlink, http://www.gao.gov/products/GAO-12-24] or key 
components. For more information, contact Gerald L. Dillingham, Ph.D. 
at (202) 512-2834 or dillinghamg@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

FAA Has Taken Actions to Reduce Risk in the Terminal Area: 

Reported Surface and Airborne Incidents Have Increased, and Several 
Key Factors Likely Contribute to Trends: 

Enhanced Oversight and Additional Information about Incidents Could 
Help Improve Safety in the Terminal Area: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Technologies to Improve Runway Safety: 

Appendix III: Technical Appendix: 

Appendix IV: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: Terminal Area Safety Incident Types: 

Table 2: Key GAO and Department of Transportation IG Recommendations 
and FAA Actions, as of Sept. 30, 2011: 

Table 3: Comparison of Top Factors Identified as Contributing to 
Controller Safety Incidents across Programs: 

Table 4: Technologies Intended to Reduce the Number and Severity of 
Runway Incursions or Excursions: 

Table 5: Estimated Change in Runway Incursions after the Installation 
of ASDE-X Airfield Surveillance Technology: 

Figures: 

Figure 1: The Terminal Area: 

Figure 2: Runway Incidents: Incursions and Excursions: 

Figure 3: FAA Standards for Airborne Separation: 

Figure 4: Key FAA Policy Actions for Surface Safety at Airports Since 
2007: 

Figure 5: Deployment of the ASDE-X Surface Surveillance System at 35 
Airports: 

Figure 6: Key FAA Policy Actions to Increase Data Collection for 
Safety Incidents Involving Air Traffic Controllers: 

Figure 7: Changes in FAA's Threshold for Incident Review: 

Figure 8: Number and Rate of Runway Incursions at Towered Airports, 
Fiscal Years 2001-2010: 

Figure 9: Number of Serious (Category A or B) Runway Incursions at 
Towered Airports, Fiscal Years 2001-2010: 

Figure 10: Distribution of Runway Incursion Types, Fiscal Year 2010: 

Figure 11: Rate of Runway Incursions Involving or Not Involving 
General Aviation per Million Air Traffic Control Tower Operations, 
Fiscal Years 2001-2010: 

Figure 12: Operational Errors in Towers and TRACONS, Third Quarter 
Fiscal Year 2007-Second Quarter Fiscal Year 2011: 

Figure 13: Rates of Operational Errors per 1 Million Operations, Third 
Quarter Fiscal Year 2007-Second Quarter Fiscal Year 2011: 

Figure 14: Operational Errors at Air Traffic Control Towers and at 
TRACONS, Third Quarter Fiscal Year 2007-Second Quarter Fiscal Year 
2011: 

Figure 15: FAA Changes to Reporting Practices and Recent Trends in 
Tower and TRACON Operational Errors: 

Figure 16: Flow of Information into ATSAP is Separate from Other 
Systems FAA Uses to Track Air Traffic Safety Incidents: 

Figure 17: Incidents Reported to ATSAP in 2010 That Are Known to FAA 
versus Those That Are Unknown: 

Figure 18: Change in Reported Runway Incursion Rates by Installation 
of ASDE-X, Fiscal Year 2001-April 2011: 

Abbreviations: 

ACRP: Airport Cooperative Research Program: 

ADS-B: Automatic Dependent Surveillance-Broadcast: 

AIP: Airport Improvement Program: 

AMASS: Airport Movement Area Safety System: 

ASDE-3: Airport Surface Detection Equipment, Model 3: 

ASDE-X: Airport Surface Detection Equipment, Model X: 

ASIAS: Aviation Safety Information Analysis and Sharing: 

ATO: Air Traffic Organization: 

ATQA: Air Traffic Quality Assurance database: 

ATSAP: Air Traffic Safety Action Program: 

CEDAR: Comprehensive Electronic Data Analysis and Reporting: 

Council: Runway Safety Council: 

EMAS: Engineered Materials Arresting System: 

Eurocontrol: European Organization for the Safety of Air Navigation: 

FAA: Federal Aviation Administration: 

FAROS: Final Approach Runway Occupancy Signal: 

Flight Standards: Flight Standards Service: 

ICAO: International Civil Aviation Organization: 

IG: Department of Transportation Inspector General: 

LoSS: Losses of Standard Separation: 

NASA: National Aeronautics and Space Administration: 

NTSB: National Transportation Safety Board: 

OSHA: Occupational Safety and Health Administration: 

OSH Act: Occupational Safety and Health Act: 

RAP: Risk Analysis Process: 

Runway Safety: Office of Runway Safety: 

SRER: System Risk Event Rate: 

TARP: Traffic Analysis and Review Program: 

TCAS: Traffic Collision Avoidance System: 

TRACON: terminal radar approach control: 

[End of section] 

United States Government Accountability Office: 
Washington, DC 20548: 

October 5, 2011: 

The Honorable John D. Rockefeller, IV:
Chairman:
The Honorable Kay Bailey Hutchison:
Ranking Member:
Committee on Commerce, Science, and Transportation:
United States Senate: 

The Honorable John L. Mica:
Chairman:
The Honorable Nick J. Rahall, II:
Ranking Member:
Committee on Transportation and Infrastructure:
House of Representatives: 

The Honorable Thomas E. Petri:
Chairman:
The Honorable Jerry F. Costello:
Ranking Member:
Subcommittee on Aviation:
Committee on Transportation and Infrastructure:
House of Representatives: 

The U.S. airspace system is one of the safest in the world, but 
mishaps and close calls involving aircraft or other vehicles at 
airports or in the airspace around airports are common, occurring 
almost daily. While few of these incidents result in accidents or the 
loss of life, the risk of catastrophic accidents in the terminal area 
remains, particularly during takeoff and landing, which are considered 
the most critical phases of flight given the higher level of risk 
associated with them. In a June 2011 incident at John F. Kennedy 
International Airport in New York, for example, a jumbo jet carrying 
286 passengers and crew almost collided with another jumbo jet, which 
reportedly missed a turn and failed to stop where it should have to 
avoid the occupied runway. The Federal Aviation Administration (FAA) 
focuses its terminal area safety oversight on runways and 
taxiways[Footnote 1] and on aircraft in the air near airports. By 
contrast, safety oversight of operations in ramp areas, which include 
areas of airports where aircraft are prepared for arrival and 
departure, has historically been handled primarily by airlines and 
airports. 

You asked us to look at aviation safety and update our prior work, 
including our 2007 report on runway and ramp safety.[Footnote 2] To do 
so, we addressed the following questions: (1) What actions has FAA 
taken to improve safety in the terminal area since 2007? (2) What are 
the trends in terminal area safety and the factors contributing to 
these trends? and (3) What additional actions could FAA take to 
improve terminal area safety? 

To answer these questions we reviewed our prior work and other reports 
and literature on safety in the terminal area and analyzed FAA, 
National Transportation Safety Board (NTSB), and Occupational Safety 
and Health Administration (OSHA) data on surface and airborne 
incidents in the terminal area. In order to ensure that FAA data were 
sufficiently reliable for our purposes, we reviewed agency 
documentation and interviewed agency officials. We also obtained 
information about data from OSHA and NTSB to assure data reliability. 
We interviewed FAA officials, as well as industry experts and 
representatives from key industry associations. In addition, in order 
to obtain background information on safety in the terminal area, we 
interviewed facility managers and airport officials and observed 
facility operations and the application of key technologies at four 
air traffic control facilities and one airport, which were located 
near GAO offices. (See appendix I for more information on our 
objectives, scope, and methodology.) 

We conducted this performance audit from February 2011 to October 2011 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe the 
evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 

Background: 

The terminal area is the area around an airport extending from the 
airfield or surface to about 10,000 feet vertically and out to about 
40 miles in any direction. The terminal area includes airport surface 
areas such as runways, taxiways, and ramps,[Footnote 3] as well as the 
airspace covered by air traffic control towers--typically within 5 
miles of a towered airport--and by terminal radar approach control 
(TRACON) facilities, which typically handle air traffic to within 
about 40 miles of an airport (see figure 1).[Footnote 4] 

Figure 1: The Terminal Area: 

[Refer to PDF for image: illustration] 

Included in the illustration: 
Terminal radar; 
Ramp area; 
Air traffic control tower; 
Taxiway; 
Runway. 

Source: GAO. 

[End of figure] 

Terminal area safety incidents can occur on the surface at airports or 
in the airspace around them. Surface incidents may threaten the safety 
of aircraft, passengers, and airport workers, among others. Terminal 
area safety incidents that happen on runways and taxiways include 
incursions and excursions. Runway incursions typically involve the 
incorrect presence of an aircraft, vehicle, or person on a runway, 
[Footnote 5] and runway excursions generally occur when an aircraft 
veers off or overruns a runway (see figure 2).[Footnote 6] Ramp 
incidents can involve aircraft or airport vehicles, such as baggage 
carts or ground handling vehicles, as well as airline and airport 
employees and others. 

Figure 2: Runway Incidents: Incursions and Excursions: 

[Refer to PDF for image: illustration] 

Depiction of the following: 

Incursion; 
Excursions: 
Veer-off; 
Overrun. 

Source: GAO. 

[End of figure] 

Airborne safety incidents in the terminal area often involve a loss of 
the minimum required distance between aircraft--as airplanes fly too 
close to each other--or as individual aircraft fly too close to 
terrain or obstructions. These incidents are called "losses of 
separation," because there is a violation of FAA separation standards 
that ensure established distances are maintained between aircraft or 
other obstacles while under the control of air traffic controllers. 
Generally, air traffic controllers must maintain either vertical or 
horizontal separation between aircraft (see figure 3), and losses of 
separation occur when both of these measures are violated, based on 
phase of flight and size of the aircraft.[Footnote 7] 

Figure 3: FAA Standards for Airborne Separation: 

[Refer to PDF for image: illustration] 

The following are depicted: 

Departure and arrival phases of flight: (Under the control of air 
traffic control towers or terminal radar facilities) 
Air traffic control tower: 5 miles; 
Terminal radar: 40 miles. 

Horizontal separation: 3 or more nautical miles laterally; 
or: 
Vertical separation: 1,000 feet vertically. 

En route phases of flight: (Generally above 17,000 feet) 
Horizontal separation: 5 or more nautical miles laterally; 
or: 
Vertical separation: At least 1,000 to 2,000 feet vertically 
(depending on altitude). 

Sources: GAO and FAA. 

[End of figure] 

Safety in the terminal area is a shared responsibility among FAA, 
airlines, pilots, and airports. FAA air traffic controllers oversee 
activity on runways and taxiways, and airlines and airports provide 
primary safety oversight in ramp areas. Several FAA offices have a 
role in terminal area safety including: 

* The Office of Runway Safety (Runway Safety) within the Air Traffic 
Organization (ATO) Safety Office was established in 1999 and leads and 
coordinates the agency's runway safety efforts. Its primary mission is 
to improve runway safety by decreasing the number and severity of 
runway incursions. Runway Safety is responsible for developing a 
national runway safety plan and performance measures for runway safety 
and evaluating the effectiveness of runway safety activities. The 
office currently has an acting director.[Footnote 8] Other FAA 
offices, including the Office of Aviation Safety, the Office of 
Airports, other components of ATO, and regional offices support Runway 
Safety's work to identify hazards and analyze risk. 

* ATO manages air traffic control and develops and maintains runway 
safety technology. 

* The Office of Aviation Safety and Flight Standards Service (Flight 
Standards) within it conduct safety inspections of airlines, audit air 
traffic safety issues, and administer a program to obtain information 
about safety incidents involving pilots. 

* The Office of Airports oversees airport-related safety, including 
airport infrastructure. This includes issuing airport operating 
certificates to commercial service airports, establishing airport 
design and safety standards, and inspecting certificated airports. The 
Office of Airports also provides Airport Improvement Program (AIP) 
grants to airports to help support safety improvements. 

Airlines and airports typically oversee the safety of operations in 
ramp areas.[Footnote 9] Ramp areas are typically small, congested 
areas in which departing and arriving aircraft are serviced by ramp 
workers, who include baggage, catering, and fueling personnel. These 
areas can be dangerous for ground workers and passengers. As noted in 
our 2007 report on runway and ramp safety,[Footnote 10] FAA's 
oversight of ramp areas is generally provided indirectly through its 
certification of airlines and airports.[Footnote 11] Both NTSB and 
OSHA investigate accidents in the ramp area. Thus, NTSB investigates 
ramp accidents--and other accidents involving aircraft--that occur 
from the time any person boards an aircraft with the intention to fly 
until the time the last person disembarks the aircraft, if the 
accident results in serious or fatal injury or substantial aircraft 
damage.[Footnote 12] OSHA can conduct an inspection in response to a 
fatality, injuries, or a complaint, unless it is preempted by an 
exercise of statutory authority by FAA.[Footnote 13] 

FAA collects and analyzes information about various safety incidents 
in the terminal area in order to track incidents, identify their 
causes, and assign severity levels. Currently data are collected at 
towered airports for runway incursions, some other surface incidents, 
and for airborne incidents. By contrast, no complete data are 
collected for incidents in ramp areas. FAA categorizes incidents 
according to the actions or inactions of air traffic controllers, 
pilots, or others, such as pedestrians or vehicle operators. Table 1 
provides hypothetical examples of each type of incident. Depending on 
the type of incident identified--air traffic control surface event, 
operational error or deviation, pilot deviation, or pedestrian/vehicle 
deviation--different offices within FAA are responsible for 
investigating individual incidents.[Footnote 14] 

Table 1: Terminal Area Safety Incident Types: 

Type of incident (surface or airborne): Air traffic control surface 
events (surface); 
Example: An aircraft enters an occupied runway after an air traffic 
controller fails to ensure the pilot repeats instructions correctly, 
resulting in a runway incursion. 

Type of incident (surface or airborne): Pilot deviation (surface); 
Example: A pilot taxis across a departure runway without clearance 
from air traffic control, resulting in a runway incursion. 

Type of incident (surface or airborne): Pilot deviation (airborne); 
Example: A pilot levels off at an incorrect altitude and flies too 
closely to another aircraft, resulting in a loss of separation. 

Type of incident (surface or airborne): Operational error (airborne); 
Example: An air traffic controller does not maintain separation 
standards when sequencing two aircraft on approach to an airport for 
landing, resulting in a loss of separation. 

Type of incident (surface or airborne): Operational deviation 
(airborne); 
Example: An air traffic controller at a TRACON fails to coordinate 
with the tower as an aircraft approaches the airport. 

Type of incident (surface or airborne): Vehicle/pedestrian deviation 
(surface); 
Example: An aircraft fueling truck crosses a runway without 
authorization from air traffic control, resulting in a runway 
incursion. 

Type of incident (surface or airborne): Other airport incidents 
(surface); 
Example: An aircraft slides from a taxiway onto a departure runway as 
the result of an accumulation of snow and ice on the ground, resulting 
in a runway incursion. 

Source: GAO analysis of FAA information. 

[End of table] 

FAA is in the process of implementing a data-driven, risk-based 
approach to safety oversight that FAA expects will help it 
continuously improve safety by identifying hazards, assessing and 
mitigating risk, and measuring performance.[Footnote 15] For decades, 
the aviation industry and FAA have used data reactively to identify 
the causes of aviation accidents and incidents and take actions to 
prevent their recurrence. Using a safety management system approach, 
the agency plans to use aviation safety data to identify conditions 
that could lead to incidents, allowing it to address risks 
proactively. FAA's current approach for analyzing information about 
safety in the terminal area includes separate approaches for surface 
and airborne incidents. 

* Surface incidents. For runway incursions, Runway Safety collects 
information from the Administrator's Daily Bulletin and the Air 
Traffic Quality Assurance database (ATQA)--a mandatory reporting 
system with incident information recorded by FAA air traffic 
controller supervisors, support specialists, and managers--and other 
sources such as incident investigations.[Footnote 16] Runway Safety 
determines how an event will be categorized (e.g., air traffic 
control, pilot, or vehicle/pedestrian deviation, etc.), and the runway 
incursion severity classification team, which consists of 
representatives from the Office of Airports, Flight Standards, and ATO 
Terminal Services, determines the severity of the incursion. 

* Airborne incidents. For terminal area incidents that occur in the 
air, the primary source of information is ATQA.[Footnote 17] FAA 
recently adopted a new process for analyzing these incident data and 
has taken steps to increase the amount and quality of information 
collected. FAA officials stated that, prior to this change, data were 
limited to the information collected in ATQA from FAA managers and 
supervisors, with limited input from individual controllers through 
controller statements gathered during incident investigations. We will 
discuss the new system in more detail in the following section. 

FAA Has Taken Actions to Reduce Risk in the Terminal Area: 

Procedural and Technological Changes to Improve Runway Safety: 

FAA has taken several steps since 2007 to further improve surface 
safety at airports, focusing most notably on efforts to reduce the 
number and severity of runway incursions--the agency's key performance 
measures for this area.[Footnote 18] (See figure 4.) As part of its 
2007 Call to Action Plan, the agency implemented new safety approaches 
and developed milestones for the implementation of various mid-and 
long-term initiatives, such as conducting safety reviews of 20 
airports where incursions were of greatest concern, upgrading airport 
markings at airports, and reviewing cockpit and air traffic 
procedures. Additionally, FAA's 2009-2011 National Runway Safety Plan 
establishes priorities for each FAA office involved in reducing 
incursion risks and identifies performance targets for reducing the 
risk of runway incursions, including an overall goal to reduce total 
runway incursions by 10 percent from 1,009 in fiscal year 2008 to 908 
incursions by the end of fiscal year 2013. In 2010, FAA issued an 
order that further strengthened the role of Runway Safety as the 
agency's focus for addressing incursions and improving runway safety. 
[Footnote 19] FAA has also proposed new rules related to airport 
safety management systems that address ramp areas. 

Figure 4: Key FAA Policy Actions for Surface Safety at Airports Since 
2007: 

[Refer to PDF for image: timeline] 

August 2007: 
FAA “call to action”to improve runway safety. 

August 2007: 
New Director appointed to Office of Runway Safety. 

October 2007: 
ICAO runway incursion definition adopted by FAA. 

2009: 
FAA releases National Plan for Runway Safety. 

September 2010: 
FAA issues Runway Safety Program order. 

October 2010: 
FAA issues Notice of Proposed Rulemaking on airport safety management 
systems. 

Source: GAO. 

[End of figure] 

Additionally, FAA established local and regional runway safety action 
teams that assess runway safety issues at particular airports, 
formulate runway safety action plans to address these concerns, and 
execute their runway safety programs. FAA also established the Runway 
Safety Council (Council) with aviation industry stakeholders to 
develop a systemic approach to improving runway safety.[Footnote 20] 
The Council's Root Cause Analysis Team--comprised of representatives 
from FAA and airlines--investigates severe runway incursions to 
determine root causes in order to identify systemic risks. The Root 
Cause Analysis Team presents recommendations to the Council, which in 
turn, assigns accepted recommendations to FAA or the aviation 
industry, based on which is best able to address root causes and 
prevent further incursions. The Council is responsible for tracking 
recommendations and ensuring that they get implemented. 

FAA's layered approach to addressing runway safety includes a range of 
actions, such as encouraging airport improvements, including improving 
runway safety areas; changes to airport layout and runway markings, 
signage, and lighting; providing training for pilots and air traffic 
controllers; mitigating wildlife hazards; and researching, testing, 
and deploying new technology. According to its 2009-2011 National 
Runway Safety Plan and annual safety reports, FAA's efforts to 
decrease the risk of surface incidents include: 

* Improving runway safety areas. In order to reduce fatalities and 
injuries from runway excursions, the Office of Airports has provided 
between $200 and $300 million annually since 2000 through AIP grants 
to improve runway safety areas, which are unobstructed areas 
surrounding runways. 

* Outreach to general aviation pilots.[Footnote 21] Regional runway 
safety action team meetings, briefings, and clinics for general 
aviation pilots and flight instructors discuss the importance of 
runway safety and how to avoid incursions. FAA also provided training, 
printed materials, and electronic media such as DVDs explaining runway 
safety. 

* New terminology. FAA adopted international air traffic controller- 
pilot terminology for taxi clearance instructions to help avoid 
miscommunication between pilots on the taxiway and runway and air 
traffic controllers. These included new mandatory detailed taxi 
instructions, including directional turns, for all aircraft and 
vehicles taxiing to and from ramps and runways. 

* Upgraded markings. Markings--such as enhanced centerlines drawn on 
taxiways and runways--were installed at all 549 FAA-certificated 
commercial airports in 2010.[Footnote 22] 

* Hot spot identification. Hot spots--locations on runways or taxiways 
with a history of collisions or incursions or the heightened potential 
for such incidents--have been identified on airport diagrams to alert 
pilots of complex locations on runways and taxiways. 

* Airport layout. Some airports have relocated taxiways, allowing 
pilots to avoid crossing active runways during the taxi phase. These 
"end around taxiways" facilitate ground movement and minimize 
conflicts with aircraft operating on runways. The Office of Airports 
has also released guidance on the design of taxiways and aprons to 
help prevent runway incursions. 

* Training. FAA has developed video programs, training modules, and 
best practices for pilots, controllers, and airport personnel aimed at 
heightening awareness of situations that could lead to incursions. FAA 
now also requires that runway incursion prevention be included in 
refresher training for controllers, pilots, and all certificated 
airport employees. 

* Research and development of best practices and other useful 
information. Runway Safety's Web site has resources, best practices, 
and statistics on runway safety. Moreover, Runway Safety has produced 
DVDs and Pilot's Guide brochures, as well as runway incursion safety 
alerts for airport operators. 

* Wildlife Hazard Mitigation. In addition to an active research 
program for developing practical techniques for mitigating bird 
strikes, FAA has encouraged all certificated airports to conduct 
wildlife hazard assessments and is pursuing rulemaking to make it 
mandatory for certificated airports to do so.[Footnote 23] FAA 
currently provides AIP funds to hire qualified wildlife biologists to 
develop assessments and mitigation plans, as needed. 

A number of available technological systems are intended to help 
reduce the number and severity of runway incursions, and FAA has made 
progress installing several of these systems since 2007. For example, 
in order to prevent collisions, FAA completed installation of the 
Airport Surface Detection Equipment, Model X (ASDE-X) system at 35 
major airports, which provides air traffic controllers a visual 
representation of traffic on runways and taxiways (see figure 5). 
Other systems that will provide safety information directly to pilots 
are being installed or tested. For example, runway status lights, an 
automatic series of lights that give pilots a visible warning when 
runways are not clear to enter, cross, or depart on, are planned to be 
installed at 23 airports by August 2016. See appendix II for more 
information on technologies to improve runway safety. 

Figure 5: Deployment of the ASDE-X Surface Surveillance System at 35 
Airports: 

[Refer to PDF for image: illustrated U.S. map] 

Map depicts the location of the 35 ASDE-X deployment sites: 

Hartsfield-Jackson Atlanta International; 
Baltimore Washington International; 
Boston Logan International; 
Bradley International (Hartford, CT); 
Charlotte Douglas International; 
Chicago Midway; 
Chicago O'Hare International; 
Dallas/Ft. Worth International;
Denver International; 
Detroit Metro Wayne County; 
Ft. Lauderdale/Hollywood; 
General Mitchell International (Milwaukee, WI); 
George Bush Intercontinental (Houston, TX); 
Honolulu International-Hickam AFB; 
John F. Kennedy International (New York, NY); 
John Wayne-Orange County (Santa Ana, CA); 
Lambert-St. Louis International; 
Las Vegas McCarran International; 
Los Angeles International;
Louisville International-Standiford Field; 
Memphis International; 
Miami International
Minneapolis-St. Paul International; 
Newark International; 
New York LaGuardia; 
Orlando International; 
Philadelphia International; 
Phoenix Sky Harbor International; 
Ronald Reagan Washington National; 
Salt Lake City International; 
San Diego International; 
Seattle-Tacoma International; 
Theodore Francis Green (Providence, RI); 
Washington Dulles International; 
William P. Hobby (Houston, TX). 

Sources: FAA and Map Resources (map). 

[End of figure] 

To date, runway excursions have not received the same level of 
attention from FAA as incursions. However, excursions can be as 
dangerous as incursions; according to research by the Flight Safety 
Foundation, excursions have resulted in more fatalities than 
incursions globally. FAA is now planning efforts to track and assess 
excursions as well. According to FAA officials, in response to 
recommendations that we and others have made, Runway Safety will begin 
overseeing runway excursions on October 1, 2011. Specific 
responsibilities include collecting and analyzing data to develop 
steps to reduce the risk of such incidents. According to FAA 
officials, the office plans to develop an official definition of an 
excursion, develop a data collection instrument and performance 
metrics that would enable it to collect and evaluate excursion data, 
and develop training and steps to help mitigate excursions.[Footnote 
24] According to a timeline from Runway Safety, it will be several 
years before this program is totally implemented and FAA has detailed 
information about excursions.[Footnote 25] 

FAA recently issued two proposed rules for airports under the agency's 
authority to issue airport operating certificates. The first proposed 
rule, issued in October 2010,[Footnote 26] would require airports to 
establish safety management systems for ramps areas, as well as other 
parts of the airfield, including runways and taxiways.[Footnote 27] As 
previously noted, FAA historically has not primarily overseen safety 
in ramp areas, which are typically controlled by airlines or--to a 
lesser extent--airports using their own practices. FAA's proposal 
would require airports to establish safety management systems for the 
entire airfield environment in order to ensure that: 

* individuals are trained on the safety implications of working on the 
surface of the terminal area; 

* hazards are identified proactively, and analysis systems are in 
place; 

* data analysis, tracking, and reporting systems are available for 
trend analysis and to gain lessons learned; and: 

* there is timely communication of safety issues to all stakeholders. 

A second proposed rule, issued in February 2011, would establish 
minimum training standards for all personnel who access ramp areas. 
[Footnote 28] Required training would occur at least yearly and 
include familiarization with airport markings, signs, and lighting, as 
well as procedures for operating in the nonmovement (ramp) area. The 
public comment period for these proposed rules closed during July 
2011. FAA has not indicated when the rules would be finalized. 

We reported in 2007 that FAA lacked ground handling standards for ramp 
areas.[Footnote 29] In the absence of agency standards, other 
organizations have developed tools to improve ramp safety. For 
example, the Flight Safety Foundation has collected best practices and 
developed a template of standard operating procedures to assist ramp 
supervisors in developing their own procedures.[Footnote 30] The 
guidelines are wide ranging and include the reporting of safety 
information, ramp safety rules, the positioning of equipment and 
safety cones, refueling, and caring for passengers, among other areas. 
In addition to the Flight Safety Foundation guidelines, the 
International Air Transport Association, an international airline 
association, has developed a safety audit program for ground handling 
companies aimed at improving safety and cutting airline costs by 
drastically reducing ground accidents and injuries.[Footnote 31] The 
program is available to all ground service providers, who, after 
successfully completing the audit, are placed on a registry. As of 
August 2011, Seattle-Tacoma International Airport is the only domestic 
airport participating in the program.[Footnote 32] 

Data Collection and Risk-Based Analysis of Airborne Aviation Safety 
Information: 

As part of a planned shift to a risk-based safety management system 
throughout the agency, FAA has increased its data collection on safety 
incidents involving air traffic controllers by taking steps to 
encourage reporting of safety incidents (see figure 6). In particular, 
between 2008 and 2010, ATO implemented a nonpunitive, voluntary safety 
reporting program for air traffic controllers--the Air Traffic Safety 
Action Program (ATSAP)--at 322 facilities. According to officials, the 
intent of ATSAP is to provide air traffic controllers with a method 
through which potential safety hazards can be identified, evaluated, 
and addressed. Modeled on a voluntary safety reporting program that 
various airlines operate for their pilots and other employees, 
[Footnote 33] ATSAP enables air traffic controllers to report losses 
of separation and other safety issues that may help identify potential 
precursors to accidents without fear of punitive action. Voluntary, 
nonpunitive reporting is seen as a key part of safety management 
systems and is advocated by ICAO. According to FAA officials, 250-350 
ATSAP reports are filed weekly, although not all of them are reporting 
safety incidents, such as losses of separation.[Footnote 34] Reporting 
an incident into ATSAP fulfills most reporting requirements for air 
traffic controllers[Footnote 35]--that controllers report any 
potential air traffic incidents that occur while the controller is 
working and are not otherwise reported to FAA[Footnote 36]--and 
provides controllers with some protection from potential punitive 
action on the part of FAA in response to incidents, such as suspension 
or decertification. As of September 2011, the ATSAP program had issued 
more than 60 corrective action requests to address hazards identified 
as a result of its review of reports filed by controllers. 
Furthermore, officials stated that more than 100 additional safety 
concerns have been addressed through informal discussions between 
ATSAP officials and FAA facilities. 

Figure 6: Key FAA Policy Actions to Increase Data Collection for 
Safety Incidents Involving Air Traffic Controllers: 

[Refer to PDF for image: timeline] 

April 2007: 
FAA releases new definition of operational errors. 

July 2008: 
Implementation of ATSAP begins. 

July 2009: 
Information identifying individuals no longer collected for central 
FAA database. 

October 2009: 
FAA launches Risk Analysis Process. 

July 2010: 
FAA changes targets and performance measures for losses of separation. 

September 2010: 
ATSAP fully launched. 

Source: GAO. 

[End of figure] 

In other efforts to obtain more safety data, FAA has taken other steps 
to make incident reporting less punitive. For example, in July 2009, 
FAA changed its incident reporting policy such that individually 
identifying information, such as air traffic controller names and 
performance records, is no longer associated with specific incidents 
in ATQA, the central FAA database used by air traffic control managers 
or supervisors to report incidents. In addition, in July 2010, FAA 
also stopped issuing incident "not to exceed" targets to individual 
air traffic control facilities (e.g., towers, TRACONS, or en route 
facilities).[Footnote 37] According to officials, these targets 
created an incentive for underreporting of less serious incidents by 
supervisors at the facility level, and the targets were discontinued 
in order to encourage increased reporting at the agency. 

FAA is also implementing new technologies, specifically, the Traffic 
Analysis and Review Program (TARP), an error detection system that can 
be used to automatically capture losses of separation that occur while 
aircraft are under the control of air traffic control towers and 
TRACONs. Historically, FAA relied on air traffic controllers and their 
supervisors to manually report on operational errors, something we 
have noted in the past may negatively impact data quality and 
completeness.[Footnote 38] TARP automatically captures data on all 
airborne losses of separation, which, according to officials, will 
increase the volume of data FAA gathers on air traffic safety 
incidents and enable FAA to obtain a more complete picture of 
potential safety hazards. According to the fiscal year 2010 FAA 
Performance and Accountability Report, FAA has deployed TARP at 150 
air traffic control tower and TRACON facilities. According to FAA 
officials, TARP is currently being used as an audit tool for 
approximately 2 hours per month at some facilities--in lieu of full-
time use at all facilities--but further implementation of the system 
has been delayed as the agency evaluates the impact of the system on 
controllers and determines how the system will be used and how to 
handle the additional workload that will be created as more incidents 
are captured and require investigation.[Footnote 39] Following the 
completion of these steps, FAA will take 210 days to fully deploy 
TARP.[Footnote 40] Currently, incidents identified through TARP are 
being included in official incident counts. 

FAA is also shifting to a new, risk-based process for assessing a 
select category of airborne losses of separation. FAA began using the 
Risk Analysis Process (RAP)--which is adapted from a similar process 
used by the European Organization for the Safety of Air Navigation 
(Eurocontrol)[Footnote 41]--in fiscal year 2010. While the new process 
is being established, RAP will be used in tandem with the existing 
system. RAP is currently limited to Losses of Standard Separation 
(LoSS).[Footnote 42] This subset of airborne incidents includes those 
in which the separation maintained is less than 66 percent of the 
minimum separation standard for the planes involved. Under RAP, FAA 
determines both the severity and the repeatability of selected LoSS 
events (that is, how likely a certain LoSS event will occur again at 
any airport under similar circumstances based on a number of factors). 
These factors include proximity of planes to one another at the time 
of the event, rate of closure between planes, controller and pilot 
recovery, and whether or not Traffic Collision Avoidance System (TCAS) 
technology is triggered by the incident.[Footnote 43] Prior to the 
development of RAP, FAA categorized losses of separation based on 
proximity alone: the greater the loss of separation between two 
planes, the greater the severity of the incident in question.[Footnote 
44] Operational errors were then rated on an A-C scale, with those 
that retained more than 90 percent of required minimum separation 
categorized as proximity events.[Footnote 45] (Fig. 7 compares the 
threshold for review of incidents in each system.) Officials stated 
that RAP is more robust than the previous system because it is able to 
take numerous factors into account when determining event severity, as 
well as overall risk to air traffic safety. In addition, the RAP will 
assess risk for LoSS events that were not assigned a severity category 
under the old system. As a result, they said the agency will be better 
equipped to identify systemic issues in air traffic safety and to 
issue related corrective action requests. Based on analysis of 
systemic issues identified across incidents, RAP released its first 
five corrective action requests on July 19, 2011, which were developed 
to mitigate specific hazards that contribute to what RAP has 
identified as the highest risk events. 

Figure 7: Changes in FAA's Threshold for Incident Review: 

[Refer to PDF for image: illustration] 

Started in 2007: 

Assessed (severity): 
Vertical separation retained (percentage): 90%; 
Horizontal separation retained (percentage): 90%; 
Not assessed: 
Vertical separation retained (percentage): 10%; 
Horizontal separation retained (percentage): 10%. 

Started in 2011: 

Assessed (severity): 
Vertical separation retained (percentage): 66%; 
Horizontal separation retained (percentage): 66%; 
Not assessed: 
Vertical separation retained (percentage): 34%; 
Horizontal separation retained (percentage): 34%. 

Source: GAO. 

[End of figure] 

FAA Has Responded to Several Key Recommendations, but Some Areas 
Remain Unaddressed: 

We, the Department of Transportation Inspector General (IG), and NTSB 
have raised concerns about terminal area safety. For example, we 
recommended, in 2007, that FAA take several steps to enhance runway 
and ramp safety, such as updating its national runway safety plan, 
collecting data on runway excursions, and working with OSHA and 
industry to collect and analyze better information on ramp accidents. 
[Footnote 46] In 2007, FAA put in place a Director for Runway Safety 
and issued a Call to Action aimed at reducing the risk of incursions 
following several high-profile incidents (see table 2 for select 
recommendations to FAA). The IG also made recommendations to FAA about 
runway safety issues and recommended that FAA take several steps to 
reduce the risk of airborne incidents and improve oversight of this 
area. For example, the IG recommended that FAA clearly document the 
severity ratings used by FAA for runway incursions, revise the 
national plan for runway safety, and realign the Office of Runway 
Safety.[Footnote 47] With regard to airborne incidents, the IG 
recommended establishing a process to rate the severity of pilot 
deviations and corresponding performance goals, developing milestones 
for implementing TARP, and assuring that Flight Standards works with 
ATO Safety Services to determine whether losses of separation are 
pilot or controller errors, among other recommendations.[Footnote 48] 
Further, NTSB continues to include runway safety, safety management 
systems, and pilot and air traffic controller professionalism issues 
on its list of most wanted safety improvements. 

Table 2: Key GAO and Department of Transportation IG Recommendations 
and FAA Actions, as of Sept. 30, 2011: 

Agency: GAO; 
Selected recommendations: Establish the Office of Runway Safety to 
lead the agency's runway safety efforts, including preparing a new 
national runway safety plan; 
FAA response to recommendations: In 2007, FAA hired a permanent 
Director and in 2010, issued a new Order that enhanced the 
responsibilities of the Office of Runway Safety. FAA also issued the 
2009-2011 National Runway Safety Plan; 
Status: Implemented. 

Agency: GAO; 
Selected recommendations: Develop and implement a plan to collect data 
on runway overruns (excursions) for analyses of trends and causes; 
FAA response to recommendations: In January 2010, FAA's Office of 
Airports issued policy guidance to airport inspectors on how and what 
data to collect for every runway excursion, including overruns; 
Status: Agency: Implemented. 

Agency: GAO; 
Selected recommendations: Work with the aviation industry and OSHA to 
develop a mechanism to collect and analyze data on ramp accidents; 
FAA response to recommendations: FAA's proposed rulemaking covers 
safety in airports; 
Status: Open. 

Agency: IG; 
Selected recommendations: Revise FAA's National Plan for Runway Safety 
to include responsible FAA office, specific milestones, and metrics 
for each initiative; 
FAA response to recommendations: FAA issued the 2009-2011 National 
Runway Safety Plan; 
Status: Implemented. 

Agency: IG; 
Selected recommendations: Realign the Office of Runway Safety outside 
of FAA's operational lines of business, such as directly reporting to 
the Deputy Administrator; 
FAA response to recommendations: FAA believes that the Office of 
Runway Safety has demonstrated effectiveness but will periodically 
review organizational structures and processes; 
Status: Agency: Closed. 

Agency: IG; 
Selected recommendations: Develop milestones for implementing TARP as 
a full-time separation conformance tool; 
FAA response to recommendations: FAA is in the process of testing TARP 
at selected locations; 
Status: Open. 

Agency: IG; 
Selected recommendations: Establish a process to rate the severity of 
pilot deviations that cause a loss of separation and establish a 
corresponding goal to reduce the most severe incidents; 
FAA response to recommendations: FAA believes that its Risk Analysis 
Process begins to address this recommendation, as some losses of 
separation caused by pilots are evaluated under the new system; 
Status: Open. 

Sources: GAO and IG information. 

[End of table] 

Reported Surface and Airborne Incidents Have Increased, and Several 
Key Factors Likely Contribute to Trends: 

Rate of Reported Runway Incursions Has Increased since 2004, but 
Serious Incidents Have Significantly Declined: 

In fiscal years 2009 and 2010, the agency met its interim goals toward 
reducing the total number of runway incursions at towered airports, 
but the rate of incursions per million operations continued to 
increase (see figure 8). As noted in our 2007 report, both the number 
and rate of incursions reached a peak in fiscal year 2001, prompting 
FAA to focus on runway safety. The number and rate of incursions at 
towered airports decreased dramatically for a few years thereafter, 
though the impact of FAA's efforts on these outcomes is uncertain. 
Beginning in 2004, however, both the number and rate of incursions 
began increasing again. For example, in fiscal year 2004, there were 
733 incursions at a rate of 11.4 incursions per 1 million tower 
operations, compared with fiscal year 2010, when there were 966 
incursions at a rate of 17.8 incursions per 1 million such operations. 
Although the rate of incursions at towered airports continues to 
increase, the number of incursions at these airports peaked in fiscal 
year 2008. 

Figure 8: Number and Rate of Runway Incursions at Towered Airports, 
Fiscal Years 2001-2010: 

[Refer to PDF for image: combined vertical bar and line graph] 

Fiscal year: 2001; 
Number of runway incursions: 1,058; 
Rate of runway incursions (per 1 million tower operations): 15.78. 

Fiscal year: 2002; 
Number of runway incursions: 872; 
Rate of runway incursions (per 1 million tower operations): 13.26. 

Fiscal year: 2003; 
Number of runway incursions: 732; 
Rate of runway incursions (per 1 million tower operations): 11.52. 

Fiscal year: 2004; 
Number of runway incursions: 733; 
Rate of runway incursions (per 1 million tower operations): 11.44. 

Fiscal year: 2005; 
Number of runway incursions: 781; 
Rate of runway incursions (per 1 million tower operations): 12.19. 

Fiscal year: 2006; 
Number of runway incursions: 819; 
Rate of runway incursions (per 1 million tower operations): 13.2. 

Fiscal year: 2007; 
Number of runway incursions: 893; 
Rate of runway incursions (per 1 million tower operations): 14.38. 

Fiscal year: 2008; 
Number of runway incursions: 1,009; 
Rate of runway incursions (per 1 million tower operations): 16.7. 

Fiscal year: 2009; 
Number of runway incursions: 951; 
Rate of runway incursions (per 1 million tower operations): 17.03. 

Fiscal year: 2010; 
Number of runway incursions: 966; 
Rate of runway incursions (per 1 million tower operations): 17.83. 

Source: GAO analysis of FAA data. 

[End of figure] 

The most serious runway incursions at towered airports--where 
collisions are narrowly avoided--decreased by a large amount from 
fiscal year 2001 to 2010, and FAA met or exceeded its goals for 
reducing the rate of these incidents.[Footnote 49] FAA classifies the 
severity of runway incursions into four categories--A through D--and 
its performance targets call for the reduction of the most severe 
incursions (category A and B) to a rate of no more than 0.45 per 
million air traffic control tower operations by fiscal year 2010 and 
for the rate to remain at or below that level through fiscal year 
2013. The number of the most severe incidents at towered airports also 
dropped from fiscal year 2001 to 2010. Thus, category A and B 
incursions at these airports decreased from 53 to just 6 during that 
time, with category A incursions decreasing from 20 to 4, and category 
B incursions decreasing from 33 to 2 (see figure 9). 

Figure 9: Number of Serious (Category A or B) Runway Incursions at 
Towered Airports, Fiscal Years 2001-2010: 

[Refer to PDF for image: vertical bar graph] 

Fiscal year: 2001; 
Number of serious incursions (categories A and B): 53. 

Fiscal year: 2002; 
Number of serious incursions (categories A and B): 37. 

Fiscal year: 2003; 
Number of serious incursions (categories A and B): 32. 

Fiscal year: 2004; 
Number of serious incursions (categories A and B): 28. 

Fiscal year: 2005; 
Number of serious incursions (categories A and B): 29. 

Fiscal year: 2006; 
Number of serious incursions (categories A and B): 31. 

Fiscal year: 2007; 
Number of serious incursions (categories A and B): 24. 

Fiscal year: 2008; 
Number of serious incursions (categories A and B): 25. 

Fiscal year: 2009; 
Number of serious incursions (categories A and B): 12. 

Fiscal year: 2010; 
Number of serious incursions (categories A and B): 6. 

Source: GAO analysis of FAA data. 

[End of figure] 

In fiscal year 2010, the majority of incursions at towered airports 
were classified as pilot deviations (65 percent), followed by vehicle/ 
pedestrian deviations (19 percent), and operational errors and 
deviations by air traffic controllers (16 percent) (see figure 10). 
Further, for every year since 2001, pilot deviations comprised the 
majority of runway incursions at these airports, and the proportion 
involving these errors steadily increased from about 55 percent of all 
incursions in fiscal year 2001 to 65 percent in fiscal year 2010. 

Figure 10: Distribution of Runway Incursion Types, Fiscal Year 2010: 

[Refer to PDF for image: pie-chart] 

Air traffic controller errors or deviations: 16%; 
Pedestrian or vehicle deviations: 19%; 
Pilot deviations: 65%. 

Source: GAO analysis of FAA data. 

[End of figure] 

We previously reported that most runway incursions at towered airports 
involved general aviation aircraft and that trend continues. General 
aviation aircraft make up nearly a third of total operations at 
towered airports but have consistently accounted for about 60 percent 
of incidents each year since 2001. More specifically, the rate of 
incursions per million tower operations involving at least one general 
aviation aircraft is higher than the rate of incursions not involving 
general aviation aircraft, and the rate has increased every year since 
fiscal year 2004 (see figure 11). Further, general aviation aircraft 
were involved in over 70 percent of the most serious--category A and 
B--incursions from fiscal year 2001 through the second quarter of 
fiscal year 2011. According to FAA officials, general aviation pilots 
may be more susceptible to incursions and other incidents because of 
their varying degrees of experience and frequency of flying. 
Furthermore, general aviation pilots do not generally undergo the same 
training as commercial airline pilots do. 

Figure 11: Rate of Runway Incursions Involving or Not Involving 
General Aviation per Million Air Traffic Control Tower Operations, 
Fiscal Years 2001-2010: 

[Refer to PDF for image: multiple line graph] 

Rate of incursions (per 1 million operations): 

Fiscal year: 2001; 
Rate of incursions involving at least one general aviation aircraft: 
9.08; 
Rate of incursions not involving general aviation aircraft: 6.7. 

Fiscal year: 2002; 
Rate of incursions involving at least one general aviation aircraft: 8; 
Rate of incursions not involving general aviation aircraft: 5.26. 

Fiscal year: 2003; 
Rate of incursions involving at least one general aviation aircraft: 
6.55; 
Rate of incursions not involving general aviation aircraft: 4.97. 

Fiscal year: 2004; 
Rate of incursions involving at least one general aviation aircraft: 
6.6; 
Rate of incursions not involving general aviation aircraft: 4.84. 

Fiscal year: 2005; 
Rate of incursions involving at least one general aviation aircraft: 
6.93; 
Rate of incursions not involving general aviation aircraft: 5.26. 

Fiscal year: 2006; 
Rate of incursions involving at least one general aviation aircraft: 
7.59; 
Rate of incursions not involving general aviation aircraft: 5.61. 

Fiscal year: 2007; 
Rate of incursions involving at least one general aviation aircraft: 
8.52; 
Rate of incursions not involving general aviation aircraft: 5.86. 

Fiscal year: 2008; 
Rate of incursions involving at least one general aviation aircraft: 
9.63; 
Rate of incursions not involving general aviation aircraft: 7.07. 

Fiscal year: 2009; 
Rate of incursions involving at least one general aviation aircraft: 
10.4; 
Rate of incursions not involving general aviation aircraft: 6.62. 

Fiscal year: 2010; 
Rate of incursions involving at least one general aviation aircraft: 
11.11; 
Rate of incursions not involving general aviation aircraft: 6.72. 

Source: GAO analysis of FAA data. 

[End of figure] 

With regard to runway excursions, our review of NTSB data found that 
general aviation aircraft are also involved in most runway excursions. 
Although FAA does not yet formally collect information on runway 
excursions, NTSB provided us with accident investigation reports on 
493 accidents that involved runway overruns or excursions since 2008. 
Seven of these accidents were fatal, resulting in 14 
fatalities.[Footnote 50] Our review of these reports found that 97 
percent of the accidents involving excursions referred to the 
involvement of at least one general aviation aircraft. 

In our 2007 report, we found that efforts to address the occurrence of 
safety incidents in ramp areas were hindered by the lack of data on 
the nature, extent, and cost of ramp incidents and accidents and by 
the absence of industrywide ground handling standards. As discussed 
above, FAA collects no comprehensive data on incidents in the ramp 
area, and NTSB does not routinely collect data on ramp incidents that 
do not result in injury or aircraft damage.[Footnote 51] Likewise, as 
mentioned above, OSHA, the primary source of ramp fatality data, does 
not collect data on incidents that do not result in at least three 
serious injuries or fatalities. In the ramp area, OSHA data on worker 
fatalities show the number of deaths in the ramp area to have varied 
between 3 and 11 from 2000 to 2010. The rate remained constant--
between 4 and 6 deaths per year--from 2008 to 2010. 

Rate and Number of Reported Airborne Operational Errors Increased 
since 2007, Including the Most Serious Incidents: 

The rate of reported airborne operational errors in the terminal area 
increased considerably in recent years. From the second quarter of 
fiscal year 2008 to the second quarter of fiscal year 2011, the rate 
and number of reported airborne operational errors increased 
significantly. During this time period, the rate of reported airborne 
operational errors in the terminal area nearly doubled, increasing 97 
percent, and the number of reported airborne operational errors 
increased from 220 to 378.[Footnote 52] The rate of incidents began a 
notable climb in the fourth quarter of fiscal year 2009, peaked in the 
second quarter of fiscal year 2010, and remained at rates higher than 
the historical average through the second quarter of 2011 (see figure 
12). FAA officials attributed at least some portion of the spike in 
reported incidents during the second quarter of fiscal year 2010 to 
approximately 150 events that occurred as the result of the 
misinterpretation of an arrival waiver at one TRACON facility. 
[Footnote 53] 

Figure 12: Operational Errors in Towers and TRACONS, Third Quarter 
Fiscal Year 2007-Second Quarter Fiscal Year 2011: 

[Refer to PDF for image: combined vertical bar and line graph] 

Fiscal year/quarter: 2007/3; 
Number of operational errors: 189; 
Rate of operational errors (per million operations): 6.8. 

Fiscal year/quarter: 2007/4; 
Number of operational errors: 194; 
Rate of operational errors (per million operations): 6.9. 

Fiscal year/quarter: 2008/1; 
Number of operational errors: 180; 
Rate of operational errors (per million operations): 7.0. 

Fiscal year/quarter: 2008/2; 
Number of operational errors: 220; 
Rate of operational errors (per million operations): 9.0. 

Fiscal year/quarter: 2008/3; 
Number of operational errors: 252; 
Rate of operational errors (per million operations): 9.5. 

Fiscal year/quarter: 2008/4; 
Number of operational errors: 233; 
Rate of operational errors (per million operations): 8.6. 

Fiscal year/quarter: 2009/1; 
Number of operational errors: 213; 
Rate of operational errors (per million operations): 9.1. 

Fiscal year/quarter: 2009/2; 
Number of operational errors: 173; 
Rate of operational errors (per million operations): 7.7. 

Fiscal year/quarter: 2009/3; 
Number of operational errors: 217; 
Rate of operational errors (per million operations): 8.9. 

Fiscal year/quarter: 2009/4; 
Number of operational errors: 287; 
Rate of operational errors (per million operations): 11.4. 

Fiscal year/quarter: 2010/1; 
Number of operational errors: 270; 
Rate of operational errors (per million operations): 12.1. 

Fiscal year/quarter: 2010/2; 
Number of operational errors: 432; 
Rate of operational errors (per million operations): 20.1. 

Fiscal year/quarter: 2010/3; 
Number of operational errors: 332; 
Rate of operational errors (per million operations): 13.6. 

Fiscal year/quarter: 2010/4; 
Number of operational errors: 366; 
Rate of operational errors (per million operations): 14.7. 

Fiscal year/quarter: 2011/1; 
Number of operational errors: 306; 
Rate of operational errors (per million operations): 13.7. 

Fiscal year/quarter: 2011/2; 
Number of operational errors: 378; 
Rate of operational errors (per million operations): 17.7. 

Source: GAO analysis of FAA data. 

[End of figure] 

While the rate of airborne operational errors has increased over time 
in both the TRACON and tower environments, the rate of errors in the 
TRACON environment has increased more. Between the second quarters of 
fiscal years 2008 and 2011, the rate of operational errors in the 
TRACON environment increased from 8.5 to 22.6 operational errors per 
million air traffic control operations--a 166 percent increase (see 
figure 13). In comparison, operational errors increased by 53 percent 
in the tower environment. 

Figure 13: Rates of Operational Errors per 1 Million Operations, Third 
Quarter Fiscal Year 2007-Second Quarter Fiscal Year 2011: 

[Refer to PDF for image: multiple line graph] 

Rate of operational errors (per million operations): 

Fiscal year/quarter: 2007/3; 
Tower: 6.9; 
TRACON: 6.8. 

Fiscal year/quarter: 2007/4; 
Tower: 5.9; 
TRACON: 8.2. 

Fiscal year/quarter: 2008/1; 
Tower: 6.0; 
TRACON: 8.3. 

Fiscal year/quarter: 2008/2; 
Tower: 9.3; 
TRACON: 8.5. 

Fiscal year/quarter: 2008/3; 
Tower: 8.5; 
TRACON: 10.8. 

Fiscal year/quarter: 2008/4; 
Tower: 8.4; 
TRACON: 8.8. 

Fiscal year/quarter: 2009/1; 
Tower: 7.6; 
TRACON: 11.2. 

Fiscal year/quarter: 2009/2; 
Tower: 7.1; 
TRACON: 8.6. 

Fiscal year/quarter: 2009/3; 
Tower: 6.1; 
TRACON: 12.9. 

Fiscal year/quarter: 2009/4; 
Tower: 8.8; 
TRACON: 15.0. 

Fiscal year/quarter: 2010/1; 
Tower: 8.5; 
TRACON: 17.1. 

Fiscal year/quarter: 2010/2; 
Tower: 10.0; 
TRACON: 34.1. 

Fiscal year/quarter: 2010/3; 
Tower: 10.5; 
TRACON: 17.9. 

Fiscal year/quarter: 2010/4; 
Tower: 10.2; 
TRACON: 20.9. 

Fiscal year/quarter: 2011/1; 
Tower: 11.0; 
TRACON: 17.5. 

Fiscal year/quarter: 2011/2; 
Tower: 14.2; 
TRACON: 22.6. 

Source: GAO analysis of FAA data. 

Note: TRACON rates of operational errors are per 1 million TRACON 
operations, and tower rates of operational errors are per 1 million 
tower operations. 

[End of figure] 

Overall, the rate of the most severe airborne operational errors more 
than doubled between the second quarter of fiscal year 2008 and the 
second quarter of 2011.[Footnote 54] While the least severe (category 
C) incidents are more numerous than the most severe, the most severe 
(category A) incidents increased from 5 in the second quarter of 2008 
to 14 in the second quarter of 2011. In comparison, category C 
operational errors increased by 135 percent, and category B 
operational errors decreased by 5 percent. 

Figure 14: Operational Errors at Air Traffic Control Towers and at 
TRACONS, Third Quarter Fiscal Year 2007-Second Quarter Fiscal Year 
2011: 

[Refer to PDF for image: combined vertical bar and multiple line graph] 

Rate of operational errors (per million operations): 

Fiscal year/quarter: 2007/3; 
Rate of A errors: 0.5; 
Rate of B errors: 1.8; 
Rate of C errors: 2.0; 
Number of operational errors: 189. 

Fiscal year/quarter: 2007/4; 
Rate of A errors: 0.2; 
Rate of B errors: 1.6; 
Rate of C errors: 3.4; 
Number of operational errors: 194. 

Fiscal year/quarter: 2008/1; 
Rate of A errors: 0.1; 
Rate of B errors: 1.9; 
Rate of C errors: 3.2; 
Number of operational errors: 180. 

Fiscal year/quarter: 2008/2; 
Rate of A errors: 0.2; 
Rate of B errors: 2.4; 
Rate of C errors: 2.8; 
Number of operational errors: 220. 

Fiscal year/quarter: 2008/3; 
Rate of A errors: 0.5; 
Rate of B errors: 2.0; 
Rate of C errors: 3.8; 
Number of operational errors: 252. 

Fiscal year/quarter: 2008/4; 
Rate of A errors: 0.2; 
Rate of B errors: 1.8; 
Rate of C errors: 3.6; 
Number of operational errors: 233. 

Fiscal year/quarter: 2009/1; 
Rate of A errors: 0.5; 
Rate of B errors: 2.4; 
Rate of C errors: 4.1; 
Number of operational errors: 213. 

Fiscal year/quarter: 2009/2; 
Rate of A errors: 0.3; 
Rate of B errors: 1.4; 
Rate of C errors: 3.3; 
Number of operational errors: 173. 

Fiscal year/quarter: 2009/3; 
Rate of A errors: 0.3; 
Rate of B errors: 1.7; 
Rate of C errors: 3.7; 
Number of operational errors: 217. 

Fiscal year/quarter: 2009/4; 
Rate of A errors: 0.4; 
Rate of B errors: 2.5; 
Rate of C errors: 4.8; 
Number of operational errors: 287. 

Fiscal year/quarter: 2010/1; 
Rate of A errors: 0.2; 
Rate of B errors: 2.4; 
Rate of C errors: 6.1; 
Number of operational errors: 270. 

Fiscal year/quarter: 2010/2; 
Rate of A errors: 0.4; 
Rate of B errors: 4.2; 
Rate of C errors: 7.6; 
Number of operational errors: 432. 

Fiscal year/quarter: 2010/3; 
Rate of A errors: 0.7; 
Rate of B errors: 2.6; 
Rate of C errors: 6.1; 
Number of operational errors: 332. 

Fiscal year/quarter: 2010/4; 
Rate of A errors: 0.6; 
Rate of B errors: 2.2; 
Rate of C errors: 6.7; 
Number of operational errors: 366. 

Fiscal year/quarter: 2011/1; 
Rate of A errors: 0.6; 
Rate of B errors: 1.7; 
Rate of C errors: 5.7; 
Number of operational errors: 306. 

Fiscal year/quarter: 2011/2; 
Rate of A errors: 0.7; 
Rate of B errors: 2.3; 
Rate of C errors: 6.6; 
Number of operational errors: 378. 

Source: GAO analysis of FAA data. 

[End of figure] 

These incident rates do not meet FAA goals under both the prior 
severity system and using new risk assessment measures. In fiscal year 
2010, FAA reported 3.32 category A and B operational errors per 
million air traffic control operations, significantly exceeding its 
targeted rate for fiscal year 2010 of 2.05 per million operations. 
[Footnote 55] In fiscal year 2011, FAA replaced its operational error 
measure with a new measure--the System Risk Event Rate (SRER)--a 12-
month rolling rate of the most serious LoSS events per thousand such 
events.[Footnote 56] The rate of high-risk events also increased using 
this measure. According to data provided by FAA, the number of the 
most serious LoSS events--called high-risk events in the new risk 
assessment process[Footnote 57]--spiked from 9 events in December 2010 
to 16 events in January 2011 but has since decreased. However, the 
overall SRER increased significantly between December 2010 and 
February 2011 (from 21.9 to 29.9 high-risk LoSS events per 1000 LoSS 
events) and remains significantly elevated above FAA's target of 20 
serious LoSS events for every thousand such events. The SRER for the 
12-month period ending in April 2011 was 28.97.[Footnote 58] According 
to FAA officials, the agency's target of 20 LoSS events per thousand 
LoSS incidents represents the system performance baseline[Footnote 59] 
gathered using human reporting and may therefore be an unrealistic 
target as the agency moves to gathering data electronically. FAA plans 
to continue to collect data on and categorize events using both the 
old and new systems. Once FAA has completed a 2-year baseline period, 
it has committed to conduct an independent review of both metrics to 
determine whether any improvements are needed. 

Several Key Factors Likely Contribute to Trends in Runway Incursions 
and Airborne Operational Errors: 

Several factors have likely contributed to recent trends in runway 
incursions and airborne operational errors. The agency has noted that 
recent increases in runway incursions and airborne operational errors 
are primarily attributable to changes in FAA's reporting practices, 
which encourage increased reporting of incidents. We found evidence to 
suggest that changes to reporting policies and processes have likely 
contributed to the increased number of incidents reported--both into 
ATQA, the official database for incidents--and into ATSAP, the 
nonpunitive reporting system for air traffic controllers. In addition, 
the implementation of new technologies and procedures in the terminal 
area also likely contributed to an increase in the number of reported 
airborne incidents and runway incursions. FAA has carried out changes 
aimed at increasing reporting, and each of these factors may have 
contributed to an increase in the number of reported incidents. That 
said, it is possible that the increase in safety incidents in the 
terminal area may also reflect some real increase in the occurrence of 
safety incidents. As a comparison, we looked at the rate of en route 
operational errors, which are captured automatically by airplane 
tracking technology[Footnote 60] and would therefore not be expected 
to substantially increase by a change in reporting practices or 
procedures at the agency. We found that the average rate of en route 
operational errors in fiscal year 2010 was 38 percent higher than the 
year before, and that the overall rate increased 13 percent from the 
second quarter of fiscal year 2008 to the same quarter in 2011. 
According to FAA officials, some of the increase in reported en route 
errors may be attributable to increased confidence in the nonpunitive 
nature of the system--reflected by a decrease in the number of 
requests for reclassification of incidents from en route facilities. 

Changes to reporting processes and policies at FAA may explain in part 
the recent upward trend in reported runway incursions and airborne 
operational errors. Since operational errors and other losses of 
separation in both the tower and TRACON environments are currently 
reported manually by FAA supervisors and quality assurance staff into 
ATQA, changes in reported error rates may be partially attributable to 
changes made to encourage more comprehensive reporting of incidents. 
Most notably, as previously discussed, FAA changed its incident 
reporting policy in July 2009 such that individually identifying 
information, such as air traffic controller names and performance 
records, are no longer associated with specific events in the ATQA 
database. According to officials, this change may encourage 
controllers to share more information about incidents with supervisors 
and quality assurance officers. In addition, in fiscal year 2010, FAA 
stopped issuing incident "not to exceed" targets to individual 
facilities. According to officials, these targets may have led 
supervisors in the past to underreport less serious incidents in order 
to meet these targets. These policy changes may have increased 
reporting to an extent that these effects are apparent in incident 
rates. (See figure 15 for recent FAA changes to reporting practices 
overlaid on report operational errors.) 

Figure 15: FAA Changes to Reporting Practices and Recent Trends in 
Tower and TRACON Operational Errors: 

[Refer to PDF for image: combined vertical bar and line graph] 

Rate of operational errors (per million operations): 

Fiscal year/Quarter: 2007/3; 
Number of operational errors: 189; 
Rate of operational errors: 6.8. 

Fiscal year/Quarter: 2007/4; 
Number of operational errors: 194; 
Rate of operational errors: 6.9. 

Fiscal year/Quarter: 2008/1; 
Number of operational errors: 180; 
Rate of operational errors: 7.0. 

Fiscal year/Quarter: 2008/2; 
Number of operational errors: 220; 
Rate of operational errors: 9.0. 

Fiscal year/Quarter: 2008/3; 
Number of operational errors: 252; 
Rate of operational errors: 9.5. 

Fiscal year/Quarter: 2008/4; 
Number of operational errors: 233; 
Rate of operational errors: 8.6. 

Fiscal year/Quarter: 2009/1; 
Number of operational errors: 213; 
Rate of operational errors: 9.1. 

Fiscal year/Quarter: 2009/2; 
Number of operational errors: 173; 
Rate of operational errors: 7.7. 

Fiscal year/Quarter: 2009/3; 
Number of operational errors: 217; 
Rate of operational errors: 8.9. 

Fiscal year/Quarter: 2009/4; 
Number of operational errors: 287; 
Rate of operational errors: 11.4. 

Fiscal year/Quarter: 2010/1; 
Number of operational errors: 270; 
Rate of operational errors: 12.1. 

Fiscal year/Quarter: 2010/2; 
Number of operational errors: 432; 
Rate of operational errors: 20.1. 

Fiscal year/Quarter: 2010/3; 
Number of operational errors: 332; 
Rate of operational errors: 13.6. 

Fiscal year/Quarter: 2010/4; 
Number of operational errors: 366; 
Rate of operational errors: 14.7. 

Fiscal year/Quarter: 2011/1; 
Number of operational errors: 306; 
Rate of operational errors: 13.7. 

Fiscal year/Quarter: 2011/2; 
Number of operational errors: 378; 
Rate of operational errors: 17.7. 

Source: GAO analysis of FAA information. 

[End of figure] 

Implementation of a nonpunitive, confidential, system of reporting for 
air traffic controllers may have also contributed to the real increase 
in the occurrence of operational errors, according to FAA officials. 
While the implementation of ATSAP may affect reporting rates--either 
by increasing reporting or by lowering the number of reports to 
supervisors given that the system satisfies reporting requirements-- 
officials told us that it could also inadvertently lead to an actual 
increase in the occurrence of operational errors or deviations. 
According to these officials, the reduced personal accountability 
ATSAP provides may make some air traffic controllers less risk averse 
in certain situations.[Footnote 61] In addition, officials also noted 
that ATSAP may present a barrier to managerial efforts to directly 
manage controller performance. For example, if a report is filed into 
ATSAP, a supervisor may have limited options to assign training or 
take other corrective actions in response to an incident, even if he 
or she is aware that an error was made by an individual air traffic 
controller, presuming the incident did not involve alcohol or drug use 
or other such violations.[Footnote 62] 

The implementation of ATSAP may have resulted in increased reporting 
of incidents, although reporting into this system does not directly 
affect official trends in operational errors. According to FAA 
officials, the confidential, nonpunitive nature of ATSAP has 
contributed to a positive change in the reporting at FAA. As a result, 
errors that previously may have gone unreported by air traffic 
controllers are now being reported to ATSAP. However, data entered 
into ATSAP are not directly available to FAA and do not feed into ATQA 
(see figure 16). In addition, it is possible that some incidents that 
would have previously been reported to FAA are now being reported only 
to ATSAP, thus decreasing the number of incidents reported to 
FAA.[Footnote 63] According to ATSAP data, approximately 35 percent of 
all incidents reported to ATSAP in 2010 were "known" to FAA--meaning 
that the incident was reported into ATQA by a supervisor or manager, 
as well as into ATSAP by an air traffic controller--while the other 65 
percent were not official reported to FAA.[Footnote 64] 

Figure 16: Flow of Information into ATSAP is Separate from Other 
Systems FAA Uses to Track Air Traffic Safety Incidents: 

[Refer to PDF for image: illustration] 

Individual controllers: 
Voluntary reporting systems: 
Example: Air Traffic Safety Action Program (ATSAP); 
Individual controllers report into system. 

Automatically collected information: 
Digital safety reporting systems: 
Example: Traffic Analysis and Review Program (TARP); 
Captures incidents automatically (not used in all facilities). 

Quality assurance staff: 
Mandatory reporting systems: 
Example: Air Traffic Quality Assurance (ATQA); 
Quality assurance staff enter data on incident once they gain 
knowledge of the event. 

Risk Analysis Process (RAP): 
Combination of: 
Digital safety reporting systems; 
Mandatory reporting systems. 

Source: GAO. 

[End of figure] 

Implementation of new technologies in the terminal area may also 
impact recent trends in surface incidents and airborne losses of 
separation. For example, since FAA's ongoing implementation of TARP 
will allow FAA to automatically capture losses of separation in the 
tower and TRACON environments, it will also likely increase the number 
of reported losses of separation. According to officials, during its 
limited testing at facilities, TARP has already captured errors that 
were not being reported by air traffic controllers. For surface 
incidents, the ASDE-X system alerts controllers when aircraft or 
vehicles are at risk of colliding on runways, resulting in the 
identification of incidents that controllers might otherwise not be 
aware of. Designed as a surface surveillance system, ASDE-X helps to 
prevent collisions by raising alarms when aircraft appear to be at 
risk of colliding. As these alerts draw attention to near misses or 
potential collisions, they also serve to notify personnel of possible 
incursions and thus may have contributed to an increase in reported 
events, even as they may have prevented accidents. We analyzed the 
number of reported incursions at airports with ASDE-X and found that, 
at many of these airports, the number of reported incursions actually 
increased after their ASDE-X systems became operational. (For more 
information about our analysis of how the number of runway incursions 
changed after the installation of ASDE-X, see appendix III.) Officials 
with the Sensis Corporation, the developer of ASDE-X, acknowledged to 
us that this may be a side effect of the deployment of the system. 

Enhanced Oversight and Additional Information about Incidents Could 
Help Improve Safety in the Terminal Area: 

FAA has taken steps to improve safety in the terminal area since 2007 
and has both reduced the number of serious incursions and undertaken 
successful efforts to increase reporting of incidents, but we 
identified two areas in which FAA could further improve management of 
data and technology in order to take a more proactive, systemic 
approach to improving terminal area safety. These areas include: (1) 
enhanced oversight of terminal area safety, including the management 
of runway excursions and ramp areas, and (2) assurance that data for 
risk assessments are complete, meaningful, and available to decision 
makers. 

Federal Oversight of Terminal Areas Could Be Enhanced: 

Runway Safety Oversight Remains Limited: 

Stakeholders we spoke with generally lauded Runway Safety's efforts on 
incursions, but FAA could do more to expand to other aspects of runway 
safety--notably runway excursions--as well as playing a more active 
oversight role in ramp areas. As we noted earlier, FAA is rolling out 
a new program to gather and analyze data on excursions, which should 
allow the agency to better understand why excursions happen and 
develop programs to mitigate risk. FAA is exercising some additional 
authority over ramps by proposing rules to address airport safety 
management systems and training for personnel accessing ramp areas, 
but these efforts are limited and involve requiring airports to 
develop and implement their own safety guidelines. In 2007, we 
reported that the lack of standards for ramp operations hindered 
safety, and an upcoming report by the Airport Cooperative Research 
Program (ACRP) continues to find that no comprehensive standards exist 
with regard to ramp area markings, ground operations, or safety 
training.[Footnote 65] The two proposed rules by FAA on airport safety 
management systems and training establish some standards for the ramp 
area, but proposed federal oversight would still be limited. The 
proposed rule implementing safety management systems for airports 
would require airports to develop plans to identify and address 
hazards in the ramp area and on the airfield, in addition to ensuring 
that all employees with access to runways, taxiways, and ramps receive 
training on operational safety and on the airport's safety management 
system. Other aspects of ground handling, such as surface marking and 
ground operations, would continue to largely be overseen by airlines 
and the ground handling companies that are contracted by them. 

Organizational Placement of Runway Safety May Limit Efforts: 

The placement of the Runway Safety within the ATO Safety Office may 
limit its ability to serve as an effective focal point for runway and 
terminal area safety, given that aspects of runway and terminal area 
safety fall under the purview of several parts of FAA, including ATO, 
the Office of Airports, and the Office of Aviation Safety. In 2010, 
the IG recommended that the placement of Runway Safety within ATO be 
reconsidered, because the office may be limited in its ability to 
carry out cross-agency risk management efforts. Subsequently, the IG 
determined that Runway Safety had demonstrated effectiveness within 
ATO, but pointed to a need to periodically review organizational 
structures and processes to ensure that it continues to be effective. 
Runway Safety oversees data, assessments, and performance measures 
across a number of safety areas--air traffic control, pilot actions 
and training, outreach to general aviation, airport infrastructure, 
and technologies, among others--which are under the purview of 
different offices within FAA. As a result, Runway Safety has the 
potential to serve as the focal point for risk management in the 
terminal area. 

FAA Data for Risk Assessment May Not Be Complete, Meaningful, or 
Available to Decision Makers: 

Impact of Changes in Reporting Policies and Processes on Measures of 
Incidents and Risk Is Unclear: 

Multiple changes to reporting policies and processes in recent years 
make it difficult to know the extent to which the recent increases in 
some terminal area incidents are due to more accurate reporting or an 
increase in the occurrence of safety incidents or both. For example, 
FAA officials have specifically attributed the increase in airborne 
operational errors to changes in reporting practices following the 
implementation of ATSAP, but, as previously mentioned, the 
relationship between the implementation of ATSAP and an increase in 
errors is uncertain. Likewise, other changes to performance measures 
and internal reporting policies, such as removal of individually 
identifying information from ATQA, further obscure the source of 
recent trends. Without accurate and consistent measures of safety 
outcomes, FAA cannot assess the risks posed to aircraft or passengers 
over time or the impacts of its efforts to improve safety. 

FAA Risk-Assessment Processes Are Not Comprehensive: 

As we noted in a 2010 report, FAA has embarked on a data-driven, risk- 
based safety oversight approach.[Footnote 66] As part of this effort, 
FAA has established a new, risk-based measure to track losses of 
separation, but measures for runway incursions are not risk based, 
reflecting instead a simple count of incidents. Thus, FAA currently 
rates the severity of incursions based on proximity and the response 
time to avoid a collision and does not differentiate between types of 
aircraft--or the number of lives put at risk--as part of its severity 
calculation. While any loss of life is catastrophic, the impact of an 
accident involving a commercial aircraft carrying hundreds of 
passengers can have different implications than an accident involving 
smaller aircraft. According to industry stakeholders, the use of 
proximity as the main criterion for severity of incursions is overly 
simplistic. As a result, FAA may be unable to use incursion data to 
identify the most serious systemic safety issues. Similarly, the 
application of risk assessment to measures of runway safety could 
allow FAA to focus individually on the risk posed by incursions by 
large commercial aircraft, as well as the risk posed by an ever-
increasing incursion rate among general aviation operations. 
Additional attention to the root causes of incidents involving general 
aviation could potentially identify strategies addressing this ongoing 
challenge, which may include the installation of low-cost ground 
surveillance systems.[Footnote 67] While FAA officials did not detail 
immediate plans to alter the measure for incursions, officials did 
state that the agency plans to introduce surface incidents into RAP at 
the beginning of fiscal year 2012. The joint FAA-aviation industry 
Runway Safety Council is a first step toward the effort to reduce 
incursion risks. By identifying causes and making recommendations that 
could help determine what changes would be needed to make measures 
more risk-based, the Council's Root Cause Analysis Team can help 
reduce incursion risks. Further, according to FAA officials, the new 
measure being developed for excursions will be risk-based; however, 
this measure will not be fully in use until 2014. 

By contrast, FAA has taken steps to improve its ability to assess the 
risk of airborne operational errors and to collect more information. 
However, under the new risk assessment system, far fewer incidents are 
subject to analysis than were included in previous, nonrisk-based 
iterations, and the measure may therefore not account for all 
potential risk. For instance, RAP does not yet have procedures to 
assess losses of separation with terrain and with airspace boundaries. 
Currently, LoSS events in which at least 66 percent of minimum 
separation is maintained between aircraft are not assessed through 
FAA's recently launched RAP.[Footnote 68] According to FAA officials, 
the 66 percent threshold for inclusion in RAP was adopted in 
recognition of the resources required for the enhanced risk-analysis 
process. This initial threshold is not based on specific risk-based 
criteria. Furthermore, losses of separation eligible for inclusion in 
RAP are currently limited to those that occur between two or more 
radar-tracked aircraft. As a result, many incidents--such as those 
that occur between aircraft and terrain or aircraft and protected 
airspace--are currently excluded from FAA's process for assessing 
systemic risk. According to FAA officials, this exclusion is in part 
because there is no system in place through which the current RAP 
proximity inclusion threshold could be applied to these types of 
incidents, although FAA officials stated that an effort is under way 
to expand RAP to include these areas. In addition, FAA's new measure 
for assessing air traffic risk levels--the SRER--does not include many 
losses of separation that were tracked under the old measure, although 
it does expand the assessment process to include some errors caused by 
pilots. Further, while the technology has been developed to collect 
data automatically for potential operational errors involving losses 
of separation, FAA has delayed the full implementation of TARP in air 
traffic control towers and TRACONS. According to FAA officials, the 
implementation of TARP may create workload challenges for quality 
assurance staff, as the technology is likely to capture hundreds of 
potential losses of separation that were not previously being reported 
through existing channels.[Footnote 69] In 2009, the IG recommended 
that FAA establish firm timelines for the full implementation of TARP. 

Data collected through ATSAP, the nonpunitive reporting system, have 
limitations.[Footnote 70] There is the potential for serious events to 
be reported only to ATSAP and to therefore not be included in the 
official ATQA database or in RAP. Such events are referred to as 
"unknown" events. In 2010, 65 percent of incidents reported to ATSAP 
were unknown to FAA. (See figure 17.) FAA officials acknowledged that 
there are a large proportion of unknown incidents but stated that 
these incidents are likely to be minor. In addition, some information 
about incidents reported to ATSAP is available for analysis by FAA and 
the aviation industry via the Aviation Safety Information Analysis and 
Sharing (ASIAS) program, as de-identified ATSAP data are shared with 
ASIAS.[Footnote 71] Further, they noted, ATSAP reports may be 
procedural, rather than reports of incidents or operational errors. 
Available data from the ATSAP program office indicates, however, that 
some of the "unknown" reports in the system were potentially serious 
events. For example, between the program's launch in July 2008 and 
June 2011, 74 out of 253 ATSAP reports that were classified as 
potentially hazardous did not appear in ATQA, accounting for 29 
percent of these reports.[Footnote 72] In June 2011--the most recent 
month for which data are available--approximately one third of all 
ATSAP reports classified as potentially major events, and 42 percent 
of those classified as hazardous, did not appear in ATQA. According to 
officials, ATSAP allows controllers to report incidents that may have 
otherwise gone unreported, and the program facilitates early detection 
and improved awareness of operational deficiencies and adverse trends. 
These unknown events, FAA officials point out, would not likely have 
been reported into ATQA before the implementation of ATSAP. 

Figure 17: Incidents Reported to ATSAP in 2010 That Are Known to FAA 
versus Those That Are Unknown: 

[Refer to PDF for image: pie-chart] 

Unknown events (5,153 events): 65%; 
Known events (2,723 events): 35%. 

Source: GAO presentation of ATSAP data. 

[End of figure] 

Key Safety Data May Not be Available to Decision Makers: 

Information sharing challenges may impact the ability of FAA to 
analyze safety data and understand safety trends. Multiple FAA 
programs and data systems assign contributing factors to incidents, 
but factors are not coordinated across programs. For instance, both 
ATSAP and RAP have developed sets of factors that are identified as 
contributing to incidents during the incident investigation process. 
However, despite the fact that these two programs look at some of the 
same type of incidents (airborne losses of separation), program 
officials have not coordinated their development of the categories 
used to describe incidents. As a result, officials we interviewed 
stated that it is difficult to compare data across systems. For 
example, both ATSAP and RAP issued internal reports identifying top 
factors contributing to reported incidents, but there is no apparent 
overlap between the two lists. In addition, while the ATQA database 
contains more than 50 contributing factors for operational errors, FAA 
and the ATSAP program office do not use these data to identify 
systemic safety issues (see table 3). According to FAA officials, FAA 
is currently developing a common set of contributing factors for ATSAP 
and RAP, as well as a translation capability that will allow for the 
inclusion of historical data on contributing factors in future 
analyses. The IG raised concerns about the quality of ATQA data on 
contributing factors in a 2009 report, noting that FAA does not 
consistently include fatigue issues in contributing factor data it 
collects on operational errors.[Footnote 73] FAA has added 
contributing factors related to fatigue to ATSAP and is exploring ways 
to gather objective shift, schedule, and related resource management 
data to support enhanced fatigue analysis. 

Table 3: Comparison of Top Factors Identified as Contributing to 
Controller Safety Incidents across Programs: 

ATQA: 
1. Inappropriate use of displayed data; 
2. Readback; 
3. Improper use of visual data; 
4. Area of occurrence; 
5. Aircraft observation. 

RAP: 
1. Perception; 
2. Memory; 
3. Pilot actions. 

ATSAP: 
1. Clearance problem; 
2. Airspace violation; 
3. Expectation bias; 
4. Organization factor; 
5. Pilot noncompliance. 

Source: GAO analysis of FAA data. 

[End of table] 

Regional and local access to and awareness of data related to both 
individual incidents and incident trends may be limited. According to 
FAA officials we interviewed at the regional level, it is difficult 
for supervisors at the regional and facility levels to obtain 
information on incident trends specific to their area of supervision 
in part because key databases, such as ATQA, do not have the 
capability to allow regional supervisors to run region-or facility-
specific data queries. In addition, while multiple data resources may 
be available, officials stated that information on incidents is 
scattered, and no central source exists where employees can identify 
available data resources. While FAA has made advances in the quantity 
and comprehensiveness of the data it collects on incidents in the 
terminal area, officials stated that the agency faces difficulty in 
developing sophisticated databases with which to perform queries and 
modeling of the data. According to FAA officials, the full 
implementation of CEDAR will address many of the deficiencies 
identified by regional and local offices. 

The nature and scope of ramp accidents are still unknown, just as they 
were when we reported in 2007, and we were told by officials with 
Airports Council International that it can be difficult to for 
airports to get data on incidents in the ramp area--areas typically 
overseen by airlines. This will pose a challenge as airports move to 
implement safety management systems and seek to identify and mitigate 
hazards. As one aviation expert explained, even if data are available 
locally--which they may not be--the number of incidents at individual 
airports can be too few to allow for the identification of root causes 
or the proactive identification of risk. 

Conclusions: 

The Office of Runway Safety focuses on improving safety by reducing 
the number and severity of runway incursions. However, risk management 
in terminal areas involves more than just incursions--notably runway 
excursions and incidents in ramp areas. Runway Safety plans to start 
tracking runway excursions in October 2011, but it will take several 
years to develop processes for identifying and tracking incidents, 
identifying and mitigating risks, and measuring outcomes. Likewise, 
FAA does not track incidents in ramp areas, although we previously 
recommended that FAA work with the aviation industry and OSHA to 
develop a mechanism to collect and analyze data on ramp accidents. 
Airports implementing plans for safety management systems under FAA's 
proposed rule will need data that are useful, complete, and meaningful 
in order to accurately assess risk and plan for safety, but FAA cannot 
yet provide meaningful data for the assessment and management of risks 
posed by runway excursions or ramp areas. Without information on 
incidents in these areas, FAA and its safety partners are hampered in 
their ability to identify risk, develop mitigation strategies, and 
track outcomes. 

FAA addresses runway incursions as a specific type of incident and 
does not distinguish between commercial and general aviation in its 
performance measures. However, risks posed by runway safety incidents 
to passengers and aircraft in the national airspace system are 
different for commercial aircraft and general aviation. FAA 
performance measures for runway incursions--including the number, 
rate, and severity--do not reflect differences between commercial and 
general aviation and are not risk-based. The agency has installed risk-
reduction technologies at larger commercial service airports, for 
example, but in the absence of risk-based performance measures, it 
lacks the ability to prioritize projects or measure effectiveness. 
With regard to general aviation, this traffic currently accounts for 
about a third of total tower operations, but 60 percent of runway 
incursions involve these aircraft. While Runway Safety has 
acknowledged that general aviation has caused more runway incursions, 
without performance measures that reflect risk, FAA may not be able 
identify appropriate mitigation strategies to address the large--and 
growing--proportion of runway incidents--including both incursions and 
excursions--involving general aviation aircraft. Strategies to 
decrease the risk posed by safety incidents involving general aviation 
could include additional outreach to these pilots, increased 
remediation following pilot errors, or the installation of 
technologies such as low-cost ground surveillance at airports serving 
general aviation traffic. 

Safety in the terminal area is a shared responsibility among FAA, 
airlines, pilots, and airports, and there are a number of FAA offices 
that either collect or analyze terminal area incident data, but useful 
access to complete and meaningful data is limited. The agency 
currently does not have comprehensive risk-based data, sophisticated 
databases to perform queries and model data, methods of reporting that 
capture all incidents, or a level of coordination that would 
facilitate the comparison of incidents across systems. Technologies 
aimed at improving reporting have not been fully implemented. As a 
result, aviation officials managing risk using safety management 
systems, including local and regional decision makers, have limited--
if any--access to FAA incident data. For example, FAA's official 
database for air traffic safety does not allow local or regional FAA 
safety officials to run region-or facility-specific data queries. 
Further, under the new risk assessment process used for losses of 
separation, fewer incidents are assessed and accounted for in 
performance measures--such as losses of separation between aircraft 
and terrain or aircraft and protected airspace--which may distort risk 
assessment processes. Finally, according to FAA officials, one reason 
the agency has not fully implemented TARP is that implementation of 
TARP may create workload challenges for FAA quality assurance staff, 
as the technology is likely to capture hundreds of potential losses of 
separation that were not previously being reported through existing 
channels. FAA offices and others using a safety management system 
approach to manage risk should have access to complete and meaningful 
data to allow for hazard identification and risk management. The 
ability of FAA and airport officials--and the local Runway Safety 
Action Teams that they serve on-
-to identify safety risks, develop mitigation strategies, and measure 
outcomes is hindered by limited access to complete and meaningful data. 

Recommendations for Executive Action: 

To enhance oversight of terminal area safety to include the range of 
incidents that pose risks to aircraft and passengers, we recommend 
that the Secretary of Transportation direct the FAA Administrator to 
take the following three actions: 

* develop and implement plans to track and assess runway excursions 
and extend oversight to ramp safety; 

* develop separate risk-based assessment processes, measures, and 
performance goals for runway safety incidents (including both 
incursions and excursions) involving commercial aircraft and general 
aviation and expand the existing risk-based process for assessing 
airborne losses of separation to include incidents beyond those that 
occur between two or more radar-tracked aircraft; and: 

* develop plans to ensure that information about terminal area safety 
incidents, causes, and risk assessment is meaningful, complete, and 
available to appropriate decision makers. 

Agency Comments: 

We provided the Departments of Transportation and Labor, NTSB, and the 
National Aeronautics and Space Administration (NASA) with a draft of 
this report for review and comment. The Department of Transportation 
agreed to consider our recommendations and provided clarifying 
information about efforts made to improve runway safety, which we 
incorporated. The Department of Labor, NTSB, and NASA provided 
technical corrections, which we also incorporated. 

As agreed with your offices, unless you publicly announce the contents 
of this report earlier, we plan no further distribution until 7 days 
from the report date. At that time, we will send copies to the 
appropriate congressional committees, the Secretaries of 
Transportation and Labor, NTSB, the Administrator of NASA, and 
interested parties. In addition, this report will be available at no 
charge on the GAO Web site at [hyperlink, http://www.gao.gov]. 

If you or your staff members have any questions about this report, 
please contact me on (202) 512-2834 or at dillinghamg@gao.gov. Contact 
points for our Offices of Congressional Relations and Public Affairs 
may be found on the last page of this report. Key contributors to this 
report are listed in appendix IV. 

Signed by: 

Gerald L. Dillingham, Ph.D. 
Director, Physical Infrastructure Issues: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

Our objective was to review aviation safety and update our 2007 report 
on runway and ramp safety.[Footnote 74] To do so, we addressed the 
following questions: (1) What actions has the Federal Aviation 
Administration (FAA) taken to improve safety in the terminal area 
since 2007? (2) What are the trends in terminal area safety and the 
factors contributing to these trends? and (3) What additional actions 
could FAA take to improve terminal area safety? 

To identify actions FAA has taken since 2007 to improve safety in the 
terminal area and to identify additional actions FAA could take to 
improve safety, we reviewed our prior reports, as well as documents 
and reports from FAA, the Department of Transportation Inspector 
General (IG), the National Transportation Safety Board (NTSB), the 
International Civil Aviation Organization (ICAO), and others; FAA 
orders, advisory circulars, and regulations; and applicable laws. We 
also determined the roles and responsibilities of FAA, NTSB, the 
Occupational Safety and Health Administration (OSHA), the National 
Aeronautics and Space Administration (NASA), airports, and airlines 
involving runway, terminal airborne, and ramp safety. In addition to 
interviewing officials from FAA, IG, and NTSB, we interviewed aviation 
experts affiliated with the Air Line Pilots Association, Airports 
Council International, Air Transportation Association, the Flight 
Safety Foundation, and the National Air Traffic Controllers 
Association about terminal area safety practices and technologies. We 
also interviewed researchers from the Air Cooperative Research Program 
(ACRP) of the Transportation Research Board and experts affiliated 
with various aviation technology companies. To obtain information 
about air traffic control operations, observe the application of key 
technologies, and interview facility managers, we visited four FAA 
facilities that were near our GAO offices: the Potomac Consolidated 
Terminal Radar Approach Control facility, the Washington Air Route 
Traffic Control Center, and the air traffic control towers at Ronald 
Reagan Washington National Airport and the Seattle-Tacoma 
International Airport. We also interviewed airport officials with the 
Port of Seattle at Seattle-Tacoma International Airport. To obtain 
information about the Air Traffic Safety Action Program (ATSAP), we 
interviewed officials with the ATSAP program office and attended an 
Event Review Committee meeting in Renton, Washington. We also reviewed 
FAA's progress in addressing recommendations that we, IG, and NTSB 
have made in previous years and reviewed the processes that FAA uses 
to collect and assess runway and air traffic safety data. 

To identify and describe recent trends in terminal area safety and the 
factors contributing to these trends, we obtained and analyzed data 
from FAA, NTSB, and OSHA on safety incidents in the terminal area. We 
analyzed FAA runway incursion data collected from fiscal year 2001 
through the second quarter of fiscal year 2011, as well as FAA data on 
airborne operational errors from the Air Traffic Quality Assurance 
database (ATQA) from the third quarter of fiscal year 2007 through the 
second quarter of fiscal year 2011. We limited our analysis to 
airborne operational errors in order to avoid double counting of 
surface operational errors that are included in our counts of runway 
incursions. We used Operations Network data from FAA to determine 
rates of incursions and airborne operational errors per million 
operations. Rates of incursions were calculated per million tower 
operations, and rates of airborne operational errors were calculated 
per million operations performed by air traffic control towers, 
terminal radar approach control (TRACON) facilities, and en route 
facilities on a quarterly basis.[Footnote 75] We also reviewed NTSB 
data involving runway incursions and excursions from 2008 through June 
2011 and summarized OSHA data on fatalities in the ramp area from 2001 
through 2010. We used statistical models to assess the association 
between safety incidents and the concentration of general aviation 
operations and the implementation of the Airport Surface Detection 
Equipment, Model X (ASDE-X) surface surveillance system from fiscal 
year 2001 through April 2011. These models estimated how the number of 
incursions changed after airports installed ASDE-X or increased the 
proportion of operations involving general aviation. The models 
account for other factors that may contribute to incursions, such as 
long-term weather patterns, runway layouts, as well as controller and 
pilot experience. See appendix III for more information about the 
methods and results of these analyses. To assess the reliability of 
FAA data, we (1) reviewed internal FAA documents about its collection, 
entry, and maintenance of the data and (2) interviewed FAA officials 
who were knowledgeable about the content and limitations of these 
data. Both NTSB and OSHA provided information about the reliability of 
their excursion and fatality data, respectively. We determined that 
these data were sufficiently reliable for the descriptive and 
comparative analyses used in this report. 

We conducted this performance audit from February 2011 to October 2011 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe the 
evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 

[End of section] 

Appendix II: Technologies to Improve Runway Safety: 

Researching, testing, and deploying new technology is a major part of 
FAA's risk-reduction strategy. A number of available technological 
systems are intended to help reduce the number and severity of runway 
incursions. For example, to give air traffic controllers better 
visibility of activity on the airfield and help prevent collisions, 
FAA has installed the ASDE-X system at 35 major airports, while the 
Airport Surface Detection Equipment, Model 3 (ASDE-3) radar and the 
Airport Movement Area Safety System (AMASS) provide surface 
surveillance at 9 additional airports.[Footnote 76] Runway status 
lights, which will be installed at 23 airports, are a fully automatic 
series of lights that give pilots a visible warning when runways are 
not clear to enter, cross, or depart on. To mitigate the risks posed 
by runway excursions, FAA conducted research that led to the 
development of the Engineered Materials Arresting System (EMAS), a bed 
of crushable concrete designed to stop aircraft from overrunning 
runway areas. As of July 2011, EMAS has been installed at 52 runways 
at 36 airports, and there are plans to install 11 EMAS systems at 7 
others. According to FAA officials, EMAS has successfully arrested 
seven overrunning aircraft with no fatalities or serious injuries and 
little damage to the aircraft to date. (See table 4 for a brief 
description of technologies designed to improve runway safety.) 

Table 4: Technologies Intended to Reduce the Number and Severity of 
Runway Incursions or Excursions: 

Technology: ASDE-3/AMASS; 
Description: ASDE-3/AMASS is a radar-based system that tracks ground 
movements and provides an automatic visual and audio alert to 
controllers when it detects potential collisions on airport runways 
and taxiways. 

Technology: ASDE-X; 
Description: ASDE-X integrates data from a variety of sources, 
including radars, transponder multilateration systems, and Automatic 
Dependent Surveillance-Broadcast (ADS-B) to provide accurate target 
position and identification information and thus give controllers a 
more reliable view of airport operations. ASDE-X provides tower 
controllers a surface traffic situation display with visual and 
audible alerting of potential collisions. 

Technology: ADS-B; 
Description: ADS-B uses Global Positioning System signals along with 
aircraft avionics to transmit an aircraft's location to ground 
receivers. The ground receivers then transmit that information to 
controller screens and cockpit displays on aircraft equipped with ADS-
B avionics. Both pilots and controllers will be able to see other 
aircraft in the sky around them. Pilots will also be able to see bad 
weather and terrain and receive flight information such as temporary 
flight restrictions. All commercial and most general aviation aircraft 
will be required to be equipped with ADS-B by 2020 as part of FAA's 
implementation of the Next Generation Air Transportation System. 

Technology: Electronic Flight Bag with Moving Map Displays; 
Description: The FAA reached agreements with several U.S. airlines to 
fund in-cockpit runway safety systems in exchange for critical 
operational data. With Moving Map Displays and Own-Ship Position, 
pilots will see exactly where their aircraft is on the airfield, thus 
reducing the chances of losing situational awareness and being in the 
wrong place. 

Technology: Final Approach Runway Occupancy Signal (FAROS); 
Description: FAROS is designed to provide a visual alert of runway 
status to pilots intending to use a runway. Arriving aircraft 
approaching a runway for landing are alerted if the runway is occupied 
by flashing Indicator lights. 

Technology: Low-cost ground surveillance systems; 
Description: Low-cost systems designed to further reduce the risk of 
ground incidents or accidents, especially during periods of low 
visibility by providing ASDE-X-like capabilities at certain small and 
medium-sized airports. 

Technology: Runway safety area improvements; 
Description: FAA has established standards for runway safety areas, 
which are unobstructed areas surrounding a runway, to enhance safety 
in the event that an aircraft overruns, undershoots, or veers off a 
runway. FAA airport design standards generally require commercial 
airports to establish, to the extent practicable, 1,000-foot runway 
safety areas at both ends of a runway. 

Technology: Runway status lights; 
Description: Runway status lights provide warnings on runways and 
taxiways, illuminating when it is unsafe to enter, cross, or take off 
on a runway. Airport surveillance sensor inputs are processed through 
light control logic that commands in-pavement lights to illuminate red 
when traffic is on or approaching the runway. 

Technology: EMAS; 
Description: A lightweight, crushable concrete that is placed at the 
end of a runway to stop or greatly slow an aircraft that overruns the 
runway. A standard EMAS installation extends 600 feet from the end of 
the runway. However, EMAS is still effective even when less than 600 
feet of land is available for its installation. EMAS has been 
installed at airports that do not have enough land for a standard 
runway safety area. 

Source: GAO analysis of FAA information. 

[End of table] 

[End of section] 

Appendix III: Technical Appendix: 

This appendix summarizes our statistical analysis of the relationship 
between the ASDE-X airfield surface surveillance system and runway 
incursions. We focused on ASDE-X, among many other runway safety 
programs, because of its potential for persuasive impact evaluation. 
We describe how the process FAA used to install ASDE-X created a 
"quasi-experiment," which allows us to compare how incursions changed 
at airports that received the technology relative to airports that 
kept the status quo. 

Evaluating the Impact of ASDE-X on Runway Incursions: 

A key goal of ASDE-X is to make air traffic controllers more aware of 
activities on taxiways and runways in order to avoid collisions. The 
system consists of airfield radar and sensors that collect data on the 
location of aircraft and vehicles. Computers transform these data into 
continuously updated maps of the airfield, which are displayed on 
color monitors in air traffic control towers. The system warns 
controllers of potential collisions--which may draw attention to 
possible incursions--through visual and audible alarms. 

FAA used a selective and staggered process to install ASDE-X at 34 
airports from 2003 through 2011 (of 35 airports slated to receive this 
system). The variation among airports receiving the technology, as 
well as the times when they received it, allows for a quasi-
experimental evaluation. This type of analysis compares the change in 
incursions over time at airports that installed ASDE-X with the change 
at airports that did not receive the technology, also known as a 
"difference-in-difference." 

FAA selected 35 airports to receive ASDE-X after assessing 59 top-tier 
airports, and those airports that were not selected can serve as a 
valid comparison group. Although FAA originally estimated the 
financial benefits of ASDE-X at each of 59 candidate airports, 24 of 
them ultimately did not receive the technology.[Footnote 77] At a 
minimum, the 24 unsuccessful airports were similar enough to the 
treated airports to be part of the same funding process. In some 
cases, airports that did and did not receive ASDE-X had similar 
estimated benefits. For these reasons, the runner-up airports provide 
a plausible comparison group for analysis, but we use a variety of 
other comparison groups to ensure that our findings are robust. 

Many factors that may contribute to incursions are controlled in our 
analysis here. We control for variation among airports in runway and 
taxiway layouts, markings, and lighting, in addition to long-term 
variations in weather, air traffic, and pilot and controller skills. 
The staggered installation of ASDE-X makes bias due to short-term 
weather conditions or pilot and controller experience unlikely, 
because these factors would need to be correlated with 34 installation 
times throughout the country. In addition, the staggered installation 
lets us control for factors that affect all airports equally, such as 
changes in training and procedures made throughout the country at the 
same time. 

Data Sources: 

The time period of our analysis spans fiscal years 2001 through April 
2011. We assembled data on the number of incursions that occurred per 
month at each FAA-towered airport in this period, along with data on 
air traffic control tower operations. The latter data included the 
number of monthly tower operations at each airport, as well as the 
mixture of commercial and general aviation operations.[Footnote 78] 
The operations data identify the population of interest, including the 
many smaller airports with no incursions that do not appear in the 
incursion data. 

FAA provided the installation dates and locations for the ASDE-X, 
runway status lights, FAROS, and low-cost ground surveillance systems. 
We used these data to identify whether each technology was installed 
for each airport and month between fiscal year 2001 and April 2011. 

The complete dataset is a panel, with the variables above measured for 
a maximum of 485 airports and 127 months between fiscal year 2001 and 
April 2011, producing a maximum sample size of 58,917 airport-months. 

Statistical Modeling: 

We used a statistical model to estimate the association between ASDE-X 
and the number of incursions for airport i and month t. The model took 
the form of: 

E(Yit | ai,pt,tit,xit) = ai exp(äpt + atit + xitâ), 

where Yit randomly varies according to the Poisson distribution, ai is 
a vector of airport fixed effects, pt is a vector of year-month fixed 
effects, tit indicates whether ASDE-X was operational at airport i in 
month t, xit are other time-varying covariates, and ä, á, and â are 
vectors of parameters. We estimated the change in incursions after the 
installation of ASDE-X using one contemporaneous, before-and-after 
parameter, á, because ASDE-X likely has an immediate effect on 
incursions once it has been installed that does not change over 
time.[Footnote 79],[Footnote 80] 

The covariates xit included the number of air traffic control tower 
operations (to measure exposure and variation in the nature of 
activity across airports), indicators for having 25 to 60 percent and 
greater than 60 percent of operations involving general aviation 
(excluding 0 to 25 percent), and indicators for having the runway 
status lights, FAROS, or low-cost ground surveillance systems 
installed at airport i and time t. 

The models used several groups to compare the change in incursions 
before and after ASDE-X was installed, in order to assess the 
sensitivity of our results to plausible alternatives. The groups 
included: 

1. all FAA-towered airports; 

2. airports that were included in the FAA benefit-cost analysis above 
but did not receive ASDE-X; 

3. the top 100 airports in tower operations from fiscal year 2001 
through April 2011; 

4. airports that had a similar ground surveillance system, ASDE-3/ 
AMASS, installed prior to the first installation of ASDE-X (baseline); 

5. airports that did not have ASDE-3/AMASS installed at baseline; and: 

6. airports that did not have ASDE-3/AMASS installed at baseline and 
that were among the top 100 airports in tower operations from fiscal 
year 2001 through April 2011. 

Groups 2, 3, 4, and 6 control for differences among the generally 
large airports that received ASDE-X and the small airports that did 
not. Groups 4, 5, and 6 distinguish between airports that did and did 
not already have a form of ground surveillance radar and safety 
warnings. FAA considered the preexisting technology when choosing ASDE-
X sites, and the effect of ASDE-X may vary according to the system 
that was already in place. 

Runway Incursions Rise after the Installation of ASDE-X, but Better 
Reporting May Explain the Change: 

Figure 18 plots the average monthly incursion rate for airports that 
did and did not receive ASDE-X, rescaled to a ratio of the over-time 
mean to better express the trends. The smooth lines summarize the 
average incursion rate for each group and month using nonparametric 
locally weighted regression models. The vertical lines show the ASDE-X 
installation times for each airport. 

Figure 18: Change in Reported Runway Incursion Rates by Installation 
of ASDE-X, Fiscal Year 2001-April 2011: 

[Refer to PDF for image: plotted point and line graph] 

Plotted points represent actual incursions; line represent averages 
within time period. 

Ratio of average monthly incursions rate to average incursion rate in 
period: 

Fiscal year: 2001: 

October-00; 
ASDE-X airports: 1.13; 
Not ASDE-X airports: 1.37. 

November-00; 
ASDE-X airports: 1.10; 
Not ASDE-X airports: 1.35. 

December-00; 
ASDE-X airports: 1.09; 
Not ASDE-X airports: 1.33. 

January-01; 
ASDE-X airports: 1.07; 
Not ASDE-X airports: 1.31. 

February-01; 
ASDE-X airports: 1.05; 
Not ASDE-X airports: 1.29. 

March-01; 
ASDE-X airports: 1.03; 
Not ASDE-X airports: 1.27. 

April-01; 
ASDE-X airports: 1.01; 
Not ASDE-X airports: 1.24. 

May-01; 
ASDE-X airports: 0.99; 
Not ASDE-X airports: 1.23. 

June-01; 
ASDE-X airports: 0.98; 
Not ASDE-X airports: 1.21. 

July-01; 
ASDE-X airports: 0.96; 
Not ASDE-X airports: 1.19. 

August-01; 
ASDE-X airports: 0.95; 
Not ASDE-X airports: 1.17. 

September-01; 
ASDE-X airports: 0.93; 
Not ASDE-X airports: 1.16. 

Fiscal year: 2002: 

October-01; 
ASDE-X airports: 0.92 
Not ASDE-X airports: 1.14. 

November-01; 
ASDE-X airports: 0.90; 
Not ASDE-X airports: 1.12. 

December-01; 
ASDE-X airports: 0.89; 
Not ASDE-X airports: 1.11. 

January-02; 
ASDE-X airports: 0.88; 
Not ASDE-X airports: 1.09. 

February-02; 
ASDE-X airports: 0.87; 
Not ASDE-X airports: 1.08. 

March-02; 
ASDE-X airports: 0.86; 
Not ASDE-X airports: 1.06. 

April-02; 
ASDE-X airports: 0.85; 
Not ASDE-X airports: 1.05. 

May-02; 
ASDE-X airports: 0.84; 
Not ASDE-X airports: 1.03. 

June-02; 
ASDE-X airports: 0.83; 
Not ASDE-X airports: 1.02. 

July-02; 
ASDE-X airports: 0.82; 
Not ASDE-X airports: 1.01. 

August-02; 
ASDE-X airports: 0.81; 
Not ASDE-X airports: 0.99. 

September-02; 
ASDE-X airports: 0.81; 
Not ASDE-X airports: 0.98. 

Fiscal year: 2003: 

October-02; 
ASDE-X airports: 0.80; 
Not ASDE-X airports: 0.97. 

November-02; 
ASDE-X airports: 0.80; 
Not ASDE-X airports: 0.96. 

December-02; 
ASDE-X airports: 0.79; 
Not ASDE-X airports: 0.95. 

January-03; 
ASDE-X airports: 0.79; 
Not ASDE-X airports: 0.94. 

February-03; 
ASDE-X airports: 0.78; 
Not ASDE-X airports: 0.93. 

March-03; 
ASDE-X airports: 0.78; 
Not ASDE-X airports: 0.92. 

April-03; 
ASDE-X airports: 0.78; 
Not ASDE-X airports: 0.91. 

May-03; 
ASDE-X airports: 0.77; 
Not ASDE-X airports: 0.90. 

June-03; 
ASDE-X airports: 0.77; 
Not ASDE-X airports: 0.90. 

July-03; 
ASDE-X airports: 0.77; 
Not ASDE-X airports: 0.89. 

August-03; 
ASDE-X airports: 0.77; 
Not ASDE-X airports: 0.88. 

September-03; 
ASDE-X airports: 0.77; 
Not ASDE-X airports: 0.88. 

Fiscal year: 2004: 

October-03; 
ASDE-X airports: 0.78; 
Not ASDE-X airports: 0.869107 

November-03; 
ASDE-X airports: 0.78; 
Not ASDE-X airports: 0.86. 

December-03; 
ASDE-X airports: 0.79; 
Not ASDE-X airports: 0.86. 

January-04; 
ASDE-X airports: 0.79; 
Not ASDE-X airports: 0.85. 

February-04; 
ASDE-X airports: 0.80; 
Not ASDE-X airports: 0.85. 

March-04; 
ASDE-X airports: 0.80; 
Not ASDE-X airports: 0.85. 

April-04; 
ASDE-X airports: 0.81; 
Not ASDE-X airports: 0.84. 

May-04; 
ASDE-X airports: 0.82; 
Not ASDE-X airports: 0.84. 

June-04; 
ASDE-X airports: 0.82; 
Not ASDE-X airports: 0.84. 

July-04; 
ASDE-X airports: 0.83; 
Not ASDE-X airports: 0.84. 

August-04; 
ASDE-X airports: 0.84; 
Not ASDE-X airports: 0.83. 

September-04; 
ASDE-X airports: 0.84; 
Not ASDE-X airports: 0.83. 

Fiscal year: 2005: 

October-04; 
ASDE-X airports: 0.84; 
Not ASDE-X airports: 0.83. 

November-04; 
ASDE-X airports: 0.85; 
Not ASDE-X airports: 0.83. 

December-04; 
ASDE-X airports: 0.86; 
Not ASDE-X airports: 0.84. 

January-05; 
ASDE-X airports: 0.87; 
Not ASDE-X airports: 0.84. 

February-05; 
ASDE-X airports: 0.88; 
Not ASDE-X airports: 0.84. 

March-05; 
ASDE-X airports: 0.88; 
Not ASDE-X airports: 0.85. 

April-05; 
ASDE-X airports: 0.89; 
Not ASDE-X airports: 0.85. 

May-05; 
ASDE-X airports: 0.90; 
Not ASDE-X airports: 0.85. 

June-05; 
ASDE-X airports: 0.91; 
Not ASDE-X airports: 0.86. 

July-05; 
ASDE-X airports: 0.92; 
Not ASDE-X airports: 0.86. 

August-05; 
ASDE-X airports: 0.93; 
Not ASDE-X airports: 0.87. 

September-05; 
ASDE-X airports: 0.94; 
Not ASDE-X airports: 0.88. 

Fiscal year: 2006: 

October-05; 
ASDE-X airports: 0.95; 
Not ASDE-X airports: 0.88. 

November-05; 
ASDE-X airports: 0.95; 
Not ASDE-X airports: 0.89. 

December-05; 
ASDE-X airports: 0.96; 
Not ASDE-X airports: 0.89. 

January-06; 
ASDE-X airports: 0.97; 
Not ASDE-X airports: 0.89. 

February-06; 
ASDE-X airports: 0.98; 
Not ASDE-X airports: 0.90. 

March-06; 
ASDE-X airports: 0.99; 
Not ASDE-X airports: 0.90. 

April-06; 
ASDE-X airports: 1.00; 
Not ASDE-X airports: 0.91. 

May-06; 
ASDE-X airports: 1.01; 
Not ASDE-X airports: 0.92. 

June-05; 
ASDE-X airports: 1.02; 
Not ASDE-X airports: 0.92. 

July-06; 
ASDE-X airports: 1.03; 
Not ASDE-X airports: 0.93. 

August-06; 
ASDE-X airports: 1.05; 
Not ASDE-X airports: 0.94. 

September-06; 
ASDE-X airports: 1.06; 
Not ASDE-X airports: 0.94. 

Fiscal year: 2007: 

October-06; 
ASDE-X airports: 1.07; 
Not ASDE-X airports: 0.95. 

November-06; 
ASDE-X airports: 1.08; 
Not ASDE-X airports: 0.96. 

December-06; 
ASDE-X airports: 1.09; 
Not ASDE-X airports: 0.96. 

January-07; 
ASDE-X airports: 1.10; 
Not ASDE-X airports: 0.97. 

February-07; 
ASDE-X airports: 1.11; 
Not ASDE-X airports: 0.98. 

March-07; 
ASDE-X airports: 1.12; 
Not ASDE-X airports: 0.98. 

April-07; 
ASDE-X airports: 1.13; 
Not ASDE-X airports: 0.99. 

May-07; 
ASDE-X airports: 1.13; 
Not ASDE-X airports: 0.99. 

June-07; 
ASDE-X airports: 1.14; 
Not ASDE-X airports: 1.00. 

July-07; 
ASDE-X airports: 1.14; 
Not ASDE-X airports: 1.00. 

August-07; 
ASDE-X airports: 1.15; 
Not ASDE-X airports: 1.00. 

September-07; 
ASDE-X airports: 1.15; 
Not ASDE-X airports: 1.00. 

Fiscal year: 2008: 

October-07; 
ASDE-X airports: 1.15; 
Not ASDE-X airports: 1.01. 

November-07; 
ASDE-X airports: 1.16; 
Not ASDE-X airports: 1.02. 

December-07; 
ASDE-X airports: 1.16; 
Not ASDE-X airports: 1.02. 

January-08; 
ASDE-X airports: 1.16; 
Not ASDE-X airports: 1.02. 

February-08; 
ASDE-X airports: 1.16; 
Not ASDE-X airports: 1.02. 

March-08; 
ASDE-X airports: 1.16; 
Not ASDE-X airports: 1.03. 

April-08; 
ASDE-X airports: 1.17; 
Not ASDE-X airports: 1.03. 

May-08; 
ASDE-X airports: 1.17; 
Not ASDE-X airports: 1.03. 

June-08; 
ASDE-X airports: 1.17; 
Not ASDE-X airports: 1.04. 

July-08; 
ASDE-X airports: 1.17; 
Not ASDE-X airports: 1.04. 

August-08; 
ASDE-X airports: 1.17; 
Not ASDE-X airports: 1.04. 

September-08; 
ASDE-X airports: 1.17; 
Not ASDE-X airports: 1.04. 

Fiscal year: 2009: 

October-08; 
ASDE-X airports: 1.17; 
Not ASDE-X airports: 1.05. 

November-08; 
ASDE-X airports: 1.18; 
Not ASDE-X airports: 1.05. 

December-08; 
ASDE-X airports: 1.18; 
Not ASDE-X airports: 1.05. 

January-09; 
ASDE-X airports: 1.18; 
Not ASDE-X airports: 1.05. 

February-09; 
ASDE-X airports: 1.18; 
Not ASDE-X airports: 1.05. 

March-09; 
ASDE-X airports: 1.18; 
Not ASDE-X airports: 1.05. 

April-09; 
ASDE-X airports: 1.18; 
Not ASDE-X airports: 1.06. 

May-09; 
ASDE-X airports: 1.18; 
Not ASDE-X airports: 1.06. 

June-09; 
ASDE-X airports: 1.18; 
Not ASDE-X airports: 1.06. 

July-09; 
ASDE-X airports: 1.18; 
Not ASDE-X airports: 1.06. 

August-09; 
ASDE-X airports: 1.18; 
Not ASDE-X airports: 1.06. 

September-09; 
ASDE-X airports: 1.18; 
Not ASDE-X airports: 1.06. 

Fiscal year: 2010: 

October-09; 
ASDE-X airports: 1.18; 
Not ASDE-X airports: 1.06. 

November-09; 
ASDE-X airports: 1.18; 
Not ASDE-X airports: 1.06. 

December-09; 
ASDE-X airports: 1.18; 
Not ASDE-X airports: 1.06. 

January-10; 
ASDE-X airports: 1.17; 
Not ASDE-X airports: 1.06. 

February-10; 
ASDE-X airports: 1.17; 
Not ASDE-X airports: 1.06. 

March-10; 
ASDE-X airports: 1.17; 
Not ASDE-X airports: 1.06. 

April-10; 
ASDE-X airports: 1.17; 
Not ASDE-X airports: 1.06. 

May-10; 
ASDE-X airports: 1.17; 
Not ASDE-X airports: 1.06. 

June-10; 
ASDE-X airports: 1.17; 
Not ASDE-X airports: 1.06. 

July-10; 
ASDE-X airports: 1.16; 
Not ASDE-X airports: 1.06. 

August-10; 
ASDE-X airports: 1.16; 
Not ASDE-X airports: 1.06. 

September-10; 
ASDE-X airports: 1.16; 
Not ASDE-X airports: 1.06. 

Fiscal year 2011: 

October-10; 
ASDE-X airports: 1.16; 
Not ASDE-X airports: 1.06. 

November-10; 
ASDE-X airports: 1.15; 
Not ASDE-X airports: 1.06. 

December-10; 
ASDE-X airports: 1.15; 
Not ASDE-X airports: 1.06. 

January-11; 
ASDE-X airports: 1.15; 
Not ASDE-X airports: 1.06. 

February-11; 
ASDE-X airports: 1.14; 
Not ASDE-X airports: 1.05. 

March-11; 
ASDE-X airports: 1.14; 
Not ASDE-X airports: 1.05. 

April-11; 
ASDE-X airports: 1.13; 
Not ASDE-X airports: 1.05. 
  
Source: GAO analysis of FAA data. 

[End of figure] 

Prior to the first installation of ASDE-X, the incursion rate changed 
in roughly the same ways for the ASDE-X and comparison airports. As 
FAA began to install the system in late 2003, and the incursion rate 
began to increase for the ASDE-X airports, but it decreased and then 
increased at a slower rate for the comparison airports. Substituting 
the other comparison groups in these plots produces similar patterns. 
Consequently, the raw data suggest that reported incursions increased 
at airports that received ASDE-X, as compared to the change at 
airports that did not receive the system. 

Our statistical model supports similar conclusions (see table 5). 
Airports that received ASDE-X saw their reported incursions increase 
by 25.6 percent more than all other FAA-towered airports; by 18.3 
percent more than other airports that were among the 100 busiest; and 
by 15.4 percent more than airports that were unsuccessful candidates 
to receive ASDE-X in the FAA benefit-cost analysis. The confidence 
intervals of these estimates broadly overlap, suggesting that the 
differences in the changes between the airports that received ASDE-X 
and the comparison airports are not distinguishable from each other 
and therefore do not depend on the choice of comparison airports. All 
estimates hold constant fixed differences across airports, as well as 
FAA policy changes, programs, and other national trends that affect 
airports in similar ways over time.[Footnote 81] 

Table 5: Estimated Change in Runway Incursions after the Installation 
of ASDE-X Airfield Surveillance Technology: 

Comparison group: All FAA-towered airports; 
Difference in estimated change: 25.6 percent; 
95 percent confidence interval: [5.0 percent, 50.2 percent]. 

Comparison group: ASDE-X finalist airports, not selected; 
Difference in estimated change: 18.3 percent; 
95 percent confidence interval: [-2.8, 44.0]. 

Comparison group: 100 busiest airports; 
Difference in estimated change: 15.4 percent; 
95 percent confidence interval: [-5.3, 40.6]. 

Comparison group: Airports without ASDE-3/AMASS; 
Difference in estimated change: 37.0 percent; 
95 percent confidence interval: [4.0, 80.3]. 

Comparison group: Airports without ASDE-3/AMASS and among 100 busiest; 
Difference in estimated change: 28.0 percent; 
95 percent confidence interval: [-3.5, 69.9]. 

Comparison group: Airports with ASDE-3/AMASS; 
Difference in estimated change: 12.3 percent; 
95 percent confidence interval: [-10.7, 41.3]. 

Source: GAO analysis of FAA data. 

[End of table] 

The change in incursions depends on whether the ASDE-3/AMASS surface 
surveillance system was previously installed. Among airports that did 
not already have ASDE-3/AMASS, incursions increased by 37.0 percent 
after the facility installed ASDE-X, as compared with such airports 
that did not install ASDE-X. In contrast, incursions increased by only 
12.3 percent at airports that already had ASDE-3/AMASS or by about one 
third of the change at airports that did not have ASDE-3/AMASS. This 
is an intuitive pattern, because the addition of a new surveillance 
system should likely have a larger effect than the replacement of a 
similar system. 

The positive association between ASDE-X and runway incursions may 
reflect better reporting rather than less safe runways. ASDE-X is 
designed to make air traffic controllers more aware of airfield 
activity and to warn about possible collisions. In principle, this 
should reduce the risk of serious incursions, because controllers can 
prevent them before they occur. At the same time, more precise 
information from ASDE-X may allow controllers to more reliably report 
less serious, class C and D incursions, which made up 97 percent of 
the incursions in our period of analysis. These incursions previously 
might have been less visible from air traffic control towers and, 
therefore, underreported prior to the installation of ASDE-X.[Footnote 
82] 

Our statistical analysis of ASDE-X has several limitations that 
prevent us from conclusively estimating the system's impact on 
reported incursions. We were unable to obtain timely data to control 
for a number of important variables, such as short-term weather 
patterns, changes to runway lighting and markings, and implemented 
recommendations of the local and regional runway safety action teams. 
Accounting for these factors may produce different results. In this 
sense, we view our results as preliminary, pending further evaluation 
with additional covariates and years of incursion data. 

[End of section] 

Appendix IV: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Gerald L. Dillingham, Ph.D., (202) 512-2834, or dillinghamg@gao.gov: 

Staff Acknowledgments: 

In addition to the individual named above, Heather MacLeod, Assistant 
Director; Russ Burnett; Martha Chow; Dave Hooper; Delwen Jones; Molly 
Laster; Brooke Leary; Josh Ormond; and Jeff Tessin made key 
contributions to this report. 

[End of section] 

Footnotes: 

[1] Taxiways are routes that aircraft follow to and from runways. 

[2] See GAO, Aviation Runway and Ramp Safety: Sustained Efforts to 
Address Leadership, Technology, and Other Challenges Needed to Reduce 
Accidents and Incidents, [hyperlink, 
http://www.gao.gov/products/GAO-08-29] (Washington, D.C.: Nov. 20, 
2007). 

[3] Movement areas include taxiways and runways, while nonmovement 
areas include ramps, apron areas around these movement areas, and 
fueling areas. We are defining ramps to include all areas from the 
gate to runways or taxiways. 

[4] Air traffic greater than 40 miles from the airport is referred to 
as en route air traffic and is controlled by air route traffic control 
centers. Not all airports have air traffic control towers. 

[5] On October 1, 2007, FAA began using a broader definition of a 
runway incursion developed by the International Civil Aviation 
Organization (ICAO), a United Nations specialized agency, defining 
these incidents as the incorrect presence of an aircraft, vehicle, or 
person on the protected area of a surface designated for the landing 
and takeoff of aircraft. Previously, FAA had defined a runway 
incursion as "any occurrence in the runway environment involving an 
aircraft, vehicle, person, or object on the ground that creates a 
collision hazard or results in a loss of required separation when an 
aircraft is taking off, intending to take off, landing, or intending 
to land." 

[6] According to FAA officials, neither FAA nor ICAO currently has a 
formal definition for runway excursions. The Flight Safety Foundation 
defines runway excursions as when an aircraft veers off or overruns a 
runway. See Flight Safety Foundation, Reducing the Risk of Runway 
Excursions: Report of the Runway Safety Initiative, May 2009. 

[7] One exception to these minimum standards involves air traffic 
controllers using visual separation rules for aircraft. 

[8] After 2 years without a permanent Director, FAA put in place a 
Director for Runway Safety in 2007, at the time of our prior report. 
This Director retired in April 2011, and the office has had two Acting 
Directors since then. 

[9] Airports typically lease the ramp areas to air carriers. 

[10] See [hyperlink, http://www.gao.gov/products/GAO-08-29]. 

[11] Primarily through 14 C.F.R. Parts 119, 121, 135, and 139. 

[12] NTSB officials said that they currently collect data on accidents 
in the ramp area that meet the definition of an aircraft accident as 
defined by 49 C.F.R. Part 830.2. That is, "an occurrence associated 
with the operation of an aircraft which takes place between the time 
any person boards the aircraft with the intention of flight and all 
such persons have disembarked, and in which any person suffers death 
or serious injury, or in which the aircraft receives substantial 
damage." 

[13] Under the Occupational Safety and Health Act (OSH Act), OSHA has 
statutory authority to govern the occupational safety and health of 
employees. (29 U.S.C. §§651 et seq.) According to OSHA officials, the 
agency investigates ramp accidents when they involve fatalities or the 
hospitalization of three or more employees and conducts workplace 
inspections in response to complaints from workers. (OSHA conducts its 
work pursuant to the 1970 Occupational Safety and Health Act and the 
general industry safety and health standards outlined in 29 C.F.R. 
Part 1910; however, neither contains provisions that pertain 
specifically to the aviation industry.) According to a 2000 memorandum 
of understanding between OSHA and FAA relating to coordination and 
enforcement of the OSH Act, OSHA does not investigate accidents 
involving crew members on aircraft in operation. 

[14] Pilot deviations are investigated by Flight Standards; 
operational errors or deviations are investigated by ATO; and vehicle 
or pedestrian deviations are investigated by the Office of Airports. 

[15] FAA is following the framework established by ICAO for a safety 
management system. ICAO's guidance for establishing such a system can 
be found at [hyperlink, http://www.icao.int/anb/safetymanagement/]. 

[16] FAA officials told us that after FAA adopted ICAO's definition of 
runway incursions in 2007, the agency converted data for the most 
serious incidents from 2001 to 2008 to allow for trending analyses. 

[17] FAA is in the process of bringing information from ATQA into a 
new system for terminal area data collection called Comprehensive 
Electronic Data Analysis and Reporting (CEDAR). 

[18] For information on improvements made by the agency before 2007, 
see [hyperlink, http://www.gao.gov/products/GAO-08-29]. 

[19] FAA National Policy, Order 7050.1A, Runway Safety Program, 
effective September 16, 2010. This order is currently being revised by 
FAA and was expected to be final in September 2011. 

[20] As a joint government-industry body, the Runway Safety Council 
consists of officials from FAA and various industry organizations, 
such as the National Air Traffic Controllers Association, the Air Line 
Pilots Association, the Air Transport Association, Airports Council 
International, the Aircraft Owners and Pilots Association, and the 
National Association of Flight Instructors. 

[21] General aviation encompasses all civil aviation except scheduled 
passenger and cargo operations (i.e., commercial) and excludes 
military operations. It includes air medical-ambulance operations, 
flight schools, corporate aviation, and privately owned aircraft. 

[22] Upgraded markings were installed at all Part 139 airports. (Part 
139 airports are required to have FAA-issued operating certificates 
and include airports that serve unscheduled aircraft with more than 30 
seats and scheduled aircraft with more than 9 seats. See 14 C.F.R. 
Part 139). 

[23] FAA has also started an initiative to require all general 
aviation airports to do a wildlife hazard assessment or have a site 
visit from a qualified wildlife biologist. 

[24] Upon implementation of quality assurance and quality control 
directives later this year, FAA will begin collecting more information 
about excursions using an interim definition of "any instance in which 
an aircraft unintentionally maneuvers off the runway or taxiway." 

[25] FAA's Office of Airports has issued guidance to track some 
excursions, obtaining information from their regional offices, in 
order to learn about airport surface issues (such as debris or 
friction problems in the runway area) that the Office of Airports 
would need to work on with individual airports. 

[26] See 75 Fed. Reg. 62008, October 7, 2010. 

[27] An airport safety management system would be required throughout 
the airport environment, including in movement areas, such as runways 
and taxiways, and in nonmovement areas, including ramps. 

[28] See 76 Fed. Reg. 5510, February 1, 2011. 

[29] See [hyperlink, http://www.gao.gov/products/GAO-08-29]. 

[30] For more information about the Flight Safety Foundation's Ground 
Accident Prevention program, see [hyperlink, 
http://flightsafety.org/archives-and-resources/ground-accident-
prevention-gap/gap-ramp-operational-safety-procedures]. 

[31] For more information about the International Air Transport 
Association's Safety Audit for Ground Operations, see [hyperlink, 
http://www.iata.org/ps/certification/isago/pages/index.aspx]. 

[32] The Port of Seattle, which manages Seattle-Tacoma International 
Airport, is requiring all airlines and ground handling companies at 
the airport to be certified with the International Air Transport 
Association's Safety Audit for Ground Operations by the end of 2011. 
According to the audit's registry, as of August 23, 2011, several 
individual companies have also been certified at individual airports, 
including LAN Airlines, S. A. in Miami, Florida, and Menzies Aviation 
in San Francisco and San Jose, California. 

[33] FAA's Aviation Safety Action Program encourages voluntary 
reporting of safety issues and events that come to the attention of 
employees of certain certificate holders by providing protection from 
legal or disciplinary action by FAA or others. 

[34] According to ATSAP officials, reports into ATSAP can also include 
procedural or equipment issues with potential safety impacts, such as 
unclear guidance or malfunctioning radar. 

[35] According to FAA officials, ATSAP does not fulfill reporting 
requirements for events that require immediate attention or events for 
which the controller involved is acting as a first level supervisor. 
For these situations, the event must also be reported directly to FAA. 

[36] Controllers are required to report any occurrence that may be an 
operational deviation, operational error, proximity event, or air 
traffic incident if the reported issue is known only to the employee 
and occurs while the employee is directly providing air traffic 
services to aircraft or vehicles or first level watch supervision. 

[37] Officials stated that, prior to 2010, FAA included facility-level 
targets for safety incidents in an effort to meet larger, agency-wide 
performance goals. 

[38] See GAO, Aviation Safety: Information on FAA's Data on 
Operational Errors at Air Traffic Control Towers, [hyperlink, 
http://www.gao.gov/products/GAO-03-1175R] (Washington, D.C.: Sept. 23, 
2003). 

[39] Some incidents identified as losses of separation by the TARP 
system involve aircraft under visual separation rules, meaning that an 
operational error did not occur. 

[40] According to FAA officials, FAA will fully implement TARP at 
selected facilities and phase in additional facilities every 30 days 
until full deployment is reached. During this period, FAA will 
evaluate workload and resources to determine whether adjustments to 
implementation are necessary. 

[41] Eurocontrol is an international organization that coordinated and 
plans air traffic control for all of Europe. 

[42] LoSS consists of radar-based operation under the control of air 
traffic operations that results in less than the required separation 
between two or more airborne aircraft. 

[43] TCAS is an aircraft collision avoidance system required by the 
FAA to reduce the incidence of midair collisions between aircraft. 
TCAS operates independently from the ground-based air traffic control 
through transponders installed in aircraft and provides traffic alerts 
and directions to pilots to avoid conflicting aircraft. 

[44] FAA officials stated that the agency will continue to categorize 
operational errors by severity as it transitions to the new, risk-
based process. 

[45] Proximity events are not considered operational errors and are 
therefore not assigned severity ratings nor included in ATQA. 

[46] See [hyperlink, http://www.gao.gov/products/GAO-08-29]. 

[47] Department of Transportation IG, Review of FAA's Call to Action 
Plan for Runway Safety, AV-2010-071 (Washington, D.C.: July 21, 2010). 

[48] Department of Transportation IG, FAA's Process for Reporting and 
Investigating Operational Errors, AV-2009-045 (Washington, D.C.: Mar. 
24, 2009). 

[49] FAA defines category A as a serious incident in which a collision 
was narrowly avoided; category B as an incident in which separation 
decreases and there is a significant potential for collision, which 
may result in a time critical corrective/evasive response to avoid a 
collision; category C as an incident characterized by ample time 
and/or distance to avoid a collision; and category D as an incident 
that meets the definition of runway incursions but with no immediate 
safety consequence. 

[50] NTSB investigates excursions that meet the definition of aircraft 
accident as specified in 49 C.F.R. Part 830.2. 

[51] NTSB investigates ramp events that meet the definition of 
aircraft accident as specified in 49 C.F.R. Part 830.2. 

[52] FAA revised the method of categorizing operational errors in the 
third quarter of fiscal year 2007 to use severity conformance ratings 
of A, B, or C. As a result, we used the enactment date of this 
definition change as the start date for our analyses of airborne 
operational errors. 

[53] According to FAA officials, the waiver was intended to allow the 
South California TRACON facility to reduce standard separation 
requirements to less than 1,000 feet or 3 miles between aircraft on 
the final approach to Los Angeles International Airport and Hawthorne 
Municipal Airport for aircraft cleared for an Instrument Landing 
System approach. Officials at the TRACON erroneously also applied the 
waiver to aircraft on a visual approach for landing. 

[54] Incident severity as recorded in ATQA is based on proximity. 
Category A operational errors are those in which greater than 66 
percent of required separation is lost, category B operational errors 
are those in which between 25 and 66 percent of required separation is 
lost, and category C operational errors are those in which between 10 
and 25 percent of required separation is lost. Losses of separation in 
which 0 and 10 percent of required separation is lost are 
characterized as proximity events by FAA and are not considered 
operational errors. 

[55] FAA performance targets for operational errors were established 
in its 2009 to 2013 Flight Plan and are target rates for all air 
traffic control activities. FAA's target rates are not specific to the 
terminal area. 

[56] SRER represents the rate of radar-tracked losses of separation. 
The equation used to calculate the SRER is [the "number of serious 
LoSS events" divided by the "total number of LoSS events multiplied by 
1000"]. The SRER is a cumulative function that is always presented as 
a rolling 12-month average. A 12-month rolling average prevents 
drastic shifts in the rate from month to month; however, large changes 
to either number used to calculate the rate can result in significant 
shifts that can be reflected in the rate for many months. 

[57] High risk events are Losses of Standard Separation in which both 
the severity and the likelihood of the incident occurring again 
receive high scores within the Risk Analysis Process (RAP). 

[58] The Department of Transportation IG is currently conducting an 
audit of FAA's LoSS Index and the SRER. 

[59] The SRER baseline target was developed using 6 months of LoSS 
data assessed through RAP. This baseline target is intended to be used 
as the process matures; allowing more data to be collected and to 
establish a more realistic target. 

[60] When the minimum separation standard is violated in en route 
airspace, the air route traffic control center's computer system 
detects it. This technology has not significantly changed in recent 
years. 

[61] Before the implementation of ATSAP, air traffic controllers were 
required to report operational errors through ATQA and controllers 
could face punitive consequences including decertification if they 
were found to have committed errors. 

[62] Reports filed into ATSAP are reviewed by a committee comprised of 
representatives from the union representing air traffic controllers, 
FAA management, and FAA aviation oversight. Controllers whose reports 
are accepted into the program are granted protection from punitive 
action on the part of FAA. Reports of events involving apparent non 
compliance with applicable air traffic control directives that are not 
inadvertent or that involve gross negligence, substance abuse, 
controlled substances, alcohol, intentional falsification, or criminal 
activity are excluded from the program. 

[63] As mentioned previously, air traffic controllers can satisfy 
reporting requirements by filing an ATSAP report, in lieu of informing 
their supervisors about incidents for inclusion in ATQA. 

[64] The rate of errors that are unknown to FAA has decreased since 
the launch of ATSAP. 

[65] Joanne Landry, ACRP Synthesis 29: Ramp Safety Practices 
(Washington, D.C.: Transportation Research Board of the National 
Academies, forthcoming). 

[66] GAO, Aviation Safety: Improved Data Quality and Analysis 
Capabilities Are Needed as FAA Plans a Risk-Based Approach to Safety 
Oversight, [hyperlink, http://www.gao.gov/products/GAO-10-414] 
(Washington, D.C.: May 6, 2010). 

[67] As we noted earlier, FAA is currently assessing a low-cost ground 
surveillance system but has not made any decisions as to whether it 
will install this system and at which airports. 

[68] FAA officials noted that data on LoSS events that are not 
currently assessed through RAP will continue to be collected 
electronically and used for trending and analysis purposes. 

[69] According to officials, FAA may have difficulty assessing all 
incidents captured by TARP. Such assessment will be necessary as some 
of the losses of separation identified by TARP may not be operational 
errors. For example, if an air traffic controller is using visual 
separation between aircraft, the proximity of aircraft captured by 
TARP may actually be appropriate, despite appearing to be a loss of 
separation. 

[70] The Department of Transportation IG is currently conducting an 
audit of FAA's implementation of ATSAP. 

[71] ATSAP information is shared with ASIAS, but incident reports do 
not include information that would identify controllers. 

[72] ATSAP Event Review Committees review controller reports to ATSAP 
and assign a severity rating to each report. Possible severity 
ratings, in order from least to most potential risk are: (1) none, (2) 
minimal, (3) minor, (4) major, (5) hazardous, and (6) catastrophic. 

[73] Department of Transportation IG, Air Traffic Control: Potential 
Fatigue Factors, AV-2009-065 (Washington, D.C.: June 29, 2009). 

[74] See [hyperlink, http://www.gao.gov/products/GAO-08-29]. 

[75] Tower operations include all takeoffs and landings, including 
flights that take off and land at the same airport, and all flights 
that fly through the airspace and are tracked by the tower without 
landing. 

[76] AMASS is essentially the safety logic, which is designed to 
detect potential collisions, for ASDE-3. This combined technology is 
usually referred to as ASDE-3/AMASS and was originally implemented at 
34 airports. All but 9 of these locations now have ASDE-X. 

[77] According to FAA officials, the agency selected airports for ASDE-
X deployment after an analysis of site-specific safety and efficiency 
benefits as compared to site-specific costs. The analysis determined 
that maximum benefit was achieved by deploying ASDE-X capability to 
airports with larger traffic counts or more complex operations (e.g., 
airports using the same runways for arrivals and departures). 

[78] We obtained the incursion data from the FAA Office of Runway 
Safety and the operations data from the FAA Operations Network 
database. 

[79] We estimated the model by maximizing the Poisson likelihood 
function conditioned on the sum of the incursions for each airport 
over time. Although the conditional ML estimator makes the airport 
fixed effects possible, the method requires variation over time in the 
number of incursions, and thus we exclude 39 airports that had zero 
incursions in each period when using all FAA-towered airports as the 
comparison group. 

[80] The variances of the model parameters are robust, "sandwich" 
estimates, clustered by airport. The 485 airports in the panel ensure 
that these estimates will be accurate approximations, even if 
incursions are not Poisson-distributed. As Wooldridge 2003 (674-675) 
notes, conditional ML estimators consistently estimate the parameters 
of a fixed effects model, even with arbitrary forms of over-and under- 
dispersion, heteroskedasticity, and serial correlation. As a result, 
we can safely use the Poisson conditional likelihood to estimate the 
parameters while using cluster-robust standard errors. 

[81] Negative binomial models produced similar estimates to those in 
table 5. The estimates varied by no more than 2.6 percentage points, 
except that the estimate for the 100 busiest airports without AMASS 
was 7.2 percentage points smaller. 

[82] Several stakeholders we interviewed agreed with this 
interpretation. Officials from the Sensis Corporation, which developed 
ASDE-X, agreed that the system may produce better reporting of 
incursions by air traffic controllers. Similarly, officials at Seattle-
Tacoma International Airport said pilots and ground crews at that 
airport have become more likely to report their own incursions, 
because they expect controllers to identify them using ASDE-X. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: