Transcript Slide 1

GA JSC SAT and Working
Group Processes
Corey Stephens
Co-Chair, GA JSC SAT
GA JSC SAT Meeting
March 22, 2011
Washington, DC
GA Joint Steering Committee
• Evolve GA JSC to a CAST like Model
–
–
–
–
Voluntary commitments
Consensus decision-making
Data driven risk management
Implementation-focused
• The GA JSC is a means to…
Focus Limited Government/Industry
Resources on Data Driven Risks and
Solutions
What is CAST?
• Work began in 1997 after two significant
accidents in 1996 (TWA 800 & ValueJet 592)
• CAST focus was set by:
–
–
White House Commission on Aviation Safety
The National Civil Aviation Review Commission
(NCARC)
• Opportunity for industry and government to
focus resources on one primary aviation
safety initiative
What is CAST?
Vision
•
Key aviation stakeholders acting cooperatively to lead the
world-wide aviation community to the highest levels of global
commercial aviation safety by focusing on the right things.
Mission
•
Enable a continuous improvement framework built on
monitoring the effectiveness of implemented actions and
modifying actions to achieve the goal.
Goal
•
•
Reduce the US commercial aviation fatal accident rate 80% by
2007
and
Maintain a continuous reduction in fatality risk in US and
International commercial aviation beyond 2007.
How CAST Works
CAST Safety Strategy
Data
Analysis
Implement Safety
Enhancements U.S.
Set Safety
Priorities
Agree on
problems and
interventions
Achieve
consensus on
priorities
Influence Safety
Enhancements Worldwide
Integrate into
existing work
and distribute
CAST Safety Strategy
Ongoing
Accident/
Incident/Studies
Master
Contributing
Factors
Incident
Analysis
Process
Emerging/
Changing Risk
Develop/Revise
Enhancements
& Metrics
Performance
To Plan
Review
CAST Plan
Industry/
Government
Action
\
Things to
Watch
Future Changes
Analysis Process
Safer
System
Information on System Performance
10-28-05 CAST-064
CAST Safety Analysis Process
Turbofans Installed on part 25 Aircraft
NTSB Accident
Incident Reports
25
2.
21.3 Reports
ASIAS data
Airclaims data
Historical
Data
20
Level 4
15
Level 3
10
5
3.
4.
Intervention Strategy
Selectove
Appr
Pr oposal
Set
s
AccidTr
entain
/ in
Ig
ncident
Review
Repor t s
Accident / I ncident
Analyze
Repor t s
8. 25d
15d
2
5d
3
1/ 2/30/
10/
9695
1/ 12/
11/
17/96
95
10/ 30/ 95
11/ 3/ 95
11/ 6/ 95
Develop
I nt er vent ion
Pr ior it ized
St r at egies
Pr epar e
Dr af t
Repor t
Revise
11/ 10/ 95
Repor t
2h
6
3d
4
5d
1/ 10/ 96
1/ 2/ 96
1/ 4/ 96
11/ 13/ 95
11/ 17/ 95
Final
Repor t
Cowl separation
Multi-related
Unknown
Threat
Combined
Threat
Causal Analysis
Cause
Cause
JSAT
5d
1/ 10/ 96
Final
Engine separation
Accident
Accident / I ncident
7
Pr epar e
Crew error
JSAT
JSAT
5
1
Case rupture
Pareto Plots
Cause Cause
5.
Multi - unrelated
Fuel
contam./exhaustion
Reverser
Multi - common other
Fire (e.g., under
cowl)
Uncontained - blades
Malfunction + crew
error
JSAT
Multi - common - env.
0
Uncontained disk/spacers
Industry
CAAM Level 3 and 4 Events
1.
Repor t
Appr ove
8
1h
9
1h
Select
I nt er vent ion
St r at egies
10
1d
1/ 11/ 96
1/ 11/ 96
1/ 11/ 96
1/ 11/ 96
1/ 11/ 96
1/ 12/ 96
6.
Safer
Skies
Industry
7.
Government
Implementation Strategy
JSIT
AvSP
Measuring Progress to Goal
Coordinated Plan
5.3-23
Commercial Aviation Safety Team (CAST)
CAST
Joint Safety
Analysis Teams (JSAT)
• Data analyses
Joint Safety
Implementation
Teams (JSIT)
• Safety
enhancement
development
Joint Implementation
Measurement Data
Analysis Team (JIMDAT)
• Master safety plan
• Enhancement
effectiveness
• Future areas of
study
CAST Accomplishments
• Forensic analysis of US and world accidents since
1987 (ongoing)
• Industry and government cooperative safety plan:
–
–
–
72 Prioritized Safety Enhancements
50 Complete and 22 underway
Projected 74% fatality risk reduction by 2020
• Development of proactive analytic processes for
incident data
• CAST was the recipient of the 2008 Collier Trophy
For achieving an unprecedented safety level in U.S.
commercial airline operations by reducing risk of a fatal
airline accident by 83 percent, resulting in two
consecutive years of no commercial scheduled airline
fatalities
For this discussion…
GA JSC Groups and their CAST Counterparts
• GA JSC = CAST
• Steering Committee = CAST ExCom
• Safety Analysis Team (SAT) = JIMDAT
• Working Groups (WGs) = JSAT/JSIT
General Aviation Joint Steering Committee (GAJSC)
Steering Committee
Co-Chairs: Bruce Landsberg (AOPA/ASF)
Tony Fazio (FAA/AVP)
Government Industry
-
FAA (AFS, AIR, ATO & ARP)
NASA (Research)
NWS
GAMA, EAA, NBAA, NATA,
& SAMA
Safety Analysis Team
Co-chairs: Corey Stephens (FAA)
Jens Hennig (GAMA)
Members: FAA, NTSB, AOPA, FSF, UAA, CGAR,
FAST, NAFI, Insurance, Academia, SAFE
Working Groups
(To include SMEs from various general
aviation segments, depending on study)
•
•
•
•
•
Strategic guidance
Management/Approval of Safety Plan
Provide direction
Membership Outreach
Provides linkage to ASIAS
•
•
•
•
•
•
Identify future areas of study/risk
Charter safety studies
Provide guidance and direction
Draw data from various areas
Develop a prioritized Safety Plan
Develop metrics to measure
effectiveness of safety solutions
• Data analyses
• Safety enhancement
• Mitigation development
Aviation
System
GAJSC Safety Strategy
GAJSC
GA Safety Plan
SAT
Amend
Safety Plan
WG
Establish
SAT
Approves Priority
/ Assigns
Resources to WG
Approves
Proposed
Mitigations
Industry
Action
Approves DIP &
Assigns Industry
Government
Responsibility
Monitor
Effect
Government
Action
Accident
Area
Proposed
Ongoing
Fatal Accident
Studies
Review
Proposed
Mitigations
Evaluate
Cost &
Benefit
Accident
Selection
NASA
FAA
Detailed Accident
Review and Propose
Mitigations
Develops Detailed
Implementation
Plans (DIP)
Develop, Revise &
Monitoring of Metrics
 Time 
NTSB
ASIAS
Academia
Identification of
System Changes
Manufacturers
Pilots
General Aviation NAS
Safety State
08-16-2011 GAJSC
GA JSC Working Group
Process
Step 1: Analysis
Typical CAST JSAT Membership
•
•
•
•
•
•
ALPA/APA
FAA (AIR, AFS, ASA, AAI,
ATO)
Airbus
EASA
ATA
Transport Canada
•
•
•
•
•
•
•
NASA
Engine companies – (PW, GE,
RR-Allison)
Boeing
RAA
NACA
AIA
NATCA
GA JSC WG Process
Charter
Development
Evaluate
Problem
Importance
Global
Review of
Characteristics/
Indicators
Assign
Standard
Problem
Statements
Identify
Intervention
Strategies
Establish
Team
Identify
Problems
(what/why)
Evaluate
Intervention
Effectiveness
Select
Data Set
Develop
Event
Sequence
Prioritize
Interventions
Review
Data
Record
Characteristics/
Indicators
Technical
Review &
Report
Results
5.5-24
Developed Event Sequence
• Facts and data
• Pilot - controller voice events
• Missed calls
• Events that occurred or should have
• Time coded each event
#
1015
Time
21:53:28
1016
1017
1018
1019
21:53:28
21:53:32
21:53:33
Event
ATC issued ATIS information Sierra: Ceiling
100’ overcast, 1/2 mile visibility and fog
F/O call 200’ above minimums
F/O calls ATC to report Marker Inbound
F/O call out 100’ above minimums
F/O fails to call out “runway not in sight” at the
minimums for the Decision Height
Develop Problem Statements
• Problem statements
–
–
–
–
What went wrong
Deficiency definition
Potential reason
Something which happened or didn’t happen
# Time
Event/Data Point
8:53:00 Aircraft took off
from Taipei Intl
1
Airport
10:45:00 F/O
briefed CAPT
2
on approach into
10:49:00 Capt gave very
basic guidance to
the F/O on aircraft
control during
approach and
landing.
3
Problem
(What)
Contributing
Factors
(Why)
F/O was
inexperienced; his
actions were not
commensurate with
1034 hours in type.
It is not normal practice
at China Airlines for
Capt and F/O to rotate
takeoffs and landings.
The FO is required to fly
aircraft "in t/o and
landing phases at least
3 times every 3 months"
(3-28) (airline culture)
Sample Standard Problem Statements
CAST Examples
•
10 FLIGHTCREW – Failure of flight crew to follow established procedures
(SOP)
•
39 AIRCRAFT EQUIPMENT – DESIGN NOT ERROR TOLERANT System
design does not provide adequate redundancy to counteract errors or
alerting of the effects of errors
•
44 FLIGHTCREW – Flight crew failure to recognize and correct unstable
approach
•
100 REGULATORS – INSUFFICIENT AIR CARRIER OVERSIGHT .
Insufficient regulatory oversight of air carrier operations including
management and training practices
Identify Intervention Strategies
• Intervention strategies
– Suggested solutions
– Things to do to prevent or mitigate the problem
– Etc.
# Time
Event/Data Point
8:53:00 Aircraft took off
from Taipei Intl
10:45:00 Airport
F/O briefed CAPT
2
on approach into
10:49:00 Capt gave very
basic guidance to
the F/O on aircraft
control during
approach and
landing.
Problem
(What)
Contributing
Factors
(Why)
F/O was
inexperienced; his
actions were not
commensurate with
1034 hours in type.
It is not normal practice
at China Airlines for
Capt and F/O to rotate
takeoffs and landings.
The FO is required to fly
aircraft "in t/o and
landing phases at least
3 times every 3 months"
(3-28) (airline culture)
Standard Problem
Statement
A
P1
1
3
20 AIRLINE
OPERATIONS - LACK
OF TRAINING
(FLIGHTCREW)
3
5
414 Airline operations – 4
training failed to
adequately develop
FIRST OFFICER
piloting skills. (SPS-20)
3
Intervention Effectiveness
• Power
–
Effectiveness of a specific intervention in reducing the likelihood that
a specific accident would have occurred (“Perfect World”)
• Confidence
–
Confidence that this specific intervention will have the desired effect
• Future Global Applicability
–
How well the intervention can be extrapolated to apply to a worldwide fleet in the future
Effectiveness Rating Scales
POWER
This scale is to be used to judge the effectiveness of a specific intervention in reducing the likelihood that a specific accident would
have occurred had the intervention been in place and operating as intended. (“perfect world”)
0
Not at all
effective
1
2
3
4
5
6
Hardly any
effect
Slightly
effective
Moderately
effective
Quite
effective
Highly
effective
Completely
effective
CONFIDENCE
This scale is to be used to define the level of confidence that you have that this specific intervention will have the desired effect.
0
Not at all
confident
1
Hardly any
confidence
2
Slightly
confident
3
Moderately
confident
4
Quite
confident
5
6
Highly
confident
Completely
confident
FUTURE GLOBAL APPLICABILITY
This scale is to be used to estimate how well the intervention can be extrapolated to apply to a world-wide fleet in the future.
(for example: how often the situation it addresses occurs in accident scenarios; whether its impact is on present and future
operations (equippage, traffic, regulatory differences); and whether it is applicable across airlines/airplanes/regions.
0
Not at all
applicable
1
Hardly any
applicable
2
Slightly
applicable
3
Moderately
applicable
4
Quite
applicable
5
Highly
applicable
6
Completely
applicable
GA JSC Working Group
Process
Step 2: Implementation
GA JSC Feasibility Scales
•
•
•
•
•
•
Technical
Financial
Operational
Schedule
Regulatory
Sociological
GA JSC Safety Enhancements
• Develop Safety Enhancements from Interventions
• Collect detailed resource information
• Prepare Detailed Implementation Plans (DIP’s)
GA JSC WG Reports
• Standard Problem Statements
• Interventions Prioritized
• Recommendations
• Detailed Implementation Plans (DIPs)
What’s a DIP?
SE 31
Loss of Control
Joint Safety Implementation Team
Implementation Plan
for
Training - Advanced Maneuvers
Statement of Work
Advanced Maneuvers Training (AMT) refers to training to prevent and recover from hazardous flight
conditions outside of the normal flight envelope, such as, inflight upsets, stalls, ground proximity and
wind shear escape maneuvers, and inappropriate energy state management conditions.
The purpose of this project is to collect and provide advanced maneuver training material and to
encourage Part 121 operators to use these materials to implement advanced maneuver ground training
and flight training using appropriate flight training equipment. Emphasis should be given to stall
onset recognition and recovery, unusual attitudes, upset recoveries, effects of icing, energy awareness
and management, and causal factors that can lead to loss of control.
Additionally, research should be conducted to determine how existing flight simulation devices can
be used effectively in AMT.
Safety Enhancement: (SE-31)
Pilots will be better trained to avoid and recover from excursions from normal flight and loss of
control.
Lead Organization for Overall Project Coordination (LOOPC):
FAA, Flight Standards (AFS)
Score:
2007-(13.0)
2020-(13.0)
100%-(13.0)
Resource Requirements:
FAA AFS-400, Air Transport Association Training Committee, National Air Carrier Association
(NACA), Regional Airline Association, manufacturers, pilot associations, Principal Operations
Inspectors (POI’s), Directors of Safety, flight operations and training departments, NASA, aircraft
manufacturers, flight simulation device manufacturers, training centers, existing training aids, and
other materials.
GA JSC SAT (Safety Analysis Team)
Process
Safety Plan Development
Develops a Prioritization Methodology (GA JSC SAT)
• Identifies the most effective solutions derived from all
accident categories
• Considers effectiveness vs. resources
• Tests solutions against fatal and hull loss accidents
• Creates draft master strategic safety plan
• Identifies areas for future study/mitigation
General Methodology for Calculating the Potential Benefit of a Safety
Enhancing Intervention
Accident
Risk
Reduction
= (
Effectiveness
that an intervention
has for reducing the
accident rate if
incorporated
,
Portion of
world fleet
with intervention
implemented
)
Spreadsheet Example – Historical Airplane Accidents &
Proposed Safety Enhancements – CAST Example
Accident
Date
Class
Location
Aircraft
Type
Description
Accident Description
Portion
of
Intervention Name
EGPWS
CFIT TRN
Accident Portion of World Fleet
Eliminated .600
.900
Intervention Effectiven
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
CFIT
1/2/1988
2/8/1988
2/27/1988
3/17/1988
6/12/1988
7/21/1988
10/17/1988
10/19/1988
2/8/1989
2/19/1989
6/7/1989
7/27/1989
8/25/1989
10/21/1989
10/26/1989
2/14/1990
6/2/1990
11/14/1990
12/4/1990
3/5/1991
8/16/1991
1/20/1992
2/15/1992
3/24/1992
6/22/1992
7/31/1992
9/28/1992
11/25/1992
11/26/1992
4/26/1993
IZMIR, TURKEY
737
LUANDA, ANGOLA
707
KYRENIA MTS, CYPRUS727
CUCUTA, COLUMBIA
727
POSADAS, ARGENTINA MD80
LAGOS, NIGERIA
707
ROME, ITALY
707
AHMEDABAD, INDIA
737
SANTA MARIA AZORES 707
KUALA LUMPUR, MALAYSIA
747
PARAMARIBO, SURINAME
DC8
TRIPOLI, LIBYA
DC10
ANKARA, TURKEY
727
TEGUCIGALPA, HONDURA
727
HUALIEN, TAIWAN
737
BANGALORE, INDIA
A320
UNALAKLEET, ALASKA 737
ZURICH, SWITZERLANDDC9
NAIROBI, KENYA
707
MT LA AGUADA, VENEZUELA
DC9
IMPHAL, INDIA
737
STRASBOURG, FRANCE A320
KANO, NIGERIA
DC8
ATHENS, GREECE
707
CRUZEIRO DO SUL, BRAZIL
737
KATMANDU, NEPAL
A310
KATMANDU, NEPAL
A300
KANO, NIGERIA
707
MANAUS, BRAZIL
707
AURANGABAD, INDIA
737
HIT MOUNTAIN ON APPROACH
.657 .950
HIT ANTENNA ON APPROACH
.586 .800
HIT MOUNTAIN ON APPROACH
.657 .950
HIT MOUNTAIN DURING CLIMB
.657 .950
CRASHED ON FINAL APPROACH
.203 .000
CRASHED ON APPROACH
.203 .000
LANDED SHORT
.203 .000
LANDED SHORT
.586 .800
TERRAIN IMPACT/DESCENT
.657 .950
TERRAIN IMPACT/APPROACH
.657 .950
TERRAIN IMPACT/FINAL APPROACH.203 .000
TERRAIN IMPACT/FINAL APPROACH.203 .000
HIT ILS ANT. ON TAKEOFF
.000 .000
TERRAIN IMPACT/APPROACH
.657 .950
TERRAIN IMPACT/DEPARTURE
.657 .950
HIT SHORT (300 FT)
.203 .000
HIT HILL 7 MILES OUT IN FOG
.657 .950
CRASHED 5 MILES SHORT
.634 .900
HIT POWER LINE ON ILS FINAL
.203 .000
HIT MOUNTAIN/APPROACH
.657 .950
A/C HIT HILL 20 MILES OUT/APPROACH
.657 .950
IMPACTED GROUND/FINAL
APPROACH
.586 .800
CFIT OUT OF PROCEDURE TRN-DARK
.586 .800
ABANDONED APPROACH-HIT MTN
.657 .950
HIT SHORT,DRK NT,DISTRACTED
.203 .000
CFIT-HIT MTN-MISSED APPROACH .657 .950
CRASHED SHORT DURING APPROACH.657 .950
LANDED SHORT MISLEADING LIGHTS
.538 .700
HIT LIGHTS ON TO/RMLG COLLAPSE
.000 .000
HIT TRUCK AFTER TAKEOFF
.000 .000
.226
.226
.226
.226
.226
.226
.226
.226
.226
.226
.226
.226
.000
.226
.226
.226
.226
.226
.226
.226
.226
.226
.226
.226
.226
.226
.226
.226
.000
.000
Basics of the Selection Spreadsheet
• Effectiveness
Each safety enhancements is evaluated against each undesired condition in the set to
determine how effective the enhancement would be at eliminating these conditions if the
enhancement were put in place.
• Implementation
Implementation level is based on the portion of the affected population with the
enhancement incorporated or predicted to be incorporated by a future date.
• Severity Weighting
To account for differences in severity or significance of the undesired conditions, a
weighting value can be entered so that the relative risk of the undesired conditions is realized.
Severity Weighting Overview
CAST Example
• To account for differences in fatality risk associated with each accident in the
data set, a severity value was applied. In this assessment, the severity value
represented the portion of people onboard that perished in the given accident.
• Example: Comparison of two fatal accidents
757 CFIT accident, 98% perished. Weighting factor is .98
747 Turbulence accident, .6% perished. Weighting factor is .006
• Hypothetically assume an assessment showed that the chance of these
accident occurring would have been reduced by proposed safety
enhancements by 50%.
• The associated portion of fatality risk eliminated can be determined
using the severity weighting factor as follows:
757 CFIT.98 x .5 = .49
747 Turbulence, .006 x .5 = .003
Analysis Tool Output
• The spreadsheet output can be set up to show the
effect that an individual safety enhancement, or
group of safety enhancements have on reducing
exposure to the undesired condition.
Fatality
Risk
Reduction
SE1
SE2
SE3
SE1 &
SE2
SE1 &
SE3
Robust CAST Methodology
• Detailed event sequence - problem
identification from worldwide accidents and
incidents
• Broad-based teams (45-50 specialists /team)
• Over 450 problem statements (contributing
factors)
• Over 900 interventions proposed
• Analyzed for effectiveness and synergy
- CAST Safety Enhancements
CAST Process Led to Integrated
Strategic Safety Plan
• Part 121 or equivalent passenger and cargo
operations studied
• Current CAST plan:
• 72 Prioritized Safety Enhancements
• 50 Complete and 22 underway
• Projected 74% fatality risk reduction by
2020
• Industry and Government implementing plan
Resource Cost Vs. Risk Reduction
CAST Example
10000
Risk Reduction
$
APPROVED PLAN
9000
Total Cost in
$ (Millions)
8000
75%
7000
6000
$
50%
2007
2020
5000
4000
3000
25%
2000
0%
$
$
$
1000
0
Resource Cost ($ Millions)
Risk Eliminated by Safety Enhancements
100%
Cost Savings
CAST Example
Part 121 Aviation Industry Cost Due to Fatal/Hull Loss Accidents
100
Historical cost of accidents
per flight cycle
Dollars/Flt. Cycle
80
Savings ~ $74/Flight Cycle
Or
~ $814 Million Dollars/Year
60
40
Cost of accident fatalities
following implementation of the
CAST plan @ 2020 levels
20
0
2007
2020
What the GA JSC can accomplish
• GA accident and incident data drives
direction of GA JSC activities
• GA JSC to charge the SAT with chartering
study groups on specific topics
• Working groups of SMEs formed to identify
risks and develop mitigations
• Mitigations are assessed and prioritized
• A cooperative industry/government GA
safety plan is developed and implemented
GA JSC SAT & WGs - Moving Forward
• History shows focused action and
introduction of new capabilities have led to
accident risk reductions
• Joint industry and government teams working
together to a common goal can further
enhance the safety of our very safe aviation
system
• Full implementation will require a coordinated
effort between industry and government
• The GA JSC is moving forward to meet the
challenge