20120112_DCP_DCIT_12_DCL_Datalink_Pilot_Resp_Time.pptx

DCL Datalink Pilot Response Time:
Juliana Goh and John Gonda
January 2012
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Facilities used in DCL Data Collection
and Validation
RCAT
MITRE Bedford
(ACTP Data)
IDEA Lab
MITRE CAASD
(PORT Data)
RCP
Validation
2
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Overview
• Phase 0: Data Comm Avionics IDEA Lab Operational
Validation Proposal approved by FAA DCP PM
– Lab Modification Requirements to support Data Comm
Today
• Phase 1: Datalinked Departure Clearance: Pilot Operational
Reaction Time (PORT) HITL with FANS scenario
• Phase 2a. POA vs AOA FANS for DCL
– MITRE Bedford RCAT data collection
Today
• Phase 2b: Follow-on IDEA Lab DCL Validation studies
– Examine results and investigate observations from Phase 1
– Use of UM 79, 83 for revisions
– Non-integrated scenario
• Phase 3 TBD
– Controller RCP study or Enroute RCP
3
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
RCAT Avionics Configuration
• Flight Management Computer
(FMC)
–
GE/Smiths
(ARINC 702)
•
P/N: 2907A4
•
Integrated FANS architecture
• Communication Management
Unit (CMU)
•
P/N: 822-1239-151
•
100% of AOA data
•
50% of POA data
Honeywell Mark II
(ARINC 724)
•
P/N: 965-0758-001
•
50% of POA data
• Very High Frequency (VHF) Data
Radio (VDR)
–
Honeywell VDR
(ARINC 750)
•
P/N: RTA44D-06728
•
100% AOA and POA
For Internal MITRE Use
FMF
INMARSAT/
IRIDIUM
MCDU
Data
Link
App
CMF
Router
–
Rockwell Collins CMU-900
(ARINC 758)
Honeywell 3000i/
ICS-120A
Database
–
VHF
(POA/VDL2)
Honeywell
RTA-44D
Rockwell CMU-900/
Honeywell MKII
High Frequency
Data Radio
HFDR
Honeywell
XK-516
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
MITRE Aviation IDEA Lab
• Boeing 777-like Implementation
• Cockpit Simulation
• En Route Simulation
• Terminal Simulation
Synchronized
visual capability
• Tower/Airport Sim.
5
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Objectives of HITL Simulation
• Main: Determine pilot response time to datalinked
departure clearances under nominal and nonnominal conditions (objective measure)
• Secondary: Preliminary examination of pilot
procedures when receiving a datalinked
departure clearance (observation/questionnaire)
• Secondary: Exploration of issues related to flight
crew receipt of datalinked departure clearances
(observation/questionnaire)
6
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Main MITRE HITL Objective
• Identify actual pilot reaction time through a significant
number of FANS lab transactions using a variety of
certified pilots, scenarios and airports
RCP: Required Communication Performance
RCTP: Required Communication Technical Performance
TRN: Monitored Transaction Time
Exchange Transaction Diagram
7
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Test Variables
• Type of clearance
– With or without runway assignment and departure
procedure (DP)
• Complexity of clearance revision
– Simple or complex
• Simple: change in squawk or initial alt or flight level change
• Complex: change in routing
• Aircraft location during receipt of clearance revision
– Non-movement or movement area
8
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Test Environment
• 737-like interface
– Selected because it presents the “worst case” situation
– All data communications done through the CDU
• Full flight deck with 180 degree view of airport
environment
• Radio chatter to simulate operational environment
carried out by confederates
9
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Test Design
With R/W
assignment
& departure
procedure
Without R/W
assignment
& without
departure
procedure
SEGMENT 1
SEGMENT 2
SEGMENT 3
At the gate
DCL Revision ramp area
(non-movement area)
DCL Revision during taxi
(movement area)
Initial DCL
Simple
Initial DCL
Complex
S1
Complex
Simple
S2
Initial DCL
Simple
Complex
S3
Initial DCL
Complex
Simple
S4
12 Crews ran through all 4 scenarios
10
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Test Procedure
• Training
– Classroom briefing on datalink procedures
– Training in the flight deck
• General familiarization of flight deck
• Three training scenarios to put procedures into action
• Crews trained to criterion: completion of 2 scenarios (6 datalink
messages) correctly without intervention from experimenter
• Test
– 4 scenarios counterbalanced
– Response time from “• ATC” to pilot responding with
“ACCEPT”
– Questionnaire after each scenario and after all 4 scenarios
11
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Training - Datalink Procedure
• Send request
• Receive message
• Read/understand message
• Input elements of message into airplane avionics
• Verification checks of data input
• Accept message
12
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Training - Datalink Procedure









13
Loading and Verifying an FMS uplinked DCL
Load…………………………………………………Select
LOAD prompt automatically updates FMC RTE
page with route clearance information.
FMC RTE ………………………………………….Select
Verify ……………………………………DEPT Airport
Verify ……………………………………DEST Airport
Verify …………………………………………..Runway If
part of the Uplinked Clearance, otherwise this
is a manual entry
Verify ………………………………………….FLT NO: For
Example:FAA123
Verify………………………………….Cleared Route
The Departure Procedure and Runway may
require manual entry
Activate (not in sim)……………………………..Select
EXEC ……………………………………………….Select to
complete the RTE page upload
For Internal MITRE Use





LEGS Page / Distance check for Uplinked DCL
LEGS ……………………………………………….Select
PLN Mode………………………………………Select
Step through the legs page using “Step” at 6R
and observe on the Navigation Display:
o Waypoints and Altitude constraints agree
with those on your filed flight plan and
navigation charts, and
o No discontinuities exist between
waypoints
Verify that an active waypoint is depicted in
1L on LEGS page 1
Select PROG page
Verify the total route distance is proper for
route of flight versus your filed flight plan
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Other Flight Deck Tasks
• Flight crews instructed to use standard crew
resource management principles
– Define pilot flying/pilot not flying roles
– Communication between pilot flying/pilot not flying
• Complete checklists (from BEFORE PUSHBACK up
to BEFORE TAKEOFF)
• Communicate via voice with ramp, ground and
tower for pushback and taxi clearances in the nonmovement and movement area
14
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Participant Profile
• 11 crews (22 pilots)
• All air transport pilots, across various carriers
• Ratings: Emb145/190, CRJ, B737/757/767/777,
Airbus 319/320
• Total flight hours: range from 1375 to 25000 hrs
(mean = 10000hrs)
15
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Scenarios
• 4 Scenarios
– 2 at KDFW
– 2 at KMIA
• Initial condition:
– preflight complete
– begin with preflight CDU (input weights, V speeds etc) and
flight plan information
• End point:
– aircraft lined up and waiting on the runway
16
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Typical Scenario
SEGMENT ONE: AT THE GATE
Request DCL
Complete datalink procedure
Complete PUSHBACK checklist
Request pushback (voice)
PUSHBACK
Assume two good engine starts
AFTER START checklist
Request taxi clearance (voice)
CAASD 425
Cleared to spot 6
Contact ground
SEGMENT 2
NON-MOVEMENT AREA REVISION
Simple or Complex revision
17
For Internal MITRE Use
BEFORE/AT SPOT
Complete datalink procedure
Request taxi to runway (voice)
CAASD 425
Cleared to RW17R
via EK L EH
SEGMENT 3
MOVEMENT AREA REVISION
Simple or Complex revision
DURING TAXI
Complete datalink procedure
Complete BEFORE TAKEOFF
Checklist
Contact tower
CAASD 425
Line up and wait
RW17R
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Response Time Distributions
(Simple Messages)
Distribution of Response TImes to Simple
Messages
0.4
0.35
Probability
0.3
0.25
0.2
0.15
0.1
0.05
0
20
40
60
80
100
120
140
160
180
200
220
240
260
280
Response Times
10th Percentile
Mean
95th Percentile
99th Percentile
99.9th Percentile
18
For Internal MITRE Use
Simple
(42 data points)
12.08
31.75
67.02
86.46
89.72
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Response Time Distributions
(Complex Messages)
Distribution of Response Times to Complex
Messages
0.3
Probability
0.25
0.2
0.15
0.1
0.05
0
20
40
60
80
100
120
140
160
180
200
220
240
260
280
Response Times
10th Percentile
Mean
95th Percentile
99th Percentile
99.9th Percentile
19
For Internal MITRE Use
Complex
(85 data points)
55.45
95.24
156.35
256.00
273.62
Note: Response times
to initial messages and
complex revisions were
combined
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Questionnaires/Observations
• Workload: acceptable
• Presentation format: acceptable
• Message content: acceptable
• Heads down time (compared to voice
communication): acceptable
• Procedures
– No noticeable difficulty in flight crews’ ability to learn and
execute the prescribed set of procedures
20
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Questionnaires/Observations
• Other issues:
– Coordination with AOC
– Highlighting only portions of the route that have changed
– Multiple open messages
• Experienced by a couple of crews. Crews appeared confused and
had to be prompted to review the log page.
• Will flight crews experience multiple open messages in the actual
implementation?
– Not being able to view route before loading into CDU
21
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Summary
• Response time:
– Complex clearance: averaged 95.24 secs
– Simple clearance: averaged 31.75 secs
• Observations/questionnaires:
– Datalink preferred over voice communications
– Prescribed procedures easy to learn and execute
– Flight crew workload, message format and content acceptable
22
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Next Steps
• HITL 2 definition:
– Builds off HITL 1
• Examine use of additional messages UM79 and UM83
• Examine use of integrated and non-integrated avionics in
delivering departure clearances
– Allows us to understand how the flight crew manages
the resources on the flight deck (examples ..)
• Measure response time
• Streamline cockpit procedures
23
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Next Steps
• Schedule:
– Andy Fry and Brent Blackwell developed
messages/scenarios in November 2011
– Data collection started 1/6/2011, finishes end of January
– Preliminary report in March, Final report end of April
24
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Backup Slides
25
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Response Time Confidence Intervals
Initial
With DP*
Simple
Complex
With DP
Without DP
Without DP
With DP
Without DP
NM
M
NM
M
NM
M
NM
M
44.07
17.78
30.38
34.93
97.37
69.63
93.99
125.78
95% CI [75.80, 101.63] [85.79, 115.64]
[28.54, 59.59]
[12.08, 23.52]
[19.29, 41.46] [26.21, 43.64] [76.57, 118.18] [59.59, 79.66]
[57.77, 130.19] [86.93, 164.62]
99% CI [71.85, 105.58] [81.22, 120.21]
[23.78, 64.35]
[10.33, 25.77]
[15.90, 44.85] [23.54, 46.31] [70.20, 124.54] [56.52, 82.73]
[46.69, 141.27] [75.04, 176.51]
Mean
88.715
100.715
* Departure procedure and runway assignment
Range across all crews and all conditions
(11 crews x 12 observations = 132 data points - 5 missing data points = 127 data points):
Minimum = 10 seconds
Maximum = 275.57 seconds (4 min 35.57 sec)
Case with 275.57 seconds is an outlier
26
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Response Time Confidence Intervals
95% Confidence Interval
200.00
180.00
Response Time
160.00
140.00
125.77
120.00
100.72
100.00
97.37
93.98
88.71
80.00
69.63
60.00
44.07
40.00
34.93
30.38
20.00
17.80
0.00
NM
With DP
Without DP
Initial
M
NM
With DP
M
Without DP
Simple
NM
M
NM
With DP
M
Without DP
Complex
Condition
27
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Response Time Confidence Intervals
99% Confidence Interval
200.00
180.00
Response Time
160.00
140.00
125.77
120.00
100.72
100.00
97.37
93.98
88.71
80.00
69.63
60.00
44.07
40.00
34.93
30.38
20.00
17.80
0.00
NM
With DP
Without DP
Initial
M
NM
With DP
M
Without DP
Simple
NM
M
NM
With DP
M
Without DP
Complex
Condition
28
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Response Time Confidence Intervals
Initial
Simple
With DP*
Without DP*
(n=11)
(n=11)
Complex
With DP*
Without DP*
(n=11)
(n=11)
With DP*
(n=11)
Without DP*
(n=11)
Mean
88.72
100.72
30.93
32.65
83.50
109.88
95% CI
[75.80, 101.63]
[85.79, 115.64]
[21.81, 40.05]
[25.30, 40.01]
[72.36, 94.64]
[78.82, 140.94]
99%CI
[71.85, 105.58]
[81.22, 120.21]
[19.02, 42.85]
[23.04, 42.26]
[68.95, 98.05]
[69.3, 150.44]
* Departure procedure and runway assignment
Range across all crews and all conditions (11 crews x 12 observations = 132 data points):
Minimum = 10 seconds
Maximum = 275.57 seconds (4 min 35.57 sec)
Case with 275.57 seconds is an outlier
Next longest response time = 157.50 seconds (2 min 37.5 seconds)
29
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Response Time Confidence Intervals
95% Confidence Intervals for Response Time
180.00
160.00
140.00
Response Time
120.00
109.88
100.72
100.00
88.71
83.50
80.00
60.00
40.00
32.65
30.93
20.00
0.00
With DP
Without DP
Initial
With DP
Without DP
Simple
With DP
Without DP
Complex
Condition
30
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Response Time Confidence Intervals
99% Confidence Intervals for Response Time
180.00
160.00
Response Time
140.00
120.00
109.88
100.72
100.00
88.71
83.50
80.00
60.00
40.00
30.93
32.65
20.00
0.00
With DP
Without DP
Initial
With DP
Without DP
Simple
With DP
Without DP
Complex
Condition
31
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
This is the copyright work of The MITRE Corporation and was produced for the U.S. Government
under Contract Number DTFAWA-10-C-00080 and is subject to Federal Aviation Administration
Acquisition Management System Clause 3.5-13, Rights in Data-General, Alt. III and Alt. IV (Oct.
1996). No other use other than that granted to the U.S. Government, or to those acting on behalf of
the U.S. Government, under that Clause is authorized without the express written permission of The
MITRE Corporation. For further information, please contact The MITRE Corporation, Contract
Office, 7515 Colshire Drive, McLean, VA 22102, (703) 983-6000.
The contents of this material reflect the views of the author and/or the Director of the Center for
Advanced Aviation System Development, and do not necessarily reflect the views of the Federal
Aviation Administration (FAA) or Department of Transportation (DOT). Neither the FAA nor the DOT
makes any warranty or guarantee, or promise, expressed or implied, concerning the content or
accuracy of the views expressed herein.
2011 The MITRE Corporation. The Government retains a nonexclusive, royalty-free right to publish
or reproduce this document, or to allow others to do so, for “Government Purposes Only.”
For Release to All FAA. This document was
prepared for authorized distribution only.
It has not been approved for public release.
Fiscal Year:
Outcome Number:
PBWP Reference:
33
For Internal MITRE Use
© 2011 The MITRE Corporation. All rights reserved.
Document Number Here
Similar pages