DCL Datalink Pilot Response Time: Juliana Goh and John Gonda January 2012 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Facilities used in DCL Data Collection and Validation RCAT MITRE Bedford (ACTP Data) IDEA Lab MITRE CAASD (PORT Data) RCP Validation 2 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Overview • Phase 0: Data Comm Avionics IDEA Lab Operational Validation Proposal approved by FAA DCP PM – Lab Modification Requirements to support Data Comm Today • Phase 1: Datalinked Departure Clearance: Pilot Operational Reaction Time (PORT) HITL with FANS scenario • Phase 2a. POA vs AOA FANS for DCL – MITRE Bedford RCAT data collection Today • Phase 2b: Follow-on IDEA Lab DCL Validation studies – Examine results and investigate observations from Phase 1 – Use of UM 79, 83 for revisions – Non-integrated scenario • Phase 3 TBD – Controller RCP study or Enroute RCP 3 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here RCAT Avionics Configuration • Flight Management Computer (FMC) – GE/Smiths (ARINC 702) • P/N: 2907A4 • Integrated FANS architecture • Communication Management Unit (CMU) • P/N: 822-1239-151 • 100% of AOA data • 50% of POA data Honeywell Mark II (ARINC 724) • P/N: 965-0758-001 • 50% of POA data • Very High Frequency (VHF) Data Radio (VDR) – Honeywell VDR (ARINC 750) • P/N: RTA44D-06728 • 100% AOA and POA For Internal MITRE Use FMF INMARSAT/ IRIDIUM MCDU Data Link App CMF Router – Rockwell Collins CMU-900 (ARINC 758) Honeywell 3000i/ ICS-120A Database – VHF (POA/VDL2) Honeywell RTA-44D Rockwell CMU-900/ Honeywell MKII High Frequency Data Radio HFDR Honeywell XK-516 © 2011 The MITRE Corporation. All rights reserved. Document Number Here MITRE Aviation IDEA Lab • Boeing 777-like Implementation • Cockpit Simulation • En Route Simulation • Terminal Simulation Synchronized visual capability • Tower/Airport Sim. 5 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Objectives of HITL Simulation • Main: Determine pilot response time to datalinked departure clearances under nominal and nonnominal conditions (objective measure) • Secondary: Preliminary examination of pilot procedures when receiving a datalinked departure clearance (observation/questionnaire) • Secondary: Exploration of issues related to flight crew receipt of datalinked departure clearances (observation/questionnaire) 6 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Main MITRE HITL Objective • Identify actual pilot reaction time through a significant number of FANS lab transactions using a variety of certified pilots, scenarios and airports RCP: Required Communication Performance RCTP: Required Communication Technical Performance TRN: Monitored Transaction Time Exchange Transaction Diagram 7 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Test Variables • Type of clearance – With or without runway assignment and departure procedure (DP) • Complexity of clearance revision – Simple or complex • Simple: change in squawk or initial alt or flight level change • Complex: change in routing • Aircraft location during receipt of clearance revision – Non-movement or movement area 8 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Test Environment • 737-like interface – Selected because it presents the “worst case” situation – All data communications done through the CDU • Full flight deck with 180 degree view of airport environment • Radio chatter to simulate operational environment carried out by confederates 9 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Test Design With R/W assignment & departure procedure Without R/W assignment & without departure procedure SEGMENT 1 SEGMENT 2 SEGMENT 3 At the gate DCL Revision ramp area (non-movement area) DCL Revision during taxi (movement area) Initial DCL Simple Initial DCL Complex S1 Complex Simple S2 Initial DCL Simple Complex S3 Initial DCL Complex Simple S4 12 Crews ran through all 4 scenarios 10 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Test Procedure • Training – Classroom briefing on datalink procedures – Training in the flight deck • General familiarization of flight deck • Three training scenarios to put procedures into action • Crews trained to criterion: completion of 2 scenarios (6 datalink messages) correctly without intervention from experimenter • Test – 4 scenarios counterbalanced – Response time from “• ATC” to pilot responding with “ACCEPT” – Questionnaire after each scenario and after all 4 scenarios 11 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Training - Datalink Procedure • Send request • Receive message • Read/understand message • Input elements of message into airplane avionics • Verification checks of data input • Accept message 12 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Training - Datalink Procedure 13 Loading and Verifying an FMS uplinked DCL Load…………………………………………………Select LOAD prompt automatically updates FMC RTE page with route clearance information. FMC RTE ………………………………………….Select Verify ……………………………………DEPT Airport Verify ……………………………………DEST Airport Verify …………………………………………..Runway If part of the Uplinked Clearance, otherwise this is a manual entry Verify ………………………………………….FLT NO: For Example:FAA123 Verify………………………………….Cleared Route The Departure Procedure and Runway may require manual entry Activate (not in sim)……………………………..Select EXEC ……………………………………………….Select to complete the RTE page upload For Internal MITRE Use LEGS Page / Distance check for Uplinked DCL LEGS ……………………………………………….Select PLN Mode………………………………………Select Step through the legs page using “Step” at 6R and observe on the Navigation Display: o Waypoints and Altitude constraints agree with those on your filed flight plan and navigation charts, and o No discontinuities exist between waypoints Verify that an active waypoint is depicted in 1L on LEGS page 1 Select PROG page Verify the total route distance is proper for route of flight versus your filed flight plan © 2011 The MITRE Corporation. All rights reserved. Document Number Here Other Flight Deck Tasks • Flight crews instructed to use standard crew resource management principles – Define pilot flying/pilot not flying roles – Communication between pilot flying/pilot not flying • Complete checklists (from BEFORE PUSHBACK up to BEFORE TAKEOFF) • Communicate via voice with ramp, ground and tower for pushback and taxi clearances in the nonmovement and movement area 14 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Participant Profile • 11 crews (22 pilots) • All air transport pilots, across various carriers • Ratings: Emb145/190, CRJ, B737/757/767/777, Airbus 319/320 • Total flight hours: range from 1375 to 25000 hrs (mean = 10000hrs) 15 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Scenarios • 4 Scenarios – 2 at KDFW – 2 at KMIA • Initial condition: – preflight complete – begin with preflight CDU (input weights, V speeds etc) and flight plan information • End point: – aircraft lined up and waiting on the runway 16 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Typical Scenario SEGMENT ONE: AT THE GATE Request DCL Complete datalink procedure Complete PUSHBACK checklist Request pushback (voice) PUSHBACK Assume two good engine starts AFTER START checklist Request taxi clearance (voice) CAASD 425 Cleared to spot 6 Contact ground SEGMENT 2 NON-MOVEMENT AREA REVISION Simple or Complex revision 17 For Internal MITRE Use BEFORE/AT SPOT Complete datalink procedure Request taxi to runway (voice) CAASD 425 Cleared to RW17R via EK L EH SEGMENT 3 MOVEMENT AREA REVISION Simple or Complex revision DURING TAXI Complete datalink procedure Complete BEFORE TAKEOFF Checklist Contact tower CAASD 425 Line up and wait RW17R © 2011 The MITRE Corporation. All rights reserved. Document Number Here Response Time Distributions (Simple Messages) Distribution of Response TImes to Simple Messages 0.4 0.35 Probability 0.3 0.25 0.2 0.15 0.1 0.05 0 20 40 60 80 100 120 140 160 180 200 220 240 260 280 Response Times 10th Percentile Mean 95th Percentile 99th Percentile 99.9th Percentile 18 For Internal MITRE Use Simple (42 data points) 12.08 31.75 67.02 86.46 89.72 © 2011 The MITRE Corporation. All rights reserved. Document Number Here Response Time Distributions (Complex Messages) Distribution of Response Times to Complex Messages 0.3 Probability 0.25 0.2 0.15 0.1 0.05 0 20 40 60 80 100 120 140 160 180 200 220 240 260 280 Response Times 10th Percentile Mean 95th Percentile 99th Percentile 99.9th Percentile 19 For Internal MITRE Use Complex (85 data points) 55.45 95.24 156.35 256.00 273.62 Note: Response times to initial messages and complex revisions were combined © 2011 The MITRE Corporation. All rights reserved. Document Number Here Questionnaires/Observations • Workload: acceptable • Presentation format: acceptable • Message content: acceptable • Heads down time (compared to voice communication): acceptable • Procedures – No noticeable difficulty in flight crews’ ability to learn and execute the prescribed set of procedures 20 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Questionnaires/Observations • Other issues: – Coordination with AOC – Highlighting only portions of the route that have changed – Multiple open messages • Experienced by a couple of crews. Crews appeared confused and had to be prompted to review the log page. • Will flight crews experience multiple open messages in the actual implementation? – Not being able to view route before loading into CDU 21 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Summary • Response time: – Complex clearance: averaged 95.24 secs – Simple clearance: averaged 31.75 secs • Observations/questionnaires: – Datalink preferred over voice communications – Prescribed procedures easy to learn and execute – Flight crew workload, message format and content acceptable 22 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Next Steps • HITL 2 definition: – Builds off HITL 1 • Examine use of additional messages UM79 and UM83 • Examine use of integrated and non-integrated avionics in delivering departure clearances – Allows us to understand how the flight crew manages the resources on the flight deck (examples ..) • Measure response time • Streamline cockpit procedures 23 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Next Steps • Schedule: – Andy Fry and Brent Blackwell developed messages/scenarios in November 2011 – Data collection started 1/6/2011, finishes end of January – Preliminary report in March, Final report end of April 24 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Backup Slides 25 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Response Time Confidence Intervals Initial With DP* Simple Complex With DP Without DP Without DP With DP Without DP NM M NM M NM M NM M 44.07 17.78 30.38 34.93 97.37 69.63 93.99 125.78 95% CI [75.80, 101.63] [85.79, 115.64] [28.54, 59.59] [12.08, 23.52] [19.29, 41.46] [26.21, 43.64] [76.57, 118.18] [59.59, 79.66] [57.77, 130.19] [86.93, 164.62] 99% CI [71.85, 105.58] [81.22, 120.21] [23.78, 64.35] [10.33, 25.77] [15.90, 44.85] [23.54, 46.31] [70.20, 124.54] [56.52, 82.73] [46.69, 141.27] [75.04, 176.51] Mean 88.715 100.715 * Departure procedure and runway assignment Range across all crews and all conditions (11 crews x 12 observations = 132 data points - 5 missing data points = 127 data points): Minimum = 10 seconds Maximum = 275.57 seconds (4 min 35.57 sec) Case with 275.57 seconds is an outlier 26 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Response Time Confidence Intervals 95% Confidence Interval 200.00 180.00 Response Time 160.00 140.00 125.77 120.00 100.72 100.00 97.37 93.98 88.71 80.00 69.63 60.00 44.07 40.00 34.93 30.38 20.00 17.80 0.00 NM With DP Without DP Initial M NM With DP M Without DP Simple NM M NM With DP M Without DP Complex Condition 27 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Response Time Confidence Intervals 99% Confidence Interval 200.00 180.00 Response Time 160.00 140.00 125.77 120.00 100.72 100.00 97.37 93.98 88.71 80.00 69.63 60.00 44.07 40.00 34.93 30.38 20.00 17.80 0.00 NM With DP Without DP Initial M NM With DP M Without DP Simple NM M NM With DP M Without DP Complex Condition 28 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Response Time Confidence Intervals Initial Simple With DP* Without DP* (n=11) (n=11) Complex With DP* Without DP* (n=11) (n=11) With DP* (n=11) Without DP* (n=11) Mean 88.72 100.72 30.93 32.65 83.50 109.88 95% CI [75.80, 101.63] [85.79, 115.64] [21.81, 40.05] [25.30, 40.01] [72.36, 94.64] [78.82, 140.94] 99%CI [71.85, 105.58] [81.22, 120.21] [19.02, 42.85] [23.04, 42.26] [68.95, 98.05] [69.3, 150.44] * Departure procedure and runway assignment Range across all crews and all conditions (11 crews x 12 observations = 132 data points): Minimum = 10 seconds Maximum = 275.57 seconds (4 min 35.57 sec) Case with 275.57 seconds is an outlier Next longest response time = 157.50 seconds (2 min 37.5 seconds) 29 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Response Time Confidence Intervals 95% Confidence Intervals for Response Time 180.00 160.00 140.00 Response Time 120.00 109.88 100.72 100.00 88.71 83.50 80.00 60.00 40.00 32.65 30.93 20.00 0.00 With DP Without DP Initial With DP Without DP Simple With DP Without DP Complex Condition 30 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here Response Time Confidence Intervals 99% Confidence Intervals for Response Time 180.00 160.00 Response Time 140.00 120.00 109.88 100.72 100.00 88.71 83.50 80.00 60.00 40.00 30.93 32.65 20.00 0.00 With DP Without DP Initial With DP Without DP Simple With DP Without DP Complex Condition 31 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here This is the copyright work of The MITRE Corporation and was produced for the U.S. Government under Contract Number DTFAWA-10-C-00080 and is subject to Federal Aviation Administration Acquisition Management System Clause 3.5-13, Rights in Data-General, Alt. III and Alt. IV (Oct. 1996). No other use other than that granted to the U.S. Government, or to those acting on behalf of the U.S. Government, under that Clause is authorized without the express written permission of The MITRE Corporation. For further information, please contact The MITRE Corporation, Contract Office, 7515 Colshire Drive, McLean, VA 22102, (703) 983-6000. The contents of this material reflect the views of the author and/or the Director of the Center for Advanced Aviation System Development, and do not necessarily reflect the views of the Federal Aviation Administration (FAA) or Department of Transportation (DOT). Neither the FAA nor the DOT makes any warranty or guarantee, or promise, expressed or implied, concerning the content or accuracy of the views expressed herein. 2011 The MITRE Corporation. The Government retains a nonexclusive, royalty-free right to publish or reproduce this document, or to allow others to do so, for “Government Purposes Only.” For Release to All FAA. This document was prepared for authorized distribution only. It has not been approved for public release. Fiscal Year: Outcome Number: PBWP Reference: 33 For Internal MITRE Use © 2011 The MITRE Corporation. All rights reserved. Document Number Here