Training sensemaking for the modern flight deck: A new perspective

Abstract

This study evaluates a novel problem-solving strategy for flight crews experiencing complex, ambiguous and opaque situations in modern, highly automated (fourth generation) aircraft. This strategy has been developed and evaluated in the EU research project Man4Gen in 2015. An experiment was performed with 14 flight crews type-rated for either the Airbus A330 or A320 from various airlines, presenting flight crews an ambiguous and opaque situation in a carefully refined simulated scenario featuring a complex energy management issue combined with a diversion in poor weather. Crew performance was evaluated on 30 distinct challenges and risks that could present themselves throughout the scenario. These performance metrics were compared to crew behavioural indicators concerning the use of the novel strategy and supporting risk information display innovations. The experiment features three sets of crews:  baseline crews (N=3 crews) did not receive training nor displays, one set of crews (N=7 crew) received only training in the strategy and were provided with a quick reference card, and a third set of crews (N=4 crews) received training in the strategy as well as access to the supplementary Risk Information System (RIS) designed to work in tandem with the strategy. This study evaluates both the strategy training and RIS impact on behaviours (through a between-group analysis), as well as the relationships between behaviours and safety performance, by exploring correlations in a within-group analysis of the entire population (N=14). This allowed for a more robust experiment setup, with less sensitivity to random effects in small group sizes.  Results on training effectiveness indicate that the training alone did not induce the desired behaviours as hypothesized (and sometimes the opposite). The RIS was effective in increasing behaviours related to contingency management. The second analysis confirmed that performance increase correlates with several hypothesized behaviours, but results were not powerful enough above post False Discovery Rate p-adjustments. Behaviours related to increased performance included efficient short-term time management before problem solving and increased time spent managing uncertainties and contingencies in a sensemaking cycle. Although the experiment featured several limitations and can be improved upon in terms of training design and possibly deploy a longitudinal study design, it presents a concrete first attempt to both operationalize and visualize resilience theory by engaging human sensemaking abilities in opaque and ambiguous situations.