Trusting Automation: Designing for Responsivity and Resilience

• Published in: Human factors Vol. 65; no. 1; pp. 137 - 165
• Main Authors: Chiou, Erin K., Lee, John D.
• Format: Journal Article
• Language: English
• Published: Los Angeles, CA SAGE Publications 01-02-2023; Human Factors and Ergonomics Society
• Subjects: Automation; Hierarchies; Humans; Literature reviews; Man-Machine Systems; Motivation; Operators; Reproducibility of Results; Resilience; Semiotics; Strategy; Systems design; Trust; Trustworthiness; trust; intelligent agents; autonomy; adaptive automation; intersubjectivity; human–automation interaction
• ISSN: 0018-7208; 1547-8181
• DOI: 10.1177/00187208211009995

Objective
This paper reviews recent articles related to human trust in automation to guide research and design for increasingly capable automation in complex work environments.
Background
Two recent trends—the development of increasingly capable automation and the flattening of organizational hierarchies—suggest a reframing of trust in automation is needed.
Method
Many publications related to human trust and human–automation interaction were integrated in this narrative literature review.
Results
Much research has focused on calibrating human trust to promote appropriate reliance on automation. This approach neglects relational aspects of increasingly capable automation and system-level outcomes, such as cooperation and resilience. To address these limitations, we adopt a relational framing of trust based on the decision situation, semiotics, interaction sequence, and strategy. This relational framework stresses that the goal is not to maximize trust, or to even calibrate trust, but to support a process of trusting through automation responsivity.
Conclusion
This framing clarifies why future work on trust in automation should consider not just individual characteristics and how automation influences people, but also how people can influence automation and how interdependent interactions affect trusting automation. In these new technological and organizational contexts that shift human operators to co-operators of automation, automation responsivity and the ability to resolve conflicting goals may be more relevant than reliability and reliance for advancing system design.
Application
A conceptual model comprising four concepts—situation, semiotics, strategy, and sequence—can guide future trust research and design for automation responsivity and more resilient human–automation systems.