Gap HARI-03: We do not know how to quantify overall human-automation-robotic system performance to inform and evaluate system designs to ensure safe and efficient space mission operations. (Previous title: SHFE-HARI-03)
Last Published:  07/31/19 10:05:30 AM (Central)
Responsible Element: Human Factors and Behavioral Performance (HFBP)
Status: Open
Description

Initial State of Gap:

Currently, NASA does not objectively quantify human and automated/robotic (HAR) performance in space operations. HAR performance is measured for specific tasks and specific HAR systems, but these are not generalizable. Furthermore, future design reference missions include new types of HAR systems that will need to contend with limited ground support, novel environmental conditions, communication latencies, and other operational constraints. With our limited operational experience with these types of HAR systems, NASA needs to identify effective methods, tools, and metrics to quantify overall HAR performance.


Interim Steps:

  • Conduct research to investigate the validity of assessment of HAR performance conducted in analog or simulator conditions versus assessment in real environmental and spaceflight conditions.
  • Formalize methods currently used for determining, assessing, and monitoring NASA HAR performance.
    • Determine and distinguish methods, tools, and metrics for assessment of HAR subsystems (e.g., displays, controls, software) and systems.
    • Determine the suitability of these methods for long-duration long-distance space missions.
    • Identify if there is a need for new methods to measure performance for future HAR systems.
  • Solicit and conduct research to evaluate key human performance metrics and their relationship to integrated HAR system performance.
    • Identify and validate HAR system performance metrics.
    • Determine the relationship between HAR system performance and human performance metrics, including situation awareness, trust, complacency, workload, and usability.
    • Evaluate HAR system performance when crew is required to transition through modes of automation, like hand-offs between humans and automation/robotic systems.
  • Develop standards for quantification of HAR performance in spaceflight operations.
    • Apply and adapt HARI validation and verification methods from external domains to spaceflight operations.
    • Recommend acceptable levels of integrated performance.

Approach:

  • Conduct observations, interviews, workshops, and TIMs as appropriate to gather subject matter expert and stakeholder knowledge, arrive at consensus on important issues, and vet products such as methods, tools, and metrics. Include consultation with both internal NASA scientific and operational community personnel as well as external experts.
  • Utilize solicited or directed projects to conduct experimental research to identify and validate methods, tools, and metrics appropriate for long-duration long-distance space missions. When appropriate, use literature reviews and existing models to refine methods, tools, and metrics.
  • Emphasize cross-risk collaborations that interact with technology for exploration missions (e.g., Risk of Inadequate Human-Computer Interaction, Risk of Performance Decrements Due to Inadequate Cooperation, Coordination, Communication, and Psychosocial Adaptation within a Team).
  • Refine and validate in an operational context the methods, tools, and metrics produced through solicited and directed research and through use of in-flight or analog data.
Target for Closure
Documentation of recommended methods, tools, and metrics necessary to quantify human-automation/robotic performance, reported in appropriate publications, standards, stakeholder panels, and/or risk Evidence Report.
Mappings
Risk Risk of Inadequate Design of Human and Automation/Robotic Integration
You are here! Gap HARI-03: We do not know how to quantify overall human-automation-robotic system performance to inform and evaluate system designs to ensure safe and efficient space mission operations. (Previous title: SHFE-HARI-03)
Active
Completed
Planned-Funded

Documentation:
No Documentation Available