HRR - Gap - HCI-04: We need to understand how emerging multi-modal and adaptive display and control technologies are best applied to the design of HCI for proposed long-duration DRM (Design Reference Missions) operations. (Previous title: SHFE-HCI-04)

NASA Logo Human Research Roadmap HRP Exploration Data

Evidence Risks Gaps Tasks Reports

Explore Search

- Risks
  - HCI
- Tasks

HCI-04: We need to understand how emerging multi-modal and adaptive display and control technologies are best applied to the design of HCI for proposed long-duration DRM (Design Reference Missions) operations. (Previous title: SHFE-HCI-04)

Last Published: 07/31/19 10:05:30 AM (Central)

Responsible Element: Human Factors and Behavioral Performance (HFBP)

Status: Open

Description

Initial state of Gap

Future spacecraft will primarily use glass cockpits, and systems may need to adapt to accommodate long-duration spaceflight conditions, and the resultant impacts to human performance. In addition, many operations are moving toward portable computing solutions that take advantage of multiple modalities (visual, auditory, tactile). Continuous monitoring for these types of new technology solutions is critical for providing mitigations for long-duration DRM operational challenges.

Interim steps for getting to the target (performed yearly)

Monitoring of leading edge human-system interface technologies for applicability to next-generation HCI challenges.
Down-select of technology for further consideration with emphasis on:

Systems that sense/learn/adapt (SRP)
Multi-modal solutions
Integration of technology and standards (SRP)
Evolvability of the technology
Degree of mitigation offered

Research Approach

Stay abreast of latest technology offerings (new tech announcements, journals, conferences, blogs)
Attend HCI-related conferences and DoD HFETAG meetings to see new technology vendors and presentations

Select promising candidates for further evaluation
Use existing facilities/analogs (e.g., iPAS, HERA) for assessments of new technologies

Target for Closure

Tech-Watch: Advanced, multi-modal and adaptive human-system interface technologies for next-generation missions.

Results may also lead to mitigations for HCI-06 and HCI-03

Mappings

Risk of Inadequate Human-Computer Interaction

Active

HCAAM: Responsive multimodal human-automation communication for augmenting human situation awareness in nominal and off nominal scenarios

Task Book: Entry Unavailable
Principal Investigator: Stirling, Leia

Short Title: Resp Multimodal for SA

Responsible HRP Element: Human Factors and Behavioral Performance

Collaborating Org(s):

Funding Status: Active - Currently funded and in progress

Procurement Mechanism(s):

Solicited

Aims:

This task is part of the Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR) deliverable.

Aim 1: Integrate and extend existing capabilities at JPL and MIT into a testbed for examining information communication between human-autonomy teams.
Aim 2: Evaluate SA, trust, and task performance within a ground-based study with selected communication modalities and information displays.
Aim 3: (Option) Perform follow-up studies for additional parameters of interest. Option is communication timing and persistence of response within this ‘call-and-response’ framework to provide guidance on level of feedback required for maintaining procedures and performance.
Aim 4: (Option) Extend the ground-based study to a flight demonstration aboard the International Space Station (ISS).

Resources (Ground Analog)

Deliverables (1)

HCAAM: Using a Human Capabilities Framework to Quantify Crew Task Performance in Human-Robotic Systems

Task Book: Entry Unavailable
Principal Investigator: Fanchiang, Christine

Short Title: Crew Task Performance Quantification

Responsible HRP Element: Human Factors and Behavioral Performance

Collaborating Org(s):

Funding Status: Active - Currently funded and in progress

Procurement Mechanism(s):

Solicited

Aims:

This task is part of the Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR) deliverable.

Aim 1: Establish a framework for quantifying human-machine system performance evaluation
Aim 2: Develop non-invasive, physiological measures of crew capabilities
Aim 3: Develop reduced-order modeling of task performance
Aim 4: Establish an assessment methodology of human-machine performance using non-invasive physiological measures.

Resources (Ground Analog)

Deliverables (1)

HCAAM: Virtual Assistant for Spacecraft Anomaly Treatment during Long Duration Exploration Missions

Task Book: Entry Unavailable
Principal Investigator: Selva, Daniel

Short Title: Virt Asst for LDEM Spcraft Anomaly

Responsible HRP Element: Human Factors and Behavioral Performance

Collaborating Org(s):

Funding Status: Active - Currently funded and in progress

Procurement Mechanism(s):

Solicited

Aims:

This task is part of the Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR) deliverable.

Aim 1: Baseline VA- To measure the impact of the baseline VA (with question answering, without the ability to engage in dialogues or take initiative) and its various skills (anomaly detection, diagnosis, resolution) on performance, CW, SA, and trust in a laboratory environment.
Aim 2: Enhanced VA- To measure the impact of advanced capabilities of the VA in terms of providing explanations and taking initiative on performance, CW, SA, and trust in a laboratory environment.
Aim 3: Analog- To deploy and validate the system in an analog environment.

Resources (Ground Analog)

Deliverables (3)

Category: Risk Characterization, Quantification
Subcategory: Evidence or Risk Characterization

Description:

Measurements of impacts of the baseline and enhanced VA on performance, CW, SA, and trust for representative users in relevant environments
Interim Definition Phase Report describing current status of research.
Final Report and Guidelines at the end of the project

Internal Customers:
None External Customers:
None Is a Customer-Supplier Agreement (CSA) Required? No

Category: Technology or Tool
Subcategory: Systems Solutions, Prototype Hardware or Software

Description:

The VA software and documentation, which will be made publicly available and open-source

Internal Customers:
None External Customers:
None Is a Customer-Supplier Agreement (CSA) Required? No

Category: Requirement
Subcategory: Guidelines/Criteria

Description:

Guidelines about how to design VAs for this purpose, with emphasis on the implementation of explanations and initiative for different tasks related to anomaly treatment.

Internal Customers:
None External Customers:
None Is a Customer-Supplier Agreement (CSA) Required? No

HCAMM: Enabling Autonomous Crew Task Performance with Multimodal Electronic Procedure Countermeasures

Task Book: Entry Unavailable
Principal Investigator: Robinson, Stephen

Short Title: Autonomous MM eProc Countermeasures

Responsible HRP Element: Human Factors and Behavioral Performance

Collaborating Org(s):

Funding Status: Active - Currently funded and in progress

Procurement Mechanism(s):

Solicited

Aims:

This task is part of the Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR) deliverable.

Aim 1: Develop multimodal enhanced electronic procedure countermeasures for effective autonomous crew task performance.
Aim 2: Evaluate crew task performance, trust and situational awareness of multimodal enhanced electronic procedure.
Aim 3: Validate effects of multimodal countermeasures in a mission analog environment.
Aim 4: Report findings and recommend NASA standards and guidelines for multimodal interactions with electronic procedures.

Resources (Ground Analog)

Deliverables (1)

HCAAM: Enhancing Situation Awareness of Automated Procedures using Adaptive Multimodal Augmented Reality Displays

Task Book: Entry Unavailable
Principal Investigator: Schreckenghost, Debra

Short Title: Enhancing SA of Auto Proc

Responsible HRP Element: Human Factors and Behavioral Performance

Collaborating Org(s):

Funding Status: Active - Currently funded and in progress

Procurement Mechanism(s):

Solicited

Aims:

This task is part of the Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR) deliverable.

Aim 1: Determine best methods to improve situation awareness and improve crew autonomy when using a virtual task assistant to prepare for and perform manual maintenance.
Aim 2: Determine best methods to improve situation awareness and reduce workload when a virtual task assistant is used to handover maintenance tasks between users.
Aim 3: Determine best methods to improve situation awareness and reduce workload when using a virtual task assistant to help manage concurrent manual and automated tasks.

Resources (Ground Analog)

Deliverables (2)

Category: Requirement
Subcategory: Guidelines/Criteria

Description:

Candidate standards and guidelines for effective approaches to the virtual task assistant. Results will include recommendations informing the development of virtual task assistants to mitigate potential performance impacts of greater crew autonomy. Results also will include recommendations for communication methods and modalities for interactions with the virtual task assistant.

Internal Customers:
None External Customers:
None Is a Customer-Supplier Agreement (CSA) Required? No

Category: Risk Characterization, Quantification
Subcategory: Evidence or Risk Characterization

Description:

A technical report describing results from each of the studies, as well as a paper suitable for peer-reviewed publication
Interim Definition Phase Report describing current status of research.
Final Report at the end of the project describing results and recommendations

Internal Customers:
None External Customers:
None Is a Customer-Supplier Agreement (CSA) Required? No

Planned-Funded

HFBP HCAAM Capabilities Refinement

Task Book: Entry Unavailable
Principal Investigator: TBD

Short Title: HCAAM Refinement

Responsible HRP Element: Human Factors and Behavioral Performance

Collaborating Org(s):

Funding Status: Planned-Funded - Task expected to be within budget

Procurement Mechanism(s):

Solicited

Aims:
This task will update the HCAAM standards and guidelines based on latest results from HFBP and Exploration Medical (ExMC) research. This will contribute to gap closure by providing updated standards and guidelines for improved decision support for task and medical tools used by crewmembers during exploration space missions.

Aim 1: Identify and recommend updated approaches to novel decision support and execution tools using automation for autonomous crews.

This task also informs several other Risks including MPTASK, Hab, HARI, Train, Team and Bmed.

Resources (Ground Analog, Other)

Deliverables (1)

Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR)

Task Book: Entry Unavailable
Principal Investigator: TBD

Short Title: HCAAM VNSCOR

Responsible HRP Element: Human Factors and Behavioral Performance

Collaborating Org(s):

Funding Status: Planned-Funded - Task expected to be within budget

Procurement Mechanism(s):

Solicited

Aims:
NASA envisions implementation of an intelligent system that will integrate knowledge of the past and current state of the crew and vehicle in order to appropriately adjust and adapt systems to facilitate mission success for a future state. These complex relationships must be maintained for both routine tasks and more complex tasks. The purpose of this task is to focus research on providing the basis for the development of standards, guidelines and automation tools for an advanced autonomous decision-support system for use by our astronaut population that will reflect both individual and team, real-time human performance support for autonomous spaceflight.

1) Characterize human capabilities and limitations related to potential performance decrements during long duration exploration mission (LDEM) spaceflight as relevant to both routine and complex task performance.

2) Determine appropriate advanced, autonomous intelligent system decision and execution support responses in order to reduce the likelihood or impact of potential decrements in human performance capabilities.

3) Perform integrated assessment of intelligent system responses in order to determine positive or negative interactions and validate recommended approaches.

4) Propose specific updates to existing standards and handbooks to enable implementation of findings for upcoming LDEM systems.

Resources (Ground Analog)

Deliverables (1)

Documentation:

No Documentation Available

Related Gaps (18)

HCI-01: We need to understand the effects of vibration and acceleration on crew task performance, and how those effects can be mitigated? (Previous title: SHFE-HCI-01)

Common Item(s)
	Risk of Inadequate Human-Computer Interaction

HCI-06: We need guidelines to ensure crewmembers receive all of the information required to accomplish necessary tasks in a timely fashion, even when operating autonomously. (Previous title: SHFE-HCI-06)

Common Item(s)
	Risk of Inadequate Human-Computer Interaction
	HCAAM: Responsive multimodal human-automation communication for augmenting human situation awareness in nominal and off nominal scenarios
	HCAMM: Enabling Autonomous Crew Task Performance with Multimodal Electronic Procedure Countermeasures
	Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR)

TRAIN-02: We need to identify effective methods and tools that can be used to train for long-duration, long-distance space missions. (Previous title: SHFE-TRAIN-02)

Common Item(s)
	Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR)

HCI-03: We need HCI guidelines (e.g., display configuration, screen-navigation) to mitigate the performance decrements and operational conditions of long duration spaceflight. (Previous title: SHFE-HCI-03)

Common Item(s)
	Risk of Inadequate Human-Computer Interaction
	HCAAM: Responsive multimodal human-automation communication for augmenting human situation awareness in nominal and off nominal scenarios
	HCAAM: Virtual Assistant for Spacecraft Anomaly Treatment during Long Duration Exploration Missions
	HCAMM: Enabling Autonomous Crew Task Performance with Multimodal Electronic Procedure Countermeasures
	HFBP HCAAM Capabilities Refinement
	Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR)

HARI-03: We do not know how to quantify overall human-automation-robotic system performance to inform and evaluate system designs to ensure safe and efficient space mission operations. (Previous title: SHFE-HARI-03)

Common Item(s)
	HCAAM: Enhancing Situation Awareness of Automated Procedures using Adaptive Multimodal Augmented Reality Displays
	HCAAM: Responsive multimodal human-automation communication for augmenting human situation awareness in nominal and off nominal scenarios
	HCAAM: Using a Human Capabilities Framework to Quantify Crew Task Performance in Human-Robotic Systems
	HCAAM: Virtual Assistant for Spacecraft Anomaly Treatment during Long Duration Exploration Missions
	Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR)

HCI-05: We need verifiable requirements that specify standard measurement techniques and metrics for evaluating the quality of user interfaces with specific attention to the usability and evolvability of an interface. (Previous title: SHFE-HCI-05)

Common Item(s)
	Risk of Inadequate Human-Computer Interaction
	HFBP HCAAM Capabilities Refinement

MPTASK-01: We need methods and tools to collect measures of missions, process, and task performance. (Previous title: SHFE-TASK-01)

Common Item(s)
	Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR)

HARI-02: We need to develop design guidelines for effective human-automation-robotic systems in operational environments that may include distributed, non-colocated adaptive mixed-agent teams with variable transmission latencies. (Previous Title: SHFE-HARI-02)

Common Item(s)
	HCAAM: Enhancing Situation Awareness of Automated Procedures using Adaptive Multimodal Augmented Reality Displays
	HCAAM: Responsive multimodal human-automation communication for augmenting human situation awareness in nominal and off nominal scenarios
	HCAAM: Virtual Assistant for Spacecraft Anomaly Treatment during Long Duration Exploration Missions
	Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR)

MPTASK-02: We need methods and tools to support mission, process, and task design. (Previous title: SHFE-TASK-02)

Common Item(s)
	HCAAM: Using a Human Capabilities Framework to Quantify Crew Task Performance in Human-Robotic Systems
	HCAMM: Enabling Autonomous Crew Task Performance with Multimodal Electronic Procedure Countermeasures
	Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR)

HCI-02: We need to understand what aspects of cognitive function and fine motor skills change during long-duration missions and how these changes affect task performance. (Previous title: SHFE-HCI-02)

Common Item(s)
	Risk of Inadequate Human-Computer Interaction

MPTASK-03: We need methods and tools for planning and dynamic re-planning of crew schedules. (Previous title: SHFE-TASK-03)

Common Item(s)
	HCAAM: Using a Human Capabilities Framework to Quantify Crew Task Performance in Human-Robotic Systems
	Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR)

HCI-07: We need to define the acceptable level of risk for HCI performance relative to terrestrial baselines. (Previous title: SHFE-HCI-08)

Common Item(s)
	Risk of Inadequate Human-Computer Interaction
	HCAAM: Responsive multimodal human-automation communication for augmenting human situation awareness in nominal and off nominal scenarios
	Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR)

Team Gap 6: We need to identify methods to support and enable multiple distributed teams to manage shifting levels of autonomy during long duration and/or distance exploration missions.

Common Item(s)
	HCAMM: Enabling Autonomous Crew Task Performance with Multimodal Electronic Procedure Countermeasures
	Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR)

HAB-05: We need to identify technologies, tools, and methods for data collection, modeling, and analysis that are appropriate for design and assessment of vehicles/habitats (e.g., net habitable volume, layout, and usage) for predetermined mission attributes, and for refinement and validation of level of acceptable risk. (Previous title: SHFE-HAB-09)

Common Item(s)
	Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR)

HARI-04: We need to identify and scope the critical human-automation/robotic mission activities and tasks that are required for future long duration, long distance space missions. (Previous title: SHFE-HARI-05)

Common Item(s)
	Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR)

Med01: We do not have a concept of operations for medical care during exploration missions.

Common Item(s)
	Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR)

Med07: We do not have the capability to comprehensively process medically-relevant information to support medical operations during exploration missions.

Common Item(s)
	Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR)

Med10: We do not have the capability to provide computed medical decision support during exploration missions.

Common Item(s)
	Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR)

Print View...

PRRs (1)

NASA Developer: Daniel Febus Responsible NASA Official: Dave Hanson Contact HRP for a 508 compliant PDF Budgets, Strategic Plans, and Accountability Reports Equal Employment Opportunity Data Posted Pursuant to the No Fear Act Information-Dissemination Policies and Inventories Freedom of Information Act President's Management Agenda Privacy Policy & Important Notices Inspector General Hotline Contact HRP Human Health and Performance DIR Johnson Space Center USA.gov ExpectMore.gov

Ground-Based Flight Analogs	Controlled Research Analog, Short Duration (2 months or less) – isolated and confined with spaceflight mission scenario [e.g., HERA, NEEMO]
Number of Subjects	20

Ground-Based Flight Analogs	Controlled Research Analog, Long Duration (4 months or more) – isolated and confined with spaceflight mission scenario [e.g., NEK/Russian Chamber], ICE Field Analog, Short Duration (4 months or less) – isolated, confined and extreme; may not have spaceflight mission scenario
Number of Subjects	0