HRR - Task - HCAAM: Conversation analysis to measure and manage trust in virtual assistants

NASA Logo Human Research Roadmap HRP Exploration Data

Evidence Risks Gaps Tasks Reports

Explore Search

- HARI-02
  - HARI
- HARI-03
  - HARI
- HCI-03
  - HCI
- HSIA-101
  - HSIA
- HSIA-201
  - HSIA
- HSIA-401
  - HSIA
- HSIA-501
  - HSIA
- HSIA-701
  - HSIA
- MPTASK-02
  - MPTASK

HCAAM: Conversation analysis to measure and manage trust in virtual assistants

Last Published: 11/23/20 11:55:12 AM (Central)

Task Book: Entry
Principal Investigator: Lee, John

Short Title: Conversation Trust Analysis

Responsible HRP Element: Human Factors and Behavioral Performance

Collaborating Org(s):

Funding Status: Active - Currently funded and in progress

Procurement Mechanism(s):

Solicited

Aims:

This task is part of the Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR) deliverable. This task aims to create conversational measures and mechanisms that allow virtual agents to use an unobtrusive approach to determine if a user is over/under trusting and mitigate this miss calibration with corrective actions and indications within the conversation.

Aim 1: Develop real-time conversational indicators of trust.
- TOCA aid (Total Organic Carbon Analyze)—As a prototypical virtual assistant
- Develop conversational indicators of trust from the AMO TOCA study data
- Develop and validate TOCA as a microworld testbed
Aim 2: Assess robustness of real-time conversational indicators of trust under long duration mission stresses.
- Data collected in a manner that mirrors the experiment from Aim 1 that used the TOCA microworld
Aim 3: Develop and evaluate conversational affordances for managing trust. Similar to Aim 1, this evaluation will be a ground-based study using participants from University of Wisconsin-Madison and NASA JSC who will work with an intelligent virtual agent in the TOCA microworld.
- Validation in the HERA environment
- Develop principles and guidelines for trust measurement and calibration

Resources (Ground Analog)

Deliverables (1)

Category: Risk Characterization, Quantification
Subcategory: Evidence or Risk Characterization

Description:

The outcomes of this research will promote more effective interactions and acceptance of virtual assistants, and will provide new analytic techniques for understanding how people work with automated agents as team members. Three advances for trustable technology require the development of new analytic techniques for understanding human interaction with automated teammates.

Deliverables include:

Interim definition phase progress report
Develop and document principles and guidelines for trust measurement and calibration
Paper and tool for trust estimation
TOCA testbed
Final report with guidelines and standards on trust and technology

Internal Customers:
Human Factors and Behavioral Performance
External Customers:
None

Mappings

Risk of Adverse Outcome Due to Inadequate Human Systems Integration Architecture

HSIA-101: We need to identify the Human Systems Integration (HSI) – relevant crew health and performance outcomes, measures, and metrics, needed to characterize and mitigate risk, for future exploration missions.

HSIA-201: We need to evaluate the demands of future exploration habitat/vehicle systems and mission scenarios (e.g. increased automation, multi-modal communication) on individuals and teams, and determine the risks these demands pose to crew health and performance.

HSIA-401: We need to determine how HSI can be applied in the vehicle/habitat and computer interface Design Phase to mitigate potential decrements in operationally-relevant performance (e.g. problem-solving, execution procedures), during increasingly earth-independent, future exploration missions (including in-mission and at landing).

HSIA-501: We need to determine how HSI will be used in the development of dynamic and adaptive mission procedures and processes, to mitigate individual and team performance decrements during increasingly earth-independent, future exploration missions (including in-mission and at landing).

HSIA-701: We need to determine how human-automation-robotic systems can be optimized for effective enhancement and monitoring of crew capabilities, health, and performance, during increasingly earth-independent, future exploration missions (including in-mission and at landing).

HCAAM: Conversation analysis to measure and manage trust in virtual assistants

Related Tasks (111)

Print View...

PRRs (4)

NASA Developer: Daniel Febus Responsible NASA Official: Susan Torney Contact HRP for a 508 compliant PDF Budgets, Strategic Plans, and Accountability Reports Equal Employment Opportunity Data Posted Pursuant to the No Fear Act Information-Dissemination Policies and Inventories Freedom of Information Act President's Management Agenda Privacy Policy & Important Notices Inspector General Hotline Contact HRP Human Health and Performance DIR Johnson Space Center USA.gov ExpectMore.gov

Ground-Based Flight Analogs Controlled Research Analog, Short Duration (2 months or less) – isolated and confined with spaceflight mission scenario [e.g., HERA, NEEMO]
Number of Subjects	16

Common Item(s)
	Risk of Adverse Outcome Due to Inadequate Human Systems Integration Architecture
	HSIA-201: We need to evaluate the demands of future exploration habitat/vehicle systems and mission scenarios (e.g. increased automation, multi-modal communication) on individuals and teams, and determine the risks these demands pose to crew health and performance.