Unlike the space vehicles of the past, controlled by hard switches and knobs, and driven by paper procedures with ground support available, future spacecraft will be controlled almost exclusively through software displays driven by electronic procedures. There are many open questions about how these procedures should be designed, how much they should be integrated with system displays, and what level of automation is appropriate. There is a lack of empirical data necessary to help make these design decisions, and these issues will become critical during future autonomous missions. Two key questions are addressed in this research:

1. When incorporating automation into electronic procedures, how much automation should be provided, and to serve what purposes?

    

2. What is the importance of a relevant graphical system display in the execution of electronic procedures, and how important is the visual integration of that display with electronic procedures? Of primary interest is the effect of these different procedure configurations on human performance, namely SA and workload.

These questions lead to the following aims addressed in this investigation:

Aim 1: Determine the effect of level of automation of procedure step execution on SA, and other human-system performance metrics.

Aim 2: In a complex, multiple-procedure scenario, determine the effect of procedure management aids (e.g., availability of task allocation information) on SA and other human-system performance metrics.

Aim 3: Determine the effect of the level of integration of system and procedural information on SA and other human-system performance metrics.

Aim 4: Collect data toward validation of an unobtrusive workload system under development by a Small Business Innovative Research (SBIR). The system, known as “Cognitive Assessment of Prediction to Promote Individualized Capability Augmentation and Reduce Decrement (CAPT PICARD)” measures workload continuously in-situ through a headband with sensor suite. The system will be used to measure workload during the study focused on level of integration of system and procedural information.

Aims 1 and 2 are addressed in Study 1 and Aim 3 is addressed in Study 2. Aim 4 will be captured in “Cognitive Assessment and Prediction to Promote Individualized Capability Augmentation and Reduce Decrement” task once complete.

The scope of SA in Study 1 included system state awareness, as well as awareness of crewmember tasking (i.e., which crewmembers were working on which procedures). Subjects performed procedures under each of three automation conditions: 1) Manual (no automation), 2) Mixed (manual commands, automated command verifications), and 3) High (automated instructions, with just a few exceptions, e.g., conditional branching statements requiring a “Yes/No” response, and status checks). In addition to evaluating different levels of automation, this study also investigated the use and management of multiple, concurrent procedures.

For Study 1 twenty-seven subjects (17 male, 10 female) participated in planetary habitat scenario-based simulations using electronic procedures. Subjects were recruited through the NASA Johnson Space Center (JSC) Human Test Subject Facility, and were crew-like in age, education, vision, hearing, and fitness level. All subjects completed a planetary habitat simulation task using Procedure Integrated Development Environment (PRIDE) procedures developed by TRACLabs.

The results indicated that the Manual condition did show the fastest response time to SA queries (approximately 2 seconds faster than the other two conditions), but the difference between that and the Mixed and High automation conditions was not significant. The workload for the Manual condition was rated as significantly higher than in the Mixed and High conditions. In addition, the majority of subjective comments from the post-test questionnaire support the claim that Manual performance results in higher workload and higher SA; whereas, High automation results in lower workload and lower SA.  In terms of preference, subjects preferred the Mixed level of automation.

The objective of study 2 was to investigate how varying levels of information integration impacted performance in the context of a primary procedural task (including malfunction procedures), and a secondary vehicle ascent monitoring task. This involved subjects learning and performing procedural tasks, including fault management, with an electronic procedures system based on the system being developed for the Orion Multi-Purpose Crew Vehicle (MPCV).

Twenty crew-like subjects (13 male, 7 female) participated in Mission Control scenario-based simulations using electronic procedures. Subjects were recruited through the NASA Johnson Space Center (JSC) Human Test Subject Facility, and were crew-like in age, education, vision, hearing, and fitness level.

An Orion-like procedures system and a mission control/fault management simulation was developed by NASA Johnson Space Center Crew Interface Rapid Prototyping Laboratory for use in this study.

Subjects performed procedural tasks in three different experimental configurations/conditions:

1. Procedures-only: In this condition, subjects were provided procedural instructions with telemetry values alongside the procedure step in the upper display region. There were no graphical system displays shown in this configuration.
2. Serial Procedures: In this condition, subjects saw telemetry values alongside the procedure steps, AND could see the value within the context of a graphical system display – but only when sending commands or on request. This configuration used only the upper display region to show subjects the procedures OR the graphical system displays (i.e., serially – one at a time).
3. Simultaneous Procedures: In this condition, subjects saw telemetry values alongside the procedural steps in the upper display region, AND saw the values within the context of a graphical system display shown in the lower display region (i.e., simultaneously – both at the same time).

Based on self-rated SA, the Study 2 hypothesis is partially confirmed. There is indication that the Simultaneous condition (high integration) provided subjects with a higher sense of SA. However, real-time assessment of workload using CAPT PICARD along with eye tracking data did not suggest that workload was lower in the Simultaneous condition.

This task partially mitigates the HCI Risk by leveraging lessons-learned from several other electronic procedures design efforts (e.g., Multi-Purpose Crew Vehicle-MPCV) to study electronic procedure features that may impact situation awareness and hinder performance. Results can be used to develop new standards for electronic procedures.

Integration:
Integration with Risk of Inadequate Design of Human and Automation/Robotic Integration.