During Phase I, Vecna demonstrated the feasibility of Automatic Video-based Motion Analysis (AVIMA), a video-based human motion analysis and visualization tool that automatically processes video footage of humans without the use of specialized markers. AVIMA features key innovations such as a video analytics module, a human motion analysis module which automatically estimates and tracks the 3D human body pose from video footage across a variety of operating scenarios, and a video analytics module which automatically interprets the subjects’ movements with respect to their operational environment, objects in the environment, and other crewmates. This video analytics module also locates, annotates, and counts how many times the crew performed a specific action, traversed a particular path, or accessed a piece of hardware.  

Phase II work leveraged the Phase I preliminary designs and prototype results. The team explored machine learning algorithms to detect human presence in a scene and completed a product that will detect when a person enters a specified region.

Mitigation/Partial mitigation of the gap/risk: This research assisted in defining requirements for an advanced tool to efficiently analyze 2-dimensional video.

Follow on task to Semantic Language and Tools for Reporting Human Factors Incidents SBIR Phase II

The collection of real-time habitability data can aid in identifying or refining habitability requirements for future flight and surface vehicles or habitats. Observation of how the human crew uses the environment can lead to improved design as well as configuration and layout to help support the crew in the performance of their mission objectives.

1. Develop software-based habitability assessment tool with capability to capture self-reported habitability and human factors concerns near real-time through photographs, video, audio, and text to facilitate insight into these reported issues.
2. Develop methods for use of hand-held and head-worn video cameras for the targeted collection of video data, with focus on crew-selected habitability and human factors issues, walk-throughs of habitats highlighting key features, and talk-throughs of task performance.
3. Evaluate these tools in an analog environment to determine /validate their effectiveness.

Short summary of major findings:

During the NASA Extreme Environment Mission Operations (NEEMO) 16 mission, participants used the Space Habitability Observation Reporting Tool (iSHORT) (the iPad-based habitability assessment tool) as well as targeted video data collection methods (using head-worn cameras and iPad cameras) to report observations during the 12-day mission.
The findings from the Habitability-Human Factors and Habitability Assessment Tool Directed Research NEEMO task have led to the development of new tasks. iSHORT and video methods, along with additional tools, will be further refined as part of the NHV Data Acquisition Tools task. These tools will also be put to use for data collection in the Vehicle NHV and Habitability Assessment (NHV/HAB Assessment) task to document NHV and habitability issues during the 1 year ISS mission, 6 month flight exploratory study, or other venues such as HERA.

Space human factors data currently come from crew debriefs.  Such debriefs rely on retrospective recall, which could suffer delays of up to six months.  Opportunities to discuss human factors issues in detail during these debriefs are limited.  In particular, the Space Human Factors Engineering Project is interested in obtaining timely and context-specific human factors incident data.  In general, the tool will be used to help detect areas where human factors can contribute to mission success, assess the effects of operational and hardware changes, and complement existing human factors data sources for operations. This research will provide the framework for an on-orbit tool with features not found in a reporting tool to date.

1.   To provide crew easy access to the tool at any time to eliminate the need for the crew to recall information retrospectively.

2.   Create an easy-to-use data gathering protocol with the following functionalities: Allow data to be entered either as text, audio, and/or video inputs.

3.   The ability for the tool to detect a system anomaly automatically and immediately record system status. At a minimum, however, the tool should provide an easily accessible event marker for the crew to mark the context and take a snapshot of the system and operator system status.

4.   A user-friendly automated data search engine for extracting meaningful incident information from the raw data. Examples of desirable search schemes include natural language, spatial, temporal searches, etc.

Mitigation/Partial Mitigation: This research will provide the framework for an on-orbit tool with features not found in a reporting tool to date.