Tech Briefs

Human factors challenges have shifted from manually controlling individual unmanned systems, to supervisory command of multiple semi-autonomous systems.

One critical aspect in developing a quantitative model of unmanned autonomous vehicle (UAV) operator and system performance has been to adopt a task-centric approach to interface design that entails an explicit representation of actions or tasks that need to be performed by the operator. The representation of work in terms of a task serves as a trace in the system that enables designers to track workload in addition to the task progress and flow of tasks among team members. In supervisory control, the focus is the flow of tasks (work) through a system that is composed of human servers and automated servers (software agents).

Figure 1. The interface for the RESCHU SP Simulation. The operator’s display is composed of two main windows placed side-by-side.

Quantitative models and methods that analyze dynamic systems of flow have been developed in the domain of queuing theory. This research may be extended to include supervisory control of unmanned vehicles. In this analysis, the “customers” to the queue are tasks that must be serviced by the human and software servers. The servers are human operators, software agents, and UVs.

Figure 2. A picture in the Payload view allows the operator to visually identify the contact as a friend or foe. Once identified as an enemy, the contact may be assigned a UUV to attack it.
In addition to physical platforms, autonomous agents working as virtual team members are prevalent tools for accomplishing missions. The coordination of actions and interactions among unmanned autonomous systems, manned systems, and a command group will be essential to accomplishing these future missions. A new problem must be addressed: How to maintain an adequate workload to avoid information overload and resulting loss of situation awareness.

The Research Environment for Supervisory Control of Heterogeneous Unmanned Vehicles (RESCHU), developed by MIT, was designed to test supervisory control tasks such as surveillance and identification. This simulation was modified by adding: 1) a complex mission scenario with an asset to protect and multiple simultaneous enemies to attack, 2) a highly automated system such as mission definition language (MDL), and 3) a highly heterogeneous team that is made of at least three different types of UVs. The new version of the simulation is called RESCHU SP.

In the RESCHU SP scenario, a single operator supervises a team of unmanned air vehicles (UAVs), unmanned surface vehicles (USVs), and unmanned underwater vehicles (UUVs). The operator’s task is to deploy the UVs to identify and destroy enemy contacts that are attacking an oil platform at sea. In addition to defending the oil platform, the operator must direct the UVs away from hazardous areas that cause damage to the UVs. The interface for the simulation is depicted in Figure 1.

The operator’s display is composed of two main windows placed side-by-side. In one window, a geo-situational display depicts the spatial position of the UV assets as well as the oilrig, unknown contacts, enemy contacts, and the hazard areas. The second adjacent window is a three-tabbed pane window that contains the following displays: Vehicle Information, The Collaborative Sensing Language (CSL) Editing Controls, and a Payload View.

In order to protect the oilrig and the UV assets, the operator must engage in five tasks: Assign to identify an unknown contact, engage to identify an unknown contact, assign to attack an enemy, engage to attack an enemy contact, and hazard avoidance. The scenario is designed such that only USVs can identify unknown contacts, and only UUVs can attack enemy contacts. The UAVs fly in predetermined flight patterns that the operator can change to avoid hazardous areas.

The operator must first select an unidentified contact, and then assign a USV to identify it. Once the USV arrives at the unknown contact’s location, the operator may engage the USV to identify the contact. This action brings up a picture in the Payload view that allows the operator to visually identify the contact as a friend or foe (Figure 2). Once identified as an enemy, the contact may be assigned a UUV to attack it. The sequence of actions to attack is analogous to the identification process. An enemy contact is assigned a UUV for attack. Once the UUV has arrived at the target’s location, the enemy icon flashes and the operator may select the icon to be engaged to attack. Once engaged, the enemy is attacked and eliminated. Its icon disappears from the geo-situational display.

The relationships among the operator, the CSL, and the UVs, and the manner in which they must process tasks, may be modeled as a network of interactive queues. In an “open” queuing system, “customers” (tasks) arrive at each of the servers. Some tasks are processed and leave the system, but other tasks may be passed from one server to another. Thus, tasks may sequentially arrive at different queues, be waited upon by different servers, and sometimes may “feedback” and return to a previous server before eventually leaving the system. Queuing theory provides quantitative tools to analyze the flow of tasks to and from each server. In addition, the overall performance of the network may be analyzed. All the components necessary to formulate and analyze a queuing system may be extracted from the RESCHU SP scenario.

This work was done by Joseph DiVita, Robert L. Morris, and Maria Olinda Rodas of Space and Naval Warfare Systems Center Pacific. NRL-0061