A comprehensive planning and control framework was designed and developed based on dynamic-data-driven, adaptive multi-scale simulation (DDDAMS) (see illustration) where dynamic data is incorporated into simulation, simulation steers the measurement process for data update and system control, and an appropriate level of simulation fidelity is selected based on the time constraints for evaluating alternative control policies using simulation.

The illustration shows an overview of the proposed DDDAMS-based planning and control framework for surveillance and crowd control via UAVs and UGVs that was developed and refined in this project. The major components of the framework include: 1) real system (UAVs, UGVs, human crowd, and environment); 2) integrated planner; 3) integrated controller; and 4) decision module for DDDAMS. The proposed framework was aimed to enhance the surveillance and crowd control capability of UAVs and UGVs in terms of their performance on crowd detection, tracking, and motion planning. In particular, the crowd coverage percentage was considered as the measure of effectiveness (MOE) in this work. An overview of different components is provided in the following paragraphs.

Integrated Controller: The integrated controller is in charge of effective and efficient control of UAVs and UGVs, where the effectiveness is supported by the integrated planner, and the computational efficiency is supported by the decision module for DDDAMS. To control UAVs and UGVs, the integrated controller performs four major functions: 1) crowd detection, 2) crowd tracking, 3) motion planning of UAV/UGV, and 4) interaction with the real system.

To achieve interactions with the real system, the hardware interface in the integrated controller acts as a medium to collect sensory data (e.g. vision data and global positioning system (GPS) data) from the real system, passes them to the command generator, receives control commands from the motion planning module, and sends the corresponding control commands to the real system.

Integrated Planner: The integrated planner, when invoked, devises an optimal control strategy for UAVs/UGVs based on predicted system performance and passes the updated control strategy to the integrated controller. The integrated planner in the proposed work was implemented in an agent-based simulation (ABS) environment, where the strategy maker selects optimal strategies for each of the same components in the command generator (i.e. crowd detection, crowd tracking and motion planning) based on simulation-based evaluation of alternative strategies against different scenarios.

This work mainly focuses on 1) evaluation of alternative estimation methods of UAV/UGV locations in t, and 2) evaluation of multi-objective weights in UAV/UGV motion planning. For estimation of UAV/UGV location, the crowd shape and boundary are characterized first via clustering technique, followed by the simulation-based evaluation on UAV/UGV locations contingent to different control strategies.

Interactions Among Components: At a given time point t, when the decision module for DDDAMS is invoked, the checking condition (catastrophic abnormality block) is processed first. The checking condition determines whether the current control system has severe problems, or performance deviations (predicted vs. real) are too extreme to recover.

Under these circumstances, the human operator should interrupt the real system run. These fatal abnormalities are due to system malfunctions, human errors, and other issues, which are out of the scope of this work. Of interest are the abnormalities where the actual and predicted system performances deviate significantly, yet every component still works in the normal condition.

Under the ordinary abnormality case, the fidelity selection algorithm is invoked next. The outputs of the fidelity selection algorithm are a combination of different fidelity levels at all considered crowd regions/cells in terms of information details (collected via UAV or UGV) to be incorporated into simulation. In general, simulating group level behaviors involves coarse scale and requires less information and computational resources (and time), while the simulation of detailed individual behavior needs a finer scale of modeling, more detailed information, and are more computationally intensive (and time-consuming).

This work was done by Young-Jun Son, Jian Liu, and Jyh-Ming Lien of the University of Arizona for the Air Force Research Laboratory. AFRL-0291

This Brief includes a Technical Support Package (TSP).
DDDAMS-based Urban Surveillance and Crowd Control via UAVs and UGVs

(reference AFRL-0291) is currently available for download from the TSP library.

Don't have an account? Sign up here.

Aerospace & Defense Technology Magazine

This article first appeared in the June, 2020 issue of Aerospace & Defense Technology Magazine.

Read more articles from this issue here.

Read more articles from the archives here.