Eyes In The Sky

New technology gives UAVs a way to process high-resolution video.

Unmanned aerial vehicles (UAVs) are critical to today’s intelligence, surveillance, and reconnaissance (ISR) missions, supplying valuable aerial imagery to ground forces. Small UAV systems are a highly flexible ISR solution since they can be quickly deployed with minimal infrastructure and manpower. Unfortunately, their performance is hindered by limited bandwidth communication links and the stringent size, weight, and power (SWAP) constraints of small UAVs, which make onboard real-time processing of imagery impossible. As a result, a potential game-changing combination of high-resolution cameras and small UAVs has not yet been realized.

The Problem

Results of the CAVE tracker.

This failure to provide quality tactical-level ISR is ironic because two key enabling technologies are already commercially available: current-generation video cameras with multi-megapixel resolution satisfying tight SWAP constraints, and hand-launched UAVs. Commanders in the field could mount these cameras on small UAVs for persistent surveillance. A high-resolution camera on a UAV at an altitude of a few hundred feet would enable them to recognize individual faces and read license plates on cars without the delays and command and control (C2) complexity typically associated with large UAVs.

Unfortunately, while these two technologies combine to get high-resolution imagery onboard small UAVs, there is no way to transmit or process that imagery in real time. “While some recent UAVs have megapixel cameras transmitting over short ranges via WiFi, we’re talking about multi-megapixel cameras transmitting over much longer distances on bandwidth-constrained channels,” explained Ross Eaton, Senior Scientist at Charles River Analytics in Cambridge, MA. The data-streams from such high-resolution cameras far exceed the bandwidth offered by even the most advanced small-UAV transmitters. Such high resolution exceeds the ability of current-generation SWAP-constrained computers to automatically exploit the data.

Further complicating matters, many ISR applications rely on high frame rate as well as high resolution. Tracking a fast-moving car, for example, is much harder at frame rates significantly below 30 frames per second (FPS) because the car moves a considerable distance between consecutive frames. Other UAV applications, such as sense and avoid (SAA), require even higher frame rates, at least in the hundreds of FPS. Slowing the frame rate to avoid the bandwidth constraint therefore is not an acceptable solution.

CAVE

One potential solution lies in improving the efficiency of the UAV’s onboard computer. A power-efficient design of an architecture using many arithmetic logic units, or ALUs, in parallel with software optimization, would be highly advantageous in today’s military environment. Charles River Analytics and Singular Computing LLC (Cambridge, MA) have teamed up to address this issue through an approach called Compact Aerial Video Exploitation (CAVE).

CAVE is an integrated hardware/software system designed to process high-resolution video data at full video rate onboard a small UAV. Its efficient and highly specialized image exploitation algorithms are designed to automate target tracking for ISR. The CAVE software performs feature-aided tracking of targets within the scene, so only narrow-bandwidth target imagery needs to be transmitted back to the user, rather than full-frame video.

The CAVE approach offers many benefits in today’s military environment since the use of UAVs keeps troops away from dangerous missions.

“Consider a situation in which a small unit commander believes there may be armed hostiles over a hill,” explained Eaton. “Instead of sending his troops over to investigate, he could send a small UAV to collect imagery and keep his troops out of harm’s way. Unfortunately, small UAVs collect poor-quality images due to motion blur and poor image tracking, putting significant workload on the operator to understand what he’s seeing in a timely fashion. By the time he’s deciphered the blurry pixels, a truck full of armed hostiles are already approaching with their weapons ready.

“This is where CAVE’s technology can help,” Eaton continued. “A CAVE-equipped UAV can fly over to the hill, examine the area, and locate moving objects. It can quickly send the commander high-resolution images of the specific regions that may require further assessment. Rather than receiving blurry images too late to be of value, the commander receives crystal clear images of a truck with four men carrying AK47s. Now the commander has a true competitive advantage in quickly dealing with a dangerous situation.”

Singular Computing is providing the computing technology that enables CAVE to operate almost two orders of magnitude faster while simultaneously consuming less than 2% of the power of a traditional tracking solution based on a general-purpose central processing unit (CPU). This results in an approximately 6400x improvement in speed and power overall, which can be game-changing where small UAVs — operating under significant SWAP constraints — are used to collect aerial imagery for military ISR.

Singular has developed a massively parallel machine based on an ALU that performs suitable floating point arithmetic using just a few thousand transistors, instead of the many hundreds of thousands used in modern floating point units (FPU). Dr. Joe Bates of Singular Computing explained, “We have developed and are commercializing a unique ‘approximate computing’ architecture, in which some of the responsibility for producing high-quality results is moved from hardware to software. For many tasks, as with CAVE, excellent results are produced but with speed/power ratios 100x better than GPUs [graphics processing units], FPGAs [field-programmable gate arrays], embedded CPUs, and DSPs [digital signal processors], and up to 10,000x better than desktop CPUs. Suitable tasks are found in vision, speech, learning, optimization, simulation, radar, radio, and big data.”

Charles River uses its extensive library of algorithms for image analysis, classification, and target tracking through its VisionKit™ tool suite. VisionKit is used to develop real-time computer vision applications, supporting video acquisition, image analysis, and a broad array of other algorithm needs. It greatly accelerates system prototyping and development, reducing development time and cost, and speeds up the prototype development for CAVE.

To see a video of CAVE in action, tracking a vehicle and maintaining visual contact despite significant changes in vehicle appearance due to orientation, lighting, and scale changes, click here.

Results of the CAVE Tracker

In summary, despite the huge potential benefit to commanders in the field, the combination of high-resolution cameras and small UAVs has proven elusive. Solutions hinge on improving the efficiency in the UAV’s onboard computer. In particular, a power-efficient design of an architecture using many ALUs in parallel, and software optimization to harness this parallelism, is critical to solving the problem. CAVE incorporates this architecture in its design to improve high-resolution ISR on small UAVs, which could be a tremendous advantage in today’s military environment.

This article was written by Ross Eaton, Senior Scientist, and Lisa Cordeiro, Web Content Manager and Technical Writer, Charles River Analytics (Cambridge, MA). For more information, Click Here .