Developing a 6DoF Aircraft Munitions Optical Tracking System

During the release of an external store, such as a bomb, from an aircraft, disturbances in the airflow surrounding the aircraft create coupled aerodynamic loads. These loads affect the released store’s trajectory which may cause the store to collide with the aircraft. Reducing the risk of such an event and ensuring the safety of delivery, jettison, and launch operations make executing dedicated flight tests essential. (Figure 1)

Today, computational fluid dynamics (CFD) simulation software is used to estimate the trajectory of separation between the aircraft and the store. But how reliable is the CFD model in real-life situations? To determine its accuracy, a flight test campaign is used to measure the store’s actual separation trajectory. Several test points are selected from the safest to the most extreme flying condition. The data sets collected from the test flights will be compared against the CFD simulation results to determine the safety level and readiness of the store release operation.

The Mission

The Brazilian Air Force Flight Test and Research Institute (Instituto de Pesquisas e Ensaios em Voo; or “IPEV”) is developing an Optical Tracking System (SisTrO - “Sistema de Trajetografia Óptico”) to measure the store Time-Space Positioning Information (TSPI) during the test flight. The mission is to guarantee a safe store release during the entire flight test campaign.

Figure 1. The SisTrO has been installed in the IPEV’s Photogrammetric Pod in the Test Bed. Shown in the diagram is a simulation of an external store. (Source: Brazilian Air Force Flight Test and Research Institute)

A scientific bibliographic and COTS product survey has shown that SisTrO is the world’s first system that can compute in real time the six degrees of freedom (6DoF) trajectory of the released store. The required bandwidth for SisTrO (or any similar system) real-time high-speed, high-resolution video frame data transmission is far above the capability of a typical Gigabit Ethernet connection. After experimenting with various solutions, IPEV found the one proposed by ADL Embedded Solutions to be the optimal solution for matching all SisTrO development system requirements.

The SisTrO hardware components include Mikrotron high-speed cameras, ADL’s trajectory and image processors, which use the EURESYS Frame Grabber, and a Telspan Data switch / PTP (Precision Time Protocol) Grand Master. SisTrO software components are responsible for image and data analysis as well as system integration. (Figure 2)

Figure 2. SisTrO Block Diagram. (Source: Brazilian Air Force Flight Test and Research Institute)

System Design Challenges

The ability of SisTrO to compute desired trajectory analyses at 6DoF and in real-time improves the efficiency and cost-effectiveness of the flight test campaign. Because SisTrO will be used in airborne conditions, the system design encountered three main challenges.

The Need for High-speed Transfer

To achieve real-time trajectory results, the SisTrO system needs a transmission rate in excess of 11 gigabits per second (Gbps) to transfer all the data collected by the camera to the image processor. However, traditional Ethernet solutions cannot meet that rate or sustain high-volume data transfer, meaning a local camera recording operation followed by data transmission would cause an unacceptable processing delay. Since most vendors provide Ethernet solutions, IPEV had not been able to find a solution for SisTrO.

The Need for Robust Processing and Storage

To achieve real-time trajectory analysis, the SisTrO processors also need to store the captured images as well as determine the Reference Marks (RM) 2D coordinates of each image frame quickly. Such calculation requires processing 400 frames per second (fps) of high-resolution video, which imposes a significant computational demand.

The Need for Reliable Performance in a Rugged Environment

The third factor that adds complexity to the situation is that SisTrO must run autonomously in an airborne environment. Therefore, its components must offer high performance along with reliability, ruggedness, and compactness, including MIL-STD-810E compliance.

As IPEV continued to search for solutions to overcome the technical bottleneck of massive data transmission, EURESYS, a system integrator specializing in producing frame grabbers, recommended ADL to IPEV.

Problem-Solving Approach

In mission-critical industrial applications such as those in the aeronautical environment, failure is unacceptable. Many applications require real-time results. Therefore, timing is critical. System performance delays or hardware failures may have grave consequences.

Industrial Internet of Things (IIoT) applications require the continuous collection, processing, and storage of a lot of data in environments that are often remote and hostile. In addition, the form factor of the single board computer (SBC) must adhere strictly to space, weight, and power (SWaP) requirements. Therefore, the SBCs’ ability to integrate data sets, interoperate with other systems, and perform in a low-maintenance and high-consistency manner will affect the effectiveness of the IIoT application.

Designing an embedded system is a complex process that involves various engineering teams in systems, mechanical, thermal, electrical, cable, and software. In addition, the design process requires input from additional prototyping, testing, and quality control teams. It is critical to optimize the design, ensure the performance and reliability of the embedded solutions, and increase design efficiency and accuracy, so the costly and time-consuming cycle of redesigning and retesting can be avoided.

Therefore, understanding the client’s need for flexibility and robust support during product customization and development is extremely important. By providing an initial solution with an 80% fit, ADL worked closely with the client to get to the final 100% with the turnkey system design service. In this service, ADL provides customer consultation including conceptualization, proof of concept, development/design, prototype, pilot run, review, quality control, and production. By supporting the client with a holistic system development process that captures the complexity of requirements, specifications, and changes that arise during system development, ADL will reduce the overall cost and time for the client.

How ADL and Brazilian Air Force Flight Test and Research Institute (IPEV) made the project a success

For years, the research team at the Brazilian Air Force Flight Test and Research Institute (IPEV) has been trying to integrate SisTrO’s software into a high-speed camera’s internal FPGA. However, intelectual property (IP) issues prevented the pursuit of such an elegant solution, leading to the decision to use a recording camera. The recording camera could transmit test point data to an image processor after each test point, i.e., store release, but the performance level possible with Gigabit Ethernet cannot match this application’s real-time requirements. Upon the first contact with ADL’s team, IPEV researchers were impressed by ADL’s ability to listen and truly understand their needs.

ADL suggested the use of the CXL-6 frame grabber by EURESYS, an interface card compatible with the ADL processor that supports the CoaXPress protocol, to transport the image frames between the camera and the ADL processor. Using this innovative hardware platform solved the technical issue of freezing the processor configuration. As a result, ADL was able to move the SisTrO project from the testing stage to a fully functional solution in less than nine months.

Detailed SisTrO Description

Figure 3. The dual ISys mounted into SisTrO’s Photogrammetric Pod. “iSys” stands for “Imaging System.” (Source: Brazilian Air Force Flight Test and Research Institute)

The complete SisTrO developed by IPEV is an innovative autonomous computational vision solution that computes, in real-time, the 6DoF (Degrees-of-Freedom) trajectory of the released store. The SisTrO components consist of two imaging systems (ISys) units inside the pod (Figure 3). Each ISys unit includes a high-speed Mikrotron camera connected to ADL’s image processor unit (ADL 7000-EPD-01).

Figure 4. ADL 7000-EPD-01 System stack and its components. (Source: Brazilian Air Force Flight Test and Research Institute)

The image processor unit is in a rugged chassis and houses the following:

  • The standalone processor, ADL-USA custom-design 7000-EPD-01 image processor, which is based on the ADLQM87PC Intel® Core™ i7 Quad Core. 7000-EPD-01 has the capacity of 2.4 GHz (p/n 292750), with 8 GB DRAM, 1600 MHz (Figure 4)

  • EURESYS InterPC C2C Link Adapter for camera shutter synchonization

  • Astronics Avionics I/O card (p/n PE1002) to acquire IRIG-B time base signals

  • EURESYS dual-channel CXP-6 video capture card (p/n ADLVIS 1660)

The ADL 7000-EPD-01 image processor acquires 400 fps images, so the 2D coordinates of all in-view RM can be determined by the image processor and routed to the trajectory processor. In parallel, ADL 700-EPD-02 computes the 3D coordinates of each RM and then makes the conversion to a 6DoF CG trajectory of the released store.

ISys performance tests proved that SisTrO could acquire and store uncompressed 10-bit gray images at 400 fps with 1920 × 1080-pixel resolution and compute the 2D coordinates of up to 32 RM center in real time. (Figure 5)

Figure 5. The Diagram of the Final Finished System. (Source: Brazilian Air Force Flight Test and Research Institute)

The system was designed for airborne operations in compliance with MIL-STD-810E. Once SisTrO is installed in a pod shell, it can be carried by any aircraft or helicopter. As an example, the SisTrO was installed on Modeling and Simulation the EMBRAER A-29, a Super Tucano advanced trainer aircraft. With the introduction of SisTrO, carrying out store release experimental flight test campaigns should be more effective without jeopardizing test results, accuracy, and safety.

The IPEV team was impressed by the ability of SisTrO to produce the required results in real-time with high accuracy (typical >±6mm @1sigma). Moving forward, IPEV plans to build two further SisTrO units with enhanced capability, such as dual but not simultaneous vision.

So far, SisTrO is the only optical tracking system designed for flight test applications that can produce the 6DoF trajectory (Figure 6) in real-time. SisTrO represents a significant advancement over similar systems, which are not qualified to be airborne or cannot work in real-time with high-speed and high-resolution images up to 1920 × 1080 at 400 frames per second. Although there are several highspeed, high-resolution cameras that can be used for flight tests and image capture and storage in real-time, they can only perform image and data processing as a post-mission operation. Therefore, these cameras will slow down the flight test campaign's productivity to a typical single test point every half day.

Figure 6. 6DoF Trajectory of a released store computed by SisTrO, espressed as the lateral, longitudinal and vertical displacements of the store and its q, f and y EULER angles. (Source: Brazilian Air Force Flight Test and Research Institute)

In contrast, SisTrO can execute up to two test points in a single flight without jeopardizing the safety of the test or the accuracy of the results. Therefore, SisTrO will improve the efficiency of such flight test applications. Furthermore, SisTrO has the capability to expand, allowing a broader range of applications.

Summary

While a vendor can provide value by supplying clients with single-board computers that are high-performing, rugged, reliable, and low-maintenance, it can create even more value by working with a client beyond hardware.

During the development of the optical tracking system SisTrO, IPEV had access to high-speed cameras and high-performing processors. However, it was very challenging to transfer a large amount of imaging data from the camera to the processors quickly and continuously. The traditional transfer via Ethernet was not up to the job.

ADL enabled IPEV to succeed in the SisTrO project by doing the following:

  • Began with a solution that is 80% complete and worked with the client to 100% completion.
  • Used a frame grabber as an interface to the processor, making it easier to finalize the processor configuration.
  • Provided the key contribution that moved the project from testing to full functionality in less than nine months.

To IPEV, ADL's technical solution enabled the success of the SisTrO project. Instead of 2 test points per day by typical systems, the SisTrO system can achieve at least 2 test points in a single flight while ensuring test safety and data accuracy.

This article was written by JC Ramirez, VP of Engineering and Product Manager, ADL Embedded Solutions, Inc. (San Diego, CA). For more information, visit here .