Test and Evaluation of Autonomy for Air Platforms
Tools, approaches, and insights to confidently approach the safe, secure, effective, and efficient testing of autonomy on air platforms.
The 2018 National Defense Strategy emphasizes that the effective implementation of autonomy is essential for future engagements. Key to this implementation is the ability to test and evaluate systems that perform autonomous tasks. The purpose of this research is to equip testers with tools, approaches, and insights to confidently approach the testing of autonomy on air platforms. The air domain is chosen specifically for its applicability to the Air Force mission and to help scope the focus of this research. The intent is not to be an exhaustive reference for testing and evaluating autonomy; rather, the goal is to provide a launching point for greater investigation.
Testing autonomy on airborne platforms brings unique challenges and, therefore, to be successful, testing should leverage the following five principles: Early User Involvement, Continuous and Cumulative Feedback, Streamlined Process and Products, Pilot Training Approach, and Human-Machine Interaction Consideration. The overarching test approach is split into two paradigms: three phases of test and Agile Development and Operations (DevOps).
First, the current flight test paradigm of plan, execute, and analyze (three phases) is reexamined through the lens of testing autonomy. Of the three phases, executing a test of autonomy is currently the biggest challenge within the test community. The goal of any test, and of autonomy in particular, is that it is safe, secure, effective, and efficient. Therefore, the optimal test execution of an autonomous task or mission should include some type of run time assurance, live-virtual-constructive, open systems architecture, and a surrogate platform.
To support the execution of autonomy testing that is responsive and timely, a close pairing of the planning and analysis phases is recommended. Different approaches, like design of experiments, human factors considerations, hardware-in-the-loop, and systems theoretic process analysis, are provided as tools for facilitating the needed planning and analysis capabilities for autonomy test. Within this paradigm, the goal is to adapt existing practices and modify to facilitate a spiral test and evaluation strategy of autonomy.
The second paradigm features a more dramatic shift from the current test practices to one that incorporates the principles of Agile and DevOps. The Agile philosophy emphasizes individuals and interactions over processes and tools, working products over comprehensive documentation, customer collaboration over contract negotiation, and responding to change over following a plan. It has been instrumental in modern software development and will certainly play a role going forward with the development of autonomy.
Additionally, DevOps is the revolutionary concept that integrates the development and operation functions of an organization. This process relies on continuous integration, continuous delivery, continuous monitoring and logging, microservices, and collaborative, cross-functional teams to implement high-quality products, quickly. As the Air Force begins to adopt more of the Agile and DevOps principles, it will be able to realize the DoD’s goals of “delivering at the speed of relevance,” “organize for innovation,” and “streamline rapid, iterative approaches from development to fielding.”
This work was done by Riley A. Liver-more and Avery W. Leonard for the Air Force Test Center. For more information, download the Technical Support Package (free white paper) below. AFRL-0305
This Brief includes a Technical Support Package (TSP).
Test and Evaluation of Autonomy for Air Platforms
(reference AFRL-0305) is currently available for download from the TSP library.
Don't have an account? Sign up here.
Top Stories
INSIDERManned Systems
Turkey's KAAN Combat Aircraft Completes First Flight - Mobility Engineering...
INSIDERMaterials
FAA Expands Boeing 737 Investigation to Manufacturing and Production Lines -...
INSIDERImaging
New Video Card Enables Supersonic Vision System for NASA's X-59 Demonstrator -...
INSIDERManned Systems
Stratolaunch Approaches Hypersonic Speed in First Powered TA-1 Test Flight -...
INSIDERUnmanned Systems
Army Ends Future Attack and Reconnaissance Helicopter Development Program -...
ArticlesEnergy
Can Solid-State Batteries Commercialize by 2030? - Mobility Engineering...
Webcasts
AR/AI
From Data to Decision: How AI Enhances Warfighter Readiness
Energy
April Battery & Electrification Summit
Manufacturing & Prototyping
Tech Update: 3D Printing for Transportation in 2024
Test & Measurement
Building an Automotive EMC Test Plan
Manufacturing & Prototyping
The Moon and Beyond from a Thermal Perspective
Software
Mastering Software Complexity in Automotive: Is Release Possible...