Reimagining Automated Test During a Pandemic

As we learned during the COVID-19 pandemic, aerospace and defense companies must continue to move forward and find workarounds to design, develop and test their products or systems. Engineers hit the challenge of how to build and test integrated hardware and software systems under pandemic-induced travel limitations.

This challenge has also helped many test system suppliers question and reevaluate their own procedures and processes and look for smart improvements they can make to help drive key deliverables, such as quality, satisfaction, security and their own green credentials, creating a new, diverse and more inclusive normality when supplying solutions to their customers. How to make these changes remotely has meant ReEmagineering each area of test system design, build, validation and test.

Pre-COVID Test System Development Process

The pre-COVID methods for deploying automated test solutions would start with the creation of hardware and software requirements specification documents. These specifications and subsequent project completion plans are essential for ensuring both parties know what is to be designed and delivered.

Most test suppliers calculate their software development planning and costs using the “Cone of Uncertainty” principle to provide an accurate price and timescale for the software aspect of the project. The specification documents would typically have to be reworked a number of times between engineering teams at the customers’ site before being signed off. Once the requirements specification documents have been created and signed off, the engineers get to work building and integrating the system holding regular on-site meetings to discuss progress and confirm adhesion to the project completion plan.

The Cone of Uncertainty.

Once the system is built, factory acceptance testing (FAT) takes place at the suppliers’ facilities with the customers’ engineers in attendance. The FAT process would show system compliance with each section of the requirements specification and provide the customer reassurance that all of the requirements have been met with the system design. Finally, the system would be shipped out and installed at the customer site where once again a site acceptance test SAT (a subset of the FAT) would take place to ensure that the system performed as specified once installed and commissioned on-site.

Traditional Test Solution Process (Pre-Covid).

The New Normal

COVID-19 has forced suppliers to find new and innovative means to deliver test solutions. Many of these changes were probably underway, but COVID has sped up this change leading to many smart improvements that will remain in place well after COVID.

Onboarding and remote working were the first areas to be re-engineered. With engineers working from home rather than in a centralized office, a number of questions had to be asked:

  • How do you get engineers to focus on project alignment?

  • How do they share project data securely?

  • How do they handle revision control? Luckily, technology came to the rescue.

Tools that have been around for a number of years but were used only occasionally in the test development environment, were now put to their intended use in earnest. Online meetings were arranged out of necessity using tools such as Microsoft Teams or Google Meet. One advantage that has arisen from using this online meeting approach has been that meetings are now extremely productive with all key stakeholders’ present, making consensus and decision making easier to manage. An additional, consequential benefit of holding online meetings is the reduction in travel leading to reduced cost and a lower carbon footprint and improving suppliers’ green credentials.

Increased Collaboration

With remote working comes increased collaboration, both from the customers’ and the suppliers’ points of view. In fact, it has led to an increased sense of involvement from the customer as they are now able to actively contribute at all stages of system development as an integral part of the team. This collaboration empowers each person in the development team to get involved, ensuring that their views are heard and executed upon.

The use of secure cloud-based storage sites enables centralized document and code storage, improving ease-of-use and reducing the chances of documentation loss. Online software and code revision tools such as Git (GitHub, GitLab, etc.) or Mercurial keep tight control of everyone working on the software aspect of the project by ensuring engineers have to check out and then check in each time they develop or modify a section of code.

Software Development

Software development has proven surprisingly easy to perform during the COVID pandemic. Most suppliers are fortunate to have engineers located throughout the US to service their clients locally. Many software developers prefer working individually, so this remote way of working is not new to them.

Regular online communication with team members is vitally important to ensure adherence to the development plan. Project managers hold video calls daily to check on progress with all team members or a select group if discussing a specific part of the project. As mentioned earlier, the software design specification needs to be watertight with a key focus on the software architecture. The software specification is probably the most important document in any automated test system as this software controls the operation and ultimately the success of the test system, which rests with how well the software performs.

The software architecture is what binds the software together, providing details on how the code is to be written and how each section of code will knit together to run seamlessly. Without a fully researched and agreed architecture, each software engineer would be free to interpret the specifications and produce the code that may work but may not be easily integrated into the final delivered software package. This could delay system delivery and add rework costs, so suppliers spend a lot of time on getting the architecture right up front, before writing any code.

The same applies when building a house: if the architect gets dimensions wrong, the result is lost time, more cost and a great deal of frustration! Suppliers have also introduced a stricter peer review process to remotely enable code from each developer to be checked for adherence to the architecture guidelines. Only when this peer review has been signed off will this section of code be considered complete and allowed to be added to the final software build.

System Integration

The way suppliers build and integrate test system hardware has also changed with a smaller team of engineers working on the project, often on shorter-length shifts. Social distancing is observed with limited numbers of engineers physically working on the test rig at any one time. Other engineers now work separately, creating electronic and mechanical sub-assemblies which are assembled onto the rig when completed and ready for on-rig testing. Many test system suppliers have also made the conscious decision to reduce the number of hardware suppliers they use, building stronger and longer-lasting partnerships with those that offer a wide range of complementary products and services.

Test Solution Process (Covid & Post-Covid).

System Maintenance

When it comes to routine maintenance or adding new functionality to test systems already installed at the customers’ site, software teams have adapted the way they do this during COVID. Prior to COVID, software engineers would often work on-site to perform the work required. Now, these services are performed remotely using fully encrypted remote PC control software tools such as TeamViewer. This also allows the customers’ engineers to view the software development or maintenance tasks in real-time while the software engineers perform the modifications required. It is strange for customers to see changes to the software being worked on, but they do get used to it.

The FAT is now performed without the customer present, but they are invited to view the process by using online video technology to view the testing. If they can’t attend, suppliers often create a video of the tests being performed, along with the resultant test results sheet showing compliance to each test performed.

The SAT is more challenging as this is required to be performed on-site. Once the system is installed by the suppliers’ engineers under the socially distanced management of the customer’s site and test management team, the customer is invited to view the reduced SAT. The suppliers’ on-site team has been slimmed down. The team consists of the project manager and typically a more restricted number of hardware and software engineers, depending on the size and complexity of the projects. All wear face coverings and social distance during the SAT.

Post COVID-19 Best Practices

So COVID-19 has thrown up some challenges for most test and measurement companies that supply test and evaluation solutions to the aerospace & defense sector. This has given suppliers the stimulus needed to look at how they develop and deploy these systems moving forward. Like the rest of society, they have had to adapt quickly. Communication is now more important than ever and has enabled suppliers to not only deliver fit-for-purpose test solutions but has also helped them build better, longer-lasting, and more empathetic relationships with their customers’ and their own internal engineering teams.

The use of online video technologies has enabled the development and support teams to communicate more effectively and get to know one another in a more socially and diverse way, building a higher level of trust between customer and supplier. Encrypted, secure software tools have enabled remote access to test solutions on-site to help ensure correct software operation quickly and easily.

Finally, one thing that the COVID pandemic has helped us understand is that we all need to be a little more tolerant and appreciative of each other, understanding that we are all doing our best to build a better way of working, communicating and reimagining success together.

This article was written by Steven Seiden, President, Acquired Data Solutions (Rockville, MD). For more information, visit here .