Modern software development and test processes encompass a wide range of best practices and development methodologies. Personal preferences and lessons learned — both good and bad — dictate most workflows. Customized tools and scripts are frequently cobbled together with internal and external automation tools. However, at the core there are a set of proven development and test methods that enable deployment of high quality software with, ideally, no defects. Adherence to coding standards, performing software verification early in the development process, checking against an established set of quality metrics, and identifying software operations that are known good or known to contain faulty code will bring quality and consistency to critical embedded software.
Establish Software Quality Models and Objectives
Embedded software in quality-critical systems continues to become more complex. Even as the systems they develop become more sophisticated, software development organizations must meet stringent software quality objectives that are mandated by the organization itself or required by the customer or by government regulations. For software teams to meet these objectives, and to ideally achieve zero-defect software, a quality model must be defined. The model establishes specific quality objectives with metrics and thresholds. These objectives provide a mechanism by which teams can communicate goals and status, both internally and with others outside the organization.
Static code analysis automation tools provide quantifiable metrics about the quality of software. As the term static implies, the tools analyze the source code without requiring program execution. Using techniques including code scanning and abstract interpretation, these tools detect errors in code and measure compliance to coding standards. The more advanced static code analysis tools can formally prove that the source code is free of specific run-time errors.
Example of a Software Quality Model With Objectives
One example of a software quality model with well-defined objectives developed by several automotive OEMs and MathWorks is described in a document titled “Software Quality Objectives for Source Code”*. This quality model comprises a set of software quality objectives, many of which implicitly require the use of static code analysis. Compliance with these objectives is assessed using the following criteria:
- Quality plan in place;
- Detailed design description completed;
- Code metrics established;
- Code standards met (for example, adherence to MISRA-C);
- No unreachable branches;
- No unreachable branches;
- Run-time errors eliminated or potential for run-time errors understood;
- Dataflow analysis completed.
Based on these criteria, the document establishes six Software Quality Objectives (SQO); SQO-1 requires meeting just a few criteria, and SQO-6 requires fulfilling all criteria. Selecting the appropriate SQO for a project depends on:
- The project’s criticality;
- The organization’s internal quality processes, which may include the application of Capability Maturity Model Integration (CMMI) or Software Process Improvement and Capability Determination (SPICE);
- The need to abide by standards such as IEC 61508, ISO 26262, or DO 178B.
Table 1 shows criteria applicable to various SQOs. Some of the criteria have an associated threshold metric. Exceeding the threshold triggers achievement of a specific SQO. For example, to achieve SQO-1, an organization must have a quality plan, a detailed design description, and a set of established code metrics. To achieve SQO-2, the organization must additionally adhere to basic coding standard rules, eliminate all systematic run-time errors, and verify that non-terminating constructs do not exist in the code. To achieve SQO-3, the organization must further prove that there are no unreachable branches (dead code).
Using a Language Subset to Comply With Coding Standards
General purpose languages, including C and C++, have been designed to cover a wide range of applications, from desktop software to embedded systems. With extensions such as C99 for the C language and those provided by the GNU or Visual C++ compilers, these languages have evolved to allow more design patterns. The drawback to these advances is the increased cost of verifying complex code. More complex languages are more difficult to verify, either by hand or using an automated tool. To simplify verification, most standards, including IEC 61508 and ISO 26262 (table A.3), restrict the use of the language to a limited subset. To comply with the standard, an organization must use only those features of the language allowed by it. The quality model illustrated in Table 1, for instance, requires adherence to the MISRA-C:2004 coding standard.
In addition to making code easier to verify, coding standards also lead to code that is easier to read, maintain, and port. In general, the adoption of a coding standard does not mean that every rule specified in the standard must be enforced. The automotive OEM quality model previously described establishes two subsets of MISRA-C coding standards. The first subset, which is required for SQO-1 through SQO-4, includes rules such as:
- 8.11: The static storage class specifier shall be used in definitions and declarations of objects and functions that have internal linkage.
- 8.12: When an array is declared with external linkage, its size shall be stated explicitly or defined implicitly by initialization.
- 13.3: Floating-point expressions shall not be tested for equality or inequality.
- 20.2: Dynamic heap memory allocation shall not be used.
The second subset, which is required for SQO-5 and SQO-6, includes rules such as:
- 8.7: Objects shall be defined at block scope if they are only accessed from within a single function.
- 9.2: Braces shall be used to indicate and match the structure in the non-zero initialization of arrays and structures.
- 13.1: Assignment operators shall not be used in expressions that yield a Boolean value.
- 20.3: The validity of values passed to library functions shall be checked.
Finding Dynamic Execution Errors In Source Code
Finding errors is what most software developers think of when attempting to improve the quality of their code. After the source file is saved and then compiled, executing it to find errors is an immediate reflex. It is possible, however, to identify many run-time errors (or dynamic execution errors) by analyzing the source code itself.
Consider the error illustrated in Table 2. In this example, the prototype of the C function does not match its implementation. As a result, when this code is compiled and executed it will produce unexpected results in an embedded system.
The prototype of Arg_f() function specifies that the argument is an int, but in the implementation of the function the argument is a float. Some compilers may detect this error and most static code analysis tools will detect this error.
Proving the Absence of Run-Time Errors
When an error has been identified and fixed, the code may still not meet all of its software quality requirements because there may be other undetected errors in the code. In the example above, for instance, there is a potential divide by zero for the statement r=1/(1- r), because r is declared as a volatile int.
Proving the absence of run-time errors means checking that there are no divide-by-zero, overflow, out-of-bounds array access, or similar errors in the code. Eliminating specific types of runtime errors is a fundamental part of many software quality models. In the automotive OEM model described above, the set of potential run-time errors that must be eliminated expands as the SQOs become more stringent.
The absence of run-time errors can be proven with code verification using mathematical techniques. Based on formal methods, these techniques are applied to detect and prove the absence of run-time errors in source code. One tool that can be used to illustrate the application of these techniques is Polyspace from MathWorks. Polyspace is a code verifier that detects and proves the absence of specific run-time errors in C, C++, or Ada source code using static analysis, so program execution is not required. Polyspace first examines the source code to determine where potential run-time errors could occur. Then it generates a report that uses color-coding to indicate the status of each element in the code (Table 3). Code tagged in green is free of certain runtime errors. Red is used to identify code that will cause a run-time error each time the code is executed. Gray code is unreachable (dead code), and orange indicates code that is unproven and might have a run-time error. Polyspace also checks code for MISRA compliance and also includes tools for data dictionary and dataflow analysis. Developers and testers can use the information generated from the code verification process to fix identified run-time errors. Moreover, they can use the metrics and results showing code proven to contain no run-time errors to meet criteria defined by their project’s software quality objectives.
To meet the SQO-2 criteria in the automotive OEM quality model, the code cannot contain any systematic run-time errors or non-terminating constructs. That is, the Polyspace results should identify no parts of the code in red. To improve software quality further and achieve SQO-3, there can be no unreachable branches in the code, so the Polyspace results cannot contain gray code.
Because unproven code (code that is tagged as orange by Polyspace) is not clearly a problem (it may fail under some conditions), the automotive OEM quality model establishes different thresholds that define the amount of such code that may exist for SQO-4, SQO-5, and SQO-6. For example, SQO-4 requires that 80% of potential divide-by-zero operations are safe. If Polyspace shows that 70% of the operations are safe, then a manual review must be conducted to demonstrate that another 10% are also proven safe or justified. For SQO-5, the threshold increases to 90%. For SQO-6, it is 100%. The software quality objectives permit the first delivery of software with some unproven code; by the time the final software is delivered (SQO-6) there must be no unproven code.
Establishment of a software quality model with well-defined objectives is a best practice for embedded applications in quality-critical environments. Static code analysis can be used to meet the criteria of such models efficiently, by checking compliance to coding standards, identifying run-time errors and dead code, and enabling teams to quantify potential run-time errors in source code.
As the complexity of embedded software grows, more automotive OEMs are relying on static code analysis in conjunction with a software quality model as a means of managing software quality. By applying consistent processes and automated tools, automotive software engineering teams are better able to identify code that may not fail and focus on code that is likely to fail. Creating high quality software is never easy, but it is possible with a process and tools that enable quality to be measured and consistently improved.
- Software Quality Objectives for Source Code. “http://www.erts2010.org/Site/0ANDGY78/Fichier/PAPIERS%20ERTS%202010/ERT S2010_0035_final.pdf” Toulouse: s.n., 2010.