Integration of user-defined static analysis tools into VectorCAST
The costs of fixing a bug depends on the phase of the software development the issue is discovered. Even though the actual reported numbers vary in the literature, it is a common understanding that the costs are heavily increasing with every development stage [1]. Logically, in order to reduce cost and to increase quality, testing activities are brought forward in the process as much as possible.
To ensure software quality, generally two types of testing tools are applied: Static analysis tools and dynamic test tools. Static analysis tools can examine the source code without actual execution. This makes these tools ideally suited for the coding phase, to make sure that code guidelines are followed and to prevent introducing potential runtime errors (e. g. caused by a zero division or by accessing non-allocated memory locations). Dynamic test tools on the other hand can verify functional requirements by an actual execution of the software at the unit, integration and system level.
Due to the efforts to test as early as possible and the advances of automation of dynamic testing activities (tests can now be executed per change in minutes by everyone instead of per release in weeks by a specialized team), dynamic tests are available earlier in the V-cycle and have become available to the development team during coding stage. Because of this reason and the fact that static analysis and dynamic testing can benefit from each other (e. g. dynamic tests can prove that certain issues found with static analysis are actually bugs), it is attractive to integrate both testing methods in one tool environment.
VectorCAST offers out of the box integrations for several popular static analysis tools: Lint [2], CodeSonar [3], Klocwork [4], Polyspace [5] and QA-C [6].
If a particular static analysis tool is not on this list, a simple but powerful Python interface for static analysis tools can be used to integrate the tool into VectorCAST.
The general procedure to integrate such a user-defined static analysis tool into VectorCAST is a three-step process as shown in the subsequent picture.
In the first step, the name of the static analysis tool, the path to an icon for the tool and the path to the Python script that is called by VectorCAST when the user clicks on the corresponding button are defined. The picture below shows as an example the settings made for the integration of Cppcheck [7].
As input for the Python script, VectorCAST stores an XML file in the current testing environment. This file contains the present context for the static analysis tool, i.e. the path to the file(s) that need to be analyzed and the include paths.
In the second step, a Python script template for the integration of user-defined static analysis tools is adapted to the actual static analysis tool (renamed to Cppcheck.py in this example). The script parses the aforementioned XML file, composes the call of the static analysis tool (including path to executable, files to be analyzed, include paths, output options etc.), executes the static analysis and then converts the results into the XML format in which VectorCAST expects the static analysis results to be in.
In a third step, the newly integrated static analysis tool can be used in the same way as the other tools that are supported off the shelf. A click on a button in the tool bar starts the static analysis of the selected source files. After the results are read into VectorCAST, the list of findings with additional information is displayed. When an issue from the list is selected, the corresponding source code line is highlighted.
In this post, the use of static and dynamic testing tools in the development cycle and the reasons for their integration has been discussed. In addition, it has been shown how a user-defined static analysis tool can be integrated into VectorCAST.
[2] Lint, https://www.gimpel.com/
[3] CodeSonar, https://www.grammatech.com/products/codesonar
[4] Klocwork, https://www.perforce.com/products/klocwork
[5] Polyspace, https://de.mathworks.com/products/polyspace.html
[6] QA-C/QA-C++, https://www.qa-systems.com/tools/qa-c/static-analysis/
[7] Cppcheck, http://cppcheck.sourceforge.net/