Project managers and testing team leaders need to monitor the testing progress. Being able to see that progress is really important because it enables executives to define the KPI (Key Performance Indicators) and measure the quality of a product or its version being tested. From these metrics, we are able to make informed decisions about the software quality management process.
In mature production processes, monitoring isn’t only about tracing the progression of tasks. Another critical area of interest is the connections between different requirements and test cases.
That’s how we can tell which tests were designed to verify a given requirement and what their execution looked like, as well as whether as part of that execution any bugs occurred.
I wanted to share what I’ve learned about that over the years of managing my testing team and show you how to implement traceability reports in Jira complemented with a test management app TestFLO.
Requirements vs. Tests vs. Defects
The primary goal of the testing process is the verification of requirements and all assumptions defined in them, as well as the identification of defects discovered during test execution. The links between the requirements, test cases, and defects that occur during test execution create a consistent causal progress which is valuable to monitor.
The requirements designated for realization in a given version of software need to be accompanied by adequate tasks that allow verifying their implementation. We also want to have easy access to knowledge about what tests were designed to measure their coverage of requirements. Another important type of information is the execution of previously designed tests.
A given test case may be executed several times in different iterations, and these executions may bring different results. Close monitoring of the execution process allows us to learn how many times a given test case was executed and with what results.
That way, we are able to tell which test cases in a given version caused problems. Tests that brought negative results should be connected with defects detected in course of test executions. It’s a good idea to verify whether that reflects the reality or there are any negative executions of test cases without any linked defects.
When it comes to defects, knowing their source is really important. And the source is simply the test, as well as the requirement linked to it.
Traceability Report in Jira/TestFLO
Jira is a comprehensive platform that includes many types of reporting, but it doesn’t offer its users a traceability report. That’s understandable, especially if we’re talking about the traceability report in the context of testing.
Let’s not forget that Jira wasn’t designed for test management but project management.
However, by installing specialized apps available on the Atlassian Marketplace, we can expand the functionalities of Jira to cover that area as well. An example of such an app is TestFLO which delivers a comprehensive solution for test management in Jira.
Traceability Report in TestFLO supports reporting of testing progress in the context of requirements. An understandable table presents the path from the requirements through test case templates to test cases and possible defects that occurred during their executions.
Each of the issues includes relevant information such as the summary, status, assignee or priority to help the user understand the process even better. Remember that just like the Test Coverage Report, the Traceability Report is based on the software version inside Jira (Fix Version) so to use it you need to add that information to your requirements.
When monitoring processes that may or may not be related to testing, we always want to have a single source of truth associated with a particular problem we’re investigating.
In test management, traceability reports serve as that type of source. They allow us to trace the complete path from requirements until defects, through tests and their executions. The causal progression will enable us to represent the detailed progress of testing in the context of given requirements.
TestFLO is an app that introduces the Traceability Report in the form of a clear table which presents issues that are related to one another. The chart makes use of the Fix Version context, a mechanism known to all Jira Software users that allows us to check the status of our version against testing.