Testing in cycles with TestFLO
The reusability and cyclical nature of tests are two features that are part of any testing process. We perform regression tests, smoke tests, as well as functional tests of the software version in cycles. Finding defects during tests usually involves applying fixes and another iteration of tests to verify them. Repeated tests are often performed over the entire scope to ensure that the fixes do not introduce bugs in other areas.
Software testing is a non-linear and unpredictable process, which is why a given version may include many testing cycles. Monitoring them is extremely important because that’s how we can identify critical areas of the application, but also improve the quality of the tests themselves by observing their performance in individual iterations.
If you use Jira for project management, the TestFLO app allows to conveniently create and execute subsequent cycles, and provide you with the means to analyze them.
Next cycle of tests
The creation of another iteration of tests collected in a given Test Plan results from the “Next iteration” action. It allows rerunning tests according to the chosen strategy:
All Test Cases – all Test Cases in the Test Plan
Not passed Test Cases – Test Cases that did not pass
Only failed Test Cases – only failed Test Cases will be included in the next iteration
Custom Test Cases – Test Cases selected by the user
Test Cases consistent with the choice of the strategy will be restored to the initial state, ready for the next execution.
In addition to creating a new iteration of tests, TestFLO also allows restoring the current iteration to its initial state using the “Reset current iteration” operation. That can be useful if the tests were carried out on a different version than required.
Operations related to the creation of a new iteration and restoration of the initial state in the current iteration modify data. Additionally, the once created iterations cannot be deleted, which makes operations more sensitive. It’s a good practice to restrict access to them to a small group of users – you can do that with the TestFLO permissions module.
When creating a new iteration, all the results of the tests in the previous iteration will be saved. They will be used to analyze the course of tests within this test plan, taking into account the testing history.
We can find aggregate information on all iterations and tests performed within the “Test Executions” tab on Test Cases and the “Test Plan Iterations” tab on the Test Plan:
Each saved result contains the essential information about the test execution such as status, initial conditions, steps, errors or assigned person:
Reporting test cycles
Analyzing subsequent test iterations can provide us with valuable information that may affect both the testing process and product quality. Test Plan Iterations Report is a tool that allows monitoring one or many Test Plans with all the iterations and individual test executions that took place within them.
This view allows seeing which tests were performed in a given iteration and what their result was. We also get information on the percentage of tests within a given iteration completed positively, which allows identifying the cycles in which the most problems occurred. Another valuable piece of information is the percentage of positive executions of a given test in a cross-section of all iterations, thanks to which we can find particularly sensitive tests, as well as critical areas in the application itself.
Almost every approach to testing relies to some extent on performing tests in cycles. Some processes will be more restrictive and require a full range of tests after the implementation of fixes, while others may rely on the re-execution of the tests that came out negative in the previous iteration.
TestFLO allows carrying out test cycles under a given test plan according to the chosen strategy. The information collected during the next test iterations is saved for analysis using a dedicated report. The Test Plan Iterations Report can be used to monitor and evaluate all tests performed under the iterations of one or more Test Plans.