RTM Test Execution report to track the progress of testing

Apps, Atlassian, Requirements & Test Management

26 July 2019 • 16 min read

    Updated by Tatiana Glazkova on April 15, 2021

    The main purpose of tests is the quality assurance of the final product. The best way to accomplish it is to track defects throughout the testing process and send them back for fixes by the development team before the software is released. It means you need to be aware of each step and implemented changes, as well as the progress of the test plan, especially, if you deal with more than one of them. Each test run can be performed by a different person, which often makes it difficult to track the process. But there is a solution – Test Execution report in RTM, which helps managers track overall progress and determine the next steps.

    What makes Test Execution a good report

    Aiming for the quality assurance of a product, we should remember to make reports clear to everyone. It may seem obvious, but there is a tendency to create reports that aren’t quite understandable for other team members or customers. They can have either a complicated form and too specialized vocabulary or too many/not enough details. Complications lead to communication problems and make it impossible to settle on an action plan. The reasons can be very different, for example, there is a need to re-execute a specific test plan or the release date of the final project should be delayed. There are some things to check when generating a Test Execution report, so that it does its job:

    • make sure your report is specific but delivers complete information;
    • always double-check the data that is included in the report;
    • pretend you are looking at the report for the first time, check if it is readable;
    • if the generated report is too big, consider applying filters to it;
    • when it comes to a meeting, remember to be on time and prepared to answer questions.

    The last point depends entirely on you, however, Atlassian Jira and Requirements and Test Management for Jira (RTM) applications can help you with the rest. Comparing these two apps, it’s necessary to say that there aren’t often enough basic functionalities in the Atlassian Jira. That’s why Requirements and Test Management for Jira (RTM) can rescue the situation. RTM is a full-featured test management application that can help you get a transparent software development process every step of the way, from requirements gathering to final release. RTM adds Requirements, Test Cases, Test Plans, Test Executions, and Defects modules to Atlassian’s Jira.

    How to track the progress of testing with RTM

    Test reports 

    Creating RTM for Jira, the fact that test management requires a collaboration of analysts, developers, testers, designers, and managers was taken into consideration. That’s why you can be sure the reports generated with RTM are clear, related to each other, have a tree-structured view, and categorized folders and subfolders. Up to now, there are four test reports in our app:

    The first one displays the relations between any two base-lined issue types, using many-to-many relationship comparison, in a special traceability matrix. The second allows you to verify if all your requirements are covered by related test cases, test plans, test case executions or defects. The Test Case Execution report gives the possibility of tracking a specified test case executions progress and is generated as a pie chart, which presents all TCE statuses. The fourth report, Test Execution, also shows the overall progress of test case executions but in the wider context, which we describe more in detail just below.

    The two last reports can also be displayed on the Test Dashboard. The feature lets us see the overall progress of our testing, as well as the statuses of particular test cases on a single screen. This functionality will surely be appreciated especially by Test Managers and IT Analysts, as it allows them to keep track and stay involved every step on the way. But of course, it’s not all of its benefits. The Test Dashboard supports the workflow of testers themselves, as using it they can immediately tell where they are in the whole process, estimate how much work they have to do, and come to many helpful conclusions.

    Setting up the Test Execution report

    Setting up the report is intuitive and doesn’t require gathering data from any external tools. This is one of the strongest advantages of bringing test management into Jira: having everything stored in one place. Thanks to that, in RTM reports are generated automatically, what prevents possible mistakes. Test Execution report presents the progress of each execution of a given test plan and the statuses of the test cases included in it. After selecting a specific Test Plan or/and Execution, you can filter displayed results by Test Plans/Executions (depending on what you consider your main indicator), RTM Environment, TCE Created Date and TCE Assignee. It’s not obligatory to complete all tabs, as it depends entirely on you how detailed you want the final report to be. The report also makes it possible to choose which Test Case Executions statuses should be presented, after clicking the Preferences icon in the upper right corner of the page. The results can be exported to .csv or .pdf file.

    Test Execution report in RTM

    Sample of a Test Execution report

    Test Execution and project management

    With the Test Execution report, based on displayed progress, you can estimate how much time you need to complete your testing process. Thanks to the colorful bars on the chart along with the exact percentage of each status, it will take just one look to know the proportion of passed and failed test cases. The report also distinguishes objects that are in progress, yet to do, blocked or passed with restrictions. Planning your further work or presenting the current course of tests to other stakeholders can be really easy from now on. The transparent form of the Test Execution shows which test execution generates the majority of failed test cases. As the results can be filtered, the report makes it easier to observe how, for example, a change in the runtime or the person in charge affects the number of defects found.

    Last but not least, the functionality makes it possible to verify if a new release didn’t cause bugs in any of the previous ones. We can spot it simply by checking the statuses of the particular executions of test plans. If after the update we have fewer successful test cases in any of the versions of our product than before, it means something went wrong. That kind of information will certainly be useful not only for testers or developers, but for product owners or managers as well.

    Key takeaway

    In order to be sure that the communication inside your team is on the highest level, you need good test reports. Transparent charts and tables help the team members understand each other and make the exchange of information easier. When projects are as complex as testing, not to mention the whole software development, the risk of misunderstanding rises. RTM for Jira lets your whole teamwork on a project in a single place. There are a few test reports that allow each person to be involved and keep tracking the process every step of the way. The Test Execution report is intuitive to create. It shows the progress of your test cases in successive executions and the percentage of a specific execution each status constitutes. Its graphic form, where all the statuses have their own colors, presents the data well. Their visual, user-friendly form is understandable for everyone.

    If you’d like to test Requirements and Test Management for Jira (RTM) yourself, take a free 30-day trial from the Atlassian Marketplace. You can also book a live demo via Calendly, if you want to see the app in action, or learn about implementing a test management process inside Jira on our blog: 

    [contact-form-7 404 "Not Found"]