RTM vs Zephyr – choose your fighter for testing in Jira

Apps, Atlassian, Requirements & Test Management

29 August 2019 • 22 min read

    More and more teams decide on bringing their tests into Jira Software. It’s not a surprise, as this solution turns out to be cheaper, faster, and also supports achieving the goal of each software development project: the best possible quality of the final release. On the Atlassian Marketplace, there are many test management apps integrated with Jira. We divide them into external and built-in apps. Selecting which one is the best choice for your project constitutes a significant part of success. There are some basic factors we should consider, like for example the number of team members involved or the complexity of our project. We’re here to analyze two of the most popular tools, keeping in mind the software development teams’ needs, and help you find out which fighter will help your project win the battle against bugs!

    Pre-eliminations

    External vs built-in apps for Jira

    Before you choose a specific tool, you have to decide on an external or a built-in extension. If you aim at full compatibility, we strongly recommend the second option, as external testing tools usually don’t have a similar interface as Jira Software. This is why bringing them into the process often requires additional training for the team. In consequence, an implementation of a new, separate app takes a lot of time yet before you start actual testing. And this is not the only one time-related obstacle.

    Using external testing software often involves the necessity of integrating and configuring supplementary tools for requirements, tests and bug tracking, which is not a problem in the case of Jira apps. Beyond wasting time, it also makes complete relations tracking impossible (or at least much more difficult). Testers often consider the race against time as one of their most common challenges. Managers, product owners, and other stakeholders also want things to be done as soon as possible, so it’s better for everyone to manage tests in a well-known environment.

    The second reason why built-in addons should be your first choice is money saving. External tools are much more expensive, and still they often don’t meet all the usability expectations. Testing is a part of software development, which includes the work of many people, so the tool designed only for testers usually won’t be sufficient.

    How a testing app for Jira can help you?

    First of all, we’ll have our whole software development project inside one place. This opens up a lot of possibilities, as this solution allows all the stakeholders stay up-to-date. Communication between the team members gets much easier.

    Let’s imagine a situation where testers work on a specific case, confirming the functionality of a new feature, and suddenly the product owner decides that it would be more cost-effective not to implement this feature yet. Without a good tracking system, managers have difficulties with seeing the testing progress, which inevitably leads to wasting time of the whole QA team.

    Also, the lack of consistency automatically delays releasing the product. And this is exactly where built-in apps for Jira come to rescue. Two of the most commonly used are Zephyr for Jira and Requirements and Test Management for Jira (RTM). What are the main differences between the competitors? It’s time to find out!

    Let the game begin

    Mission no. 1: Requirements management

    In this case, the name says it all. RTM for Jira has a transparent tree-structured view with folders and sub-folders for both requirements and tests. This way, the whole team can structure and prioritize requirements, assign related test cases to them, and track the changes implemented throughout the process. RTM introduces four dedicated issue types from which you can choose when creating requirements. Thanks to this functionality, we can keep an order yet in the initial part of the development process, which has a strong impact on the further stages of our work.

    RTM’s tree structure of requirements helps organize the project in a single place right from the beginning

    Zephyr focuses mostly on tests, so it has only one issue type for all kinds of objects. Lack of customization here practically excludes the possibility of including requirements sufficiently into the process. The second disadvantage is that if we, after all, decide to connect requirements gathered in Jira to our testing objects in Zephyr, we can’t do it, but through default Jira issue links. This solution substantially limits relations tracking options.

    Mission no. 2: Integration and compatibility with Jira

    Both apps are fully integrated with Jira, so we can seamlessly link testing objects to regular issues, Epics, and user stories. Our requirements, test cases, and defects are generated as Jira issues so we can easily search for them inside the Atlassian suite. But, it has to be admitted that when it comes to compatibility, RTM takes the lead. It features a well-known interface, built mostly with Atlaskit, which helps quickly adjust to the look and feel. Each user will be able to start working immediately after a plug and play configuration, no matter if we’re talking about a tester with many years of experience or a newcomer. Zephyr, on the other hand, has the front end of an external tool and requires navigating through the elements with the default Jira’s Issue Navigator, which can make the process really confusing.

    Zephyr's interface significantly differs from the Atlassian product look

    Zephyr’s interface differs from the Atlassian product look

    RTM for Jira makes each stage maximally intuitive for Jira users

    RTM for Jira makes each stage maximally intuitive for Jira users 

    Mission no. 3: High-grained relations between all objects

    All testing tools assure better relations tracking than standalone Jira or even Jira integrated with an external tool. Both RTM and Zephyr have traceability-related features, which make it possible to verify how the objects are connected with each other. But considering the actual value of these functionalities, RTM wins again!

    Having only one issue type for each testing object, Zephyr isn’t capable of defining the actual types of relations, and we can only choose from two possible directions: Requirements → Tests → Executions → Defects or the other way around. Generating of the Traceability report inside the app is very unintuitive, and the connections aren’t described but just displayed as a plain list of elements.

    On the contrary, in Requirements and Test Management for Jira we have the Traceability Matrix, which enables us to see correlations between any two baselined issue types using many-to-many relationship comparison. We can define which types of issues will be displayed by customizing X-Axis and Y-Axis with JQL queries.

    Deviniti’s app also provides the second relations tracking report – Requirement Coverage. Thanks to this feature, we can make sure that all the requirements are safely covered by related tests and other testing objects.

    Thanks to Traceability Matrix in RTM, you can see testing objects’ relations at a glance

    Mission no. 4: Automated testing support

    Automated testing is without any doubt a great solution for testers, as above all, it frees them from doing the same things over and over again. Nevertheless, not all teams use this approach yet, because it surely needs additional preparation to be done well. Zephyr supports test automation, providing its users with the possibility to run scripts through REST API. Also, it has additional plugins to integrate with CI tools like Jenkins or Bamboo.

    Automation is a great advantage, as this solution speeds up the whole testing process and thereby saves a lot of time and money for the company. Although RTM focused on manual testing, we know that automated testing support is a must-have of a good, fully functional tool, so we’ve added the automation feature to our app! From now on, RTM also provides users with running scripts through REST API.

    Zephyr enhances test automation by integrating native Jira experience with CI loops

    Zephyr enhances test automation by integrating native Jira experience with CI loops

    Mission no. 5: Flexible reports

    This mission ends in a draw! RTM for Jira and Zephyr are well prepared for reporting test results. Both apps give us the possibility of generating charts and tables, during or right after we perform our executions. The reports, such as Test Execution report in RTM or Cycle Summary in Zephyr, allow to track the overall progress of our test plans. In Test Case Execution report in RTM, we’ll also find the status of each Test Case. The form of reports is clear and user-friendly, so all the stakeholders can catch up. This is very important, as there are always a lot of team members involved in the testing process. Consulting the current status and the previous results on the meeting gets much easier and to the point when we have the actual reports prepared, always ready to back up our words. RTM provides us also with User Dashboard that includes both Test Execution and Test Case Execution reports.

    Test Execution report in RTM for Jira

    Test Case Execution report in RTM for Jira

    Test Metrics dashboard in Zephyr gather all the test results on one screen

    Test Metrics dashboard in Zephyr gather all the test results on one screen

    Final verdict 

    Both tools have their pros and cons, we can even find some common qualities, like for example the fact that both are identical on Server and Cloud, so in case of the need for migration, it will save a lot of time. Having been the first ever test management tool on the Atlassian Marketplace, Zephyr boasts the highest number of active installations. Even though Requirements and Test Management for Jira is the newest one, it develops quickly, answers the most recent needs of software development teams, and comes up with the new features all the time.

    To sum up, RTM for Jira should be your fighter, because it really lets you benefit from all the advantages of testing inside the Atlassian suite. With our app, you have requirements, features, tests, and defects tracked in a single place, whereas Zephyr mainly focuses on testing, which practically excludes requirements from the process. RTM allows you to keep track of the changes in even the most complex and demanding projects, where each team member can see the relations between the objects every step on the way. Our app also gives you the possibility to organize the elements in a tree-structured view. Thanks to this functionality, everyone involved sees the final product’s goals from the very beginning and understands the succeeding stages better.

    If you’ve decided to test our app, RTM for Jira, take a free 30-day trial from the Atlassian Marketplace. You can also book a live demo via Calendly to see the app in action. We also invite you to read more on bringing test management process inside Jira on Deviniti blog: 

    [contact-form-7 404 "Not Found"]