Why requirements test coverage is key to success?


15 October 2018 • 12 min read

    Requirements are an essential element of every software development process. It’s requirements that define what the process will look like and what kind of quality criteria a given system, module, or specific functionality needs to match.

    Developers implement solutions and testers design tests on the basis of information included in requirements. If requirements are clear and well-defined, testers can start writing tests at the early stage of software development – when the solution is still under construction.

    Covering requirements is a process that allows controlling how many requirements have tests designed for them, what types of tests have been designed, and how much we still need to do before we are ready to pass our test paths to execution. If you’re working with a large number of requirements, coverage reporting helps to save up a lot of time – we get the information we need quickly, instead of checking each requirement separately.

    Requirement-based tests

    Approaching testing with requirements in mind doesn’t mean that you will only design tests on the basis of requirements, but also that you will be checking requirements themselves on at the stage when they are formulated.

    By doing that, you’ll be able to eliminate mistakes that could soon turn out to be very costly. You will also be able to tell whether the requirements are complete, straightforward, testable, and coherent.

    In the next stage of your work – preparing and verifying requirements – you will be adding test cases. It’s smart to do that before your software is ready. That way, when the first version of the application is delivered, you will save up a lot of time and go straight to testing, planning next test cycles in advance.

    Each of your tests should be closely related to a requirement. Without that, you won’t be able to access the coverage of a given requirement by your test and verify which tests have already been designed. This information is critical to learning at what stage you are in your process.

    Note: Tests need to realize business aspects of requirements and always focus on user expectations regarding the application.

    Coverage level

    How do we know that we have reached an adequate level of coverage when it comes to testing it requirements?

    That question is tough to answer. In fact, the answer to this question probably doesn’t exist, especially if we’re talking about testing at the functional level and not the code level. This is quite a subjective aspect that needs to be assessed on the basis of characteristics such as priority or risk of a given requirement.

    We can assume that every requirement that describes a business need has to have at least one test designed for it that checks the so-called positive path. However, most of the time that you will want to test a broader scope that includes alternative paths.

    The factor of requirements test coverage doesn’t really tell us that much – especially when we assume that a requirement is covered by at least one test. That’s why it’s a good idea to look at each requirement separately.

    While in one case a single test may be enough, another requirement may require a different scope for test design. Test analysts will need to use their experience, and knowledge of business needs to tell whether tests designed for a particular requirement are sufficiently exhaustive.

    Test Coverage Report

    Checking the coverage of each requirement individually can be quite tricky – especially if you’re working with a version that contains dozens of them. When looking through requirements coverage, you probably want to have a quick view into particular requirements and their related tasks, as well as access to the most important information like the name or status of a given test and requirement.

    What is particularly important here are the requirements that have not been covered. We don’t want to end up in a situation where we plan the next test cycle – or worse, execute it – and turns out that a part of requirements has not been tested.

    Before starting the test execution phase, we need to be sure that the range of tests for a given version is complete.

    That’s where Jira Software comes in. By using Jira Software, we can save requirements in the form of issues. And by taking advantage of applications available on the Atlassian Marketplace, we can expand the functionalities of Jira Software to cover test management as well.

    An example of such an extension is TestFLO. The app allows defining a complete testing process, including test case creation and linking to requirements. To verify the requirements coverage, we can use the Test Coverage Report that allows displaying all the requirements for a given version together with their attributes such as status or summary. Moreover, we can filter out requirements that have and haven’t been covered.

    Key takeaway

    Building software on the basis of previously-defined requirements is a popular approach because it allows to quickly identify bugs and design tests that are closely related to a given requirement. That type of process helps teams to verify the real business needs of users efficiently and measure the rate of requirements coverage as well.

    Requirements coverage is a useful factor that shows how many requirements have been covered and what types of tests were written for them. By assessing requirements coverage, we can tell whether we can start test execution for a given version or still need to design some more tests. Naturally, we will also avoid the situation where particular requirements are never tested.

    TestFLO allows for comfortable monitoring of requirements planned for a given version together with tests related to them. Requirements and tests are equipped with the most important attributes such as summary or status to enable easier verification. The ability to filter out requirements that have and have not been covered makes the entire process of looking for information even more straightforward.

    Have you got any questions about using Jira Software and specialized apps like TestFLO for test management? Reach out to me at marcin.zurawiecki@deviniti.com, I help organizations design efficient test processes with innovative project management solutions.

    [contact-form-7 404 "Not Found"]