8 things to consider when implementing test automation


30 August 2017 • 16 min read

    Updated on the 10th of March, 2021

    Test automation is the rage right now in Agile development environments. And for a good reason. Automation works great for executing smoke or regression tests. It also helps testing teams to prepare the test data and execute complex test cases, involve a lot of data, or get located on different platforms.

    Test automation frameworks can bring testing teams plenty of benefits, but only as long as they know how to efficiently manage common test automation challenges. When used right, automation can boost the productivity of your team and enhance the quality of your products.

    8 challenges and tips to solve them

    1. What to automate?

    If you think 100% automation is an option, keep dreaming. You won’t be able to automate every single test case. Especially in areas like compatibility, user interface, or recovery where most testing needs to be carried out manually. Before you begin to think about automation, you need to come up with a set of criteria test cases must meet to be automatized. Which test cases are important to you?

    Usually, it’s the areas that are prone to risk, of interest to clients, or characterized by complex business logic. Identify these application areas and test case parameters to make sure that your automation efforts pay off.

    For example, it’s a good idea to automate test cases executed with different sets of data or in different environments. The same goes for test cases with a large amount of data. But it doesn’t make sense to automate test cases or modules that run just once.

    2. The right automation tool

    That’s another tricky area of test automation. Automation requires tools and teams might find that problematic because of various reasons:

    • they lack the expertise to make the most of a specific tool;
    • they don’t know whether the tool they need exists;
    • they want a tool that isn’t available on the market;
    • their tools of choice don’t offer 100% test coverage;
    • the cost of tools exceeds their test budget.

    According to the World Quality Report 2019-2020: Southern Europe, a significant number of respondents (39%) say they are starting too late with test automation, probably because of the region’s relative immaturity in this area. At the root of this issue lies the problem of insufficiency of skills. 

    If your team doesn’t know how to use a tool, you can organize a training session with its creator, buy an online course for your testers, or hire a consultant who can help your team master it. Reach out to the testing community if you’re still looking for the right automation tool – forums like Stack Exchange are a rich source of information.

    Sometimes your tool might not do everything you need. In that case, you should start searching for multi-tool solutions that cover areas critical to your team. If the tool you found is way out of your budget, just prepare a cost vs. benefit analysis and present your case to the executive team. An analysis of expenses coming from bugs that would have been solved if you had the right tool in place is bound to work in your favor.

    3. Team communication and collaboration

    Test automation generally requires more communication and collaboration. If your team finds it difficult to get genuinely involved in setting automation targets and objectives, you’re in trouble.

    You need the active involvement of your team here. Automation relies on past testing data and experience or even proof of concept that only your team can deliver. For the automated testing to work, the entire team needs to be on the same page.

    Automation testers should be in touch with not only the business team, but also manual testers, developers, and technical architects. They’re the ones who present the ROI of automation to the executive management.

    You need solid team management support in place before you begin automating your tests. Create a collaborative environment where team members can easily communicate to deliver test results on time and with as little risk as possible.

    Test automation

    4. A testing approach

    That point is easily one of the most significant challenges for automation engineers. Testers need to identify the right test automation approach to match their needs. To do that, they must ask themselves questions like:

    • How will we reduce the cost of implementing and maintaining test suites?
    • Will our automation tool help us develop useful test reports and metrics?
    • Will these suites have a long lifetime on our team?

    That’s especially important if you’re working in an Agile environment where the tested application changes regularly and test automation needs to be implemented in a way that recognizes these changes, keeping the team up to date with a cost-effective maintenance strategy.

    That’s why you should consider getting an automation solution that detects these issues and automatically updates or re-validates the test.

    5. High upfront expenses

    Automation costs a lot at its initial phase. While the approach is very useful in Agile contexts, the expenses involved in setting up a test automation suite prevent many teams from incorporating automated testing into their practice.

    Before presenting the automation suite to the management, make sure that you analyze and build a test automation framework, where you include elements like libraries of reusable functions. Add both hardware and software costs as well.

    And don’t forget about licensing costs if you plan to use a paid software. And even if you opt for an open-source solution, you’ll still need to consider the expenses related to training and maintaining them.

    Convincing the stakeholders that automation is worth all that effort is a difficult task. But the potential ROI is a good argument, so prepare a plan together with a budget, and go for it.

    6. Accurate use of testing resources

    Those who aren’t convinced about test automation often say that it can be carried out by manual testers thanks to tools that help to record test scripts and play them back. That’s simply not true.

    Test automation requires teams to design and maintain test automation frameworks, together with test scripts, build solutions, and other critical elements.

    For automated testing to work, teams need a solid knowledge of the framework’s design and implementation. That’s why you should build your team on testing resources that have strong programming skills and test automation knowledge.

    7. Stakeholders and end-users expectations 

    It doesn’t matter how great your tools or resources are – you should always remember that testing can’t be carried out in a vacuum. It serves an objective: improving your product.

    While test automation is great for automatically checking the system for bugs, you still need humans to complete the non-automated testing sections. And these tests bring plenty of valuable information to make the best possible decision when improving the system – not for the sake of developers, but the end-users.

    That’s why you need to ensure that your testing team, the management, and other stakeholders all agree and understand the desired results of your automation plan.

    8. When to begin and stop testing?

    That’s a question all test managers face. Hopefully, they know how to judge the importance of testing processes that have been carried out by that time. But the start is just as important as the finish. You don’t want to initiate automated testing right at the wrong stage of your software’s life cycle, do you? That’s just a loss of resources. Then start with manual testing, and your engineers will be able to tell when the system is stable enough and ready for automated testing.

    Automation is a hot topic today. However, implementing automated testing suites also presents some challenges everyone interested in automation should take into account.

    If you’d like to learn more about streamlining test management and bug tracking using Jira Software, check out this video tutorial series on our YouTube. Also, read more about requirements and test management:

    [contact-form-7 404 "Not Found"]