Launching a test automation approach is undoubtedly a long-term investment, provided that you think about it beforehand and make a selection. It is not useful to automate everything, nor should you automate everything. In reality, automated tests, although quite successful, are not sufficient in themselves to provide a complete and relevant evaluation of an application. It is necessary to combine manual and automated tests to ensure the quality of web or mobile application development and thus obtain an optimal return on investment. But how do you know which tests to leave in manual execution and which to run automatically? By thinking about four criteria: frequency, criticality, coverage and complexity.
It is essential to automate the most recurrent and repetitive tests, i.e. the tests that will be executed at each deployment. These are generally non-regression tests and acceptance tests. The question to ask yourself: when I deliver my application, what should I test each time? The tests that come up most often are the ones that should be automated as a priority to save a lot of time.
It is relevant to target functional tests that concern critical paths and functionalities, i.e. those that have a stake for the company, whether it is financial, legal or image-related. For example, in the case of an e-commerce site, if a payment page does not work, the user will not be able to finalize his purchase and this will have a direct impact on the company's turnover. The ROI is therefore more important in this particular case.
You also have to think about the coverage of the tests, which is increasingly wide with the multiplication of channels. If you have to run tests on several browsers (Chrome, Firefox, Safari, IE), on multiple devices (smartphone, tablet, computer), this quickly increases the number of executions to be carried out and automation can save considerable time.
Some tests are more difficult to automate than others, because they are technically complex, with many actions to perform, and therefore require more time to develop, but also to maintain. Most of the time, when you start automating, it is better to focus on simple tests that can be executed quickly, which allow you to identify critical bugs quickly. This does not prevent the automation of more complex cases that will use the existing cases to generate longer and more specific paths.
On the contrary, some tests are not suitable for automation, such as those concerning evolving features, but this will be the subject of a future article...