When it comes to guaranteeing the quality of your web or mobile application, a range of tests are available, including the non-regression test (NRT), also known as the regression testplays an essential role.
Read on to find out what non-regression testing is, its benefits, frequently asked questions on the subject, and best practices for performing it.
What is a non-regression test (NRT)?
In the world of Web development, a regression occurs as soon as a code change has an impact on existing code.
For example, this could be due to a bug fix, a software update, or the addition of a new feature.
A non-regression test (NRT) verifies that changes made to the software, site or mobile application, such as the addition of a new feature or an update, have had no impact on existing functionality.
The different types of non-regression tests
Non-regression testing can be carried out in a number of ways, depending on the target need or the company's resources:
Corrective non-regression test: uses existing tests, provided no major changes have been made to the product.
Full non-regression test: involves re-testing all the elements of the product, thus verifying all the changes that have been made since the beginning.
Selective non-regression testing: allows you to select certain tests from a set in order to inspect only those parts of the code that have been impacted.
Progressive non-regression testing: consists of creating new tests when established tests are no longer useful, for example when product characteristics change.
Partial non-regression test: performed when different modules are under development and about to be merged with the main version of the code.
Unit non-regression test: used to test code individually, without taking other elements into account.
Non-regression test (NRT) FAQ
Why do non-regression tests?
The first reason for non-regression testing, which should be the main reason for all other types of testing, is to guarantee the quality of the software.
Ensuring a top-quality product simultaneously improves the user experience and the company's image.
What's more, non-regression testing prevents bugs that could hinder certain functionalities, thus impacting sales and the company's reputation, while avoiding potential security flaws.
Regularly running non-regression tests saves time and money, as it's much more complex and costly to rectify a bug once it's in production.
In the context of developing mobile applications, websites or SaaS software, subject to frequent updates to meet customer requirements, non-regression testing becomes even more essential.
When should non-regression tests be carried out?
Non-regression tests can be carried out at all levels of the test plan, and we recommend that they be carried out as regularly as possible, as soon as a modification or update is made, and as early as possible in the product design phase.
In general, non-regression tests are performed whenever :
- A correction made to the code to resolve anomalies
- A new feature added
- Modification of existing functionality
- An update in the environment (e.g. data)
- Source code optimization
What are the differences between functional validation and non-regression tests?
Functional validation tests are a type of test performed to verify that each function or feature of the software application works in accordance with the requirements specifications.
They are carried out using test cases that recreate all possible scenarios, both negative and positive. Ideally, functional validation tests start at the initial stage of product development and verify :
- Missing features
- Incorrect specifications
- If interface errors persist
- Gaps that exist during the requirements phase
Well-executed functional validation tests help deliver a top-of-the-range product, helping to produce a bug-free product or software to ensure end-user satisfaction.
The differences between these two types of tests
Objectives: the aim of functional validation testing is to determine the extent to which the developed application meets the desired requirements.
The aim of non-regression testing is to verify that any changes to the application or systems have not resulted in code breakage, and that the system is functioning correctly.
Executed test cases: functional validation tests enable us to understand all cases and functionalities that have never been tested and executed before.
Test cases are executed when a fault is identified in relation to a requirement. It is then corrected and assigned for retesting. During the retest, if the defect is resolved, the related test cases that failed earlier are passed.
The non-regression suite contains cases that have been previously tested and resolved.
Basically, in non-regression testing, test cases are executed to ensure that changes have no impact on previously tested functionality.
Process used: the functional test process begins with the testers reading and understanding the requirements, which raises a discrepancy in the event of a discrepancy in the requirement, followed by the identification of the test input.
Transmit input values to systems and compare output with expected results.
If the result does not match, the fault is reported and the test case is marked as having failed.
In contrast, the non-regression testing process is entirely different, as this activity only takes place when the existing application is modified or new functionalities are added.
Feasibility of automation: functional validation tests are initially carried out manually. Once a feature is stable, test cases are automated.
In non-regression testing, test cases can be executed either manually or automatically. As test cases are already stable by default (since they have already passed their functional test) in non-regression tests, they can be automated.
Functional test cases don't require much modification, as they are fewer in number and focused on a specific functionality.
In non-regression testing, on the other hand, test scripts require more maintenance and may contain older test cases.
Test cases may contain features that have changed, new features that have been added or some features that have been removed, so the regression suite must be updated after each release.
What is a graphical non-regression test?
Graphic non-regression tests, also known as graphic comparison tests, are essential for verifying the visual quality of a site or application.
Basically, these tests ensure that every element on the same page, such as buttons, text, photos and images, is displayed correctly in terms of location, size, color and shape, whatever device or browser users are using.
This is called non-regression, because these tests are used to check that the site or mobile application resembles its previous version to the nearest pixel (pixel perfect) and is legally compliant.
What's more, these graphic tests can be used to verify the presence of legal displays that are mandatory in certain countries to ensure regulatory compliance.
How do I run non-regression tests?
Regression testing is very time-consuming and repetitive. Therefore, although some test cases have to be performed manually, most are automated using an automated testing tool to detect potential bugs that appear following a modification, and resolve them before they impact users.
Automated non-regression testing also reduces costs by saving potentially lost customers due to sub-optimal user experience, as well as speeding up the testing and debugging process.
Finally, automation enables continuous, reliable testing throughout the development cycle, ensuring that regressions are identified and resolved quickly, thus reducing the risk of defects reaching production.
What are the limits of non-regression tests?
Despite automation, it is still necessary to create scenarios, especially for complex functionalities, which can delay execution and therefore delivery.
Under time and budget constraints, QA teams have to prioritize the most essential tests, as not all regression scenarios can be executed.
Mr Suricate - French leader in no-code automated testing
At Mr Suricateour mission is to protect the customer's brand image and increase sales, while ensuring that the user experience runs smoothly by detecting bugs before and after production.
