End-to-end (E2E) testing is a way to find defects in web applications by mimicking the behavior of real users.
The major weakness of this method is that the tests do not always represent the behavior of real users, which completely defeats their purpose.
In order to solve this problem, we worked in collaboration with the Laboratoire de Recherche en Informatique (LaBRI) of the University of Bordeaux to find a way to improve E2E tests by identifying exactly when, where and how the tests differ from the behavior of real users in order to correct them.
.png?width=640&height=360&name=Untitled%20design%20(34).png)
A new method put to the test
The method that was tested in this study had two basic components:
1. An automatic registration tool
First, a non-intrusive recording tool was used to record traces of test runs, as well as traces of real users interacting with the application.
To facilitate the understanding of the recordings, screenshots were associated with each interaction (made possible by the chrome extension API).
These clear visual representations of the traces provided a detailed understanding of the targets and any specific interactions.
2. Diff algorithm (Diffing Traces)
The goal of a diff algorithm/interface was to identify and highlight the differences between the executions of a test, and this same scenario launched by one or more users.
A trace captures the interactions performed via the web interface (clicks and keystrokes) and can be recorded during the execution of a test or when a user navigates in the web application.
Comparing a test trace from a script with a user trace allows to highlight their differences in terms of interaction, and to identify :
- Interactions missed by the test.
- Actions performed by the test tool, but not by the user or vice versa.
- Any other type of discrepancy.
Evaluation of the method
At the end, we found the algorithm was indeed able to identify :
- New interactions that are more relevant to the real user experience than those used in the test.
- Poorly formed traces, allowing testers to understand where to improve their tests.
These results led us to conclude that this process perfectly complements all E2E testing toolkits by improving the quality of tests implemented using the most popular frameworks (Playwright, Cypress, etc.)
We were able to identify exactly where the tests were either out of sync with the application or too far from the behavior of a real user.
In the future, we would like to build on this field experience by including more users for each scenario to reduce the subjectivity of the results and continue in partnership with the university to further develop this method.
Mr Suricate | Dedicated to the continuous improvement of your automated tests
Even the best software must be made even more useful and relevant in the digital world that is constantly changing as well as the demands of modern users.
Low-code solutions from Mr Suricate low-code solutions have helped countless companies accelerate their digital transformation, but we also focus on innovation to better meet the needs of today's and tomorrow's companies.
