In the military jargon, a campaign represents the whole of the operations carried out by an armed force on a given place, in a limited time. And honestly, we could use exactly the same definition for a test campaign! This is perhaps why it is called a test campaign. Because it is simply the execution ofa battery of tests over a determined period of time.
What is it for? A test campaign allows you to test the application or the site as a whole or in part in the most efficient way possible. Its objectives can be multiple: to verify that the behavior of the application is still the same after software updates, to verify that the addition of new functionalities has not introduced regression on the old ones(regression tests), to verify the behavior of a new functionality to be implemented (validation tests), etc. But the purpose is always the same: to make sure that the software, or the mobile application, or the website, works properly.
As there are several families and types of tests (functional, non-functional, regression, validation, acceptance, exploratory...), there are several types of campaigns, depending on what you want to test.
The validation campaign consists in verifying that a new feature or a new product as a whole (mobile application, website) behaves as expected.
It can be made up of many test cases, which allows you to really test the new feature or product in depth.
Composed of regression tests, preferably automated, the regression campaign is used to verify that the addition of new features has not led to bugs or changes in the behavior of existing features.
It usually covers the entire product (software, mobile application, website), but requires fewer test cases than the validation campaign. And most importantly, it must be executed regularly.
The acceptance campaign ensures that a code delivery has not resulted in a critical bug or regression in the application, site or software.
It covers the main functionalities, which makes its execution rather fast (5 to 30 minutes) and requires a rather small number of test cases. Each code delivery must be accompanied by an acceptance campaign composed of vital tests.
The Suri-tip: The choice of the test campaign depends on your needs, your resources and the time available. But be aware that it is important to prioritize the different types of campaigns. To optimize time, it is important to launch and validate an acceptance campaign before launching a regression campaign, which allows you to have a quick and efficient status of the platform before launching the big guns.
If validation campaigns are difficult to automate, or at least more complex and of less interest, regression campaigns should be automated as much as possible, because they must be done regularly and therefore represent a waste of time manually. As for acceptance campaigns, they MUST be automated as a priority (if there is a choice to be made), because this allows bugs to be detected much more quickly and therefore to be corrected as quickly as possible. The quality of your application depends on it!
The success of test campaigns depends on test planning (test plans). These plans make it possible to define what will be tested (which functionality, all or part of the product), why, how, when and by whom.
As defined by the ISTQB, the test plan is " a document describing the scope, approach, resources and schedule of planned test activities. It identifies, among other things, the items and features to be tested, who will do each task, the degree of independence of testers, the test environment, the test design techniques, and the test measurement techniques to be used. "
But the test plans must also include a risk analysis, to indicate the limits of these choices and their possible impacts on the product.
What happens next? Afterwards, come the 5 other steps that constitute a test campaign:
According to the test plans, it is at this stage that the tests to be executed are chosen: unit, integration, system, acceptance, validation, regression, etc. The priority of these tests and the conditions under which they will be executed are also determined. This phase must not be missed because it is here that we decide how to evaluate the quality of the product.
Before the campaign can be executed, it must still be created in the chosen test management tool. To do this, you must set up the test environment, with its data, and also prepare the tests. Note: this step is also used to write the tests for the validation campaigns of new features.
This is the execution phase of the test campaign. The test cases are run according to the test plan and the configurations defined with the business. It is obviously at this stage that the tests are executed, but also that they are analyzed, that the anomalies identified are validated and that a decision is made to suspend the campaign if necessary.
This stage is also very important because it is here that we can study the results of the tests and verify the anomalies detected in order to proceed with maintenance and correction. It also allows us to assess the difficulties encountered, the test cases to be updated, the changes to be made for the next campaigns, etc.
Once the test campaign has been executed and analyzed, there is only one thing left to do: archive it. In the case of a regression or acceptance campaign, this means doing maintenance, setting up new test cases and deleting or modifying the old ones. Without this, the test campaign will not be as effective as it is supposed to be.
As quoted in the Tester's Tavern, a test campaign is not just about running test cases. It's a whole process that needs to be thought through and implemented. There are choices to be made, priorities to be defined, tests to be determined, not to mention the results to be analyzed. And above all, these choices must be made according to your needs, but also your human, material and financial resources. In manual as in automation, it is all a question of preparation.