We reduce the test execution time by 99% for Venca
1. Status
Venca is a leading fashion ecommerce company in Spain and Portugal. It has 3 development teams made up of about 5 people each.
Conduct releases approximately every 3 weeks on a scheduled basis.
2. Problem
They performed manual regression tests consisting of 130 tests in 6 different environments (desktop tablet and mobile, in Spain and Portugal). All on a single device.
A full regression was never run. If it had been done, the approximate time would have been 16 hours.
They had some unused test suites programmed in SpecFlow. They were based on tests with literals, hardcoded environments, the need for access to databases, etc. Patterns such as pageobject and continuous integration were not used.
In this situation, the cost in resources to perform reliable regression testing on each release was extremely high. The releases had defects that were eventually found by end customers in production, significantly worsening the user experience.
3. Solution
Testing is automated with WebdriverIO, covering web, tablet and mobile. Executions are parallelised to increase execution speed.
The entire test system is orchestrated from the CI/CD server, which is managed entirely by Redsauce, relie- ving the client of this task.
A device farm is used to run the tests on different browsers, mobiles and operating system versions.
Tests are run on demand with the possibility to link them to a trigger associated to the repository.
4. Results
The test execution time has been reduced from 16 hours to 45 minutes.
Immediate automatic reporting of all test results is available.
The green light from automated testing is considered a guarantee for release to production. This require- ment has increased the product team's confidence in the robustness of any release that has successfully passed the automatic tests.
The number of defects found in Production has decreased considerably, improving the user experience.