Playwright vs Serenity BDD

Choosing the appropriate tool for test automation is a critical decision in software development. In this article, we will explore two of the main competitors: Playwright and Serenity BDD. Each has its own strengths and distinctive features. At Redsauce, we have used both, and based on our experience, we present the main advantages of each one to help you choose the framework that best suits your project.

Playwright

Playwright is an open-source automation library for browsers developed by Microsoft and published in 2020.

It was developed by a team of engineers who had previously worked on similar projects such as Puppeteer at Google.

Later, playwright/test was incorporated, a runner specifically designed to get the most out of the API and make end-to-end tests more robust and simple.

Since its inception, Playwright has been actively maintained and has experienced rapid growth and adoption in the web testing community.

Companies such as Disney, Adobe, ING and Microsoft itself use it in some of their projects.

Features/strengths

• Provides support for development across multiple browsers, such as WebKit, Firefox, and Chromium, covering Chrome, Edge, Firefox, Opera, and Safari.

• Compatible with multiple operating systems, including Windows, Linux, and macOS.

• Supports multiple languages, such as JavaScript, TypeScript, Python, Java, and .NET, allowing you to choose the environment that best suits your needs.

• Incorporates features such as isolated browser contexts (enabling reliable parallel executions), automatic waits, smart asserts that automatically retry, and video recording of executions.

• By default, it runs test files in parallel, executing the tests in each file in series, although this is fully configurable.

• It has effective development tools such as codegen (generating code instructions by recording user actions), inspector (a graphical tool for debugging), and trace viewer (detailed viewer of execution traces).

• Developed with a modern and unrestricted architecture, it allows interaction with multi-page and multi-tab websites in a similar way to a real user. In addition, it allows you to easily manage frames and browser events.

• Besides the public GitHub repository and online documentation, they have tutorials and conference talks in their YouTube channel, and a Discord server for the community.

Cons/weaknesses

• It does not support real devices such as a smartphone or tablet, only being able to emulate them.

• It does not support old browsers.

• The reports it generates are basic (HTML, JSON, JUnit...), although custom ones can be implemented.

Serenity BDD

Serenity BDD is a software testing framework designed to facilitate the creation and maintenance of automated tests in a human-readable documentation style. It is specifically focused on the automation of acceptance and functional tests, providing detailed reports on the status of the tests.

Serenity BDD arose from the need to improve the quality and understanding of software tests in the Agile methodology. The project (initially known as Thucydides) was started by John Ferguson Smart, a renowned expert in testing and Agile development.

It is used by companies such as Yandex, General Electrics, UBS and Barclays.

Features/strengths

• Provides detailed, readable reports that include screenshots and automated documentation, aiding in the identification and resolution of issues.

• It easily integrates with many popular development and testing tools, such as JUnit and TestNG, as well as build tools such as Maven and Gradle and Java ecosystem libraries such as Spring or Lombok.

• Although not strictly necessary, when paired with Cucumber, it uses a clear and expressive language called Gherkin to write test scenarios, making it easier for development and testing teams to collaborate, and improving comprehension by non-technical individuals.

• It offers support for design patterns such as Page Object and Screenplay, making it easier to create and maintain tests.

• It automatically handles WebDriver, so that developers do not have to worry about initializing and shutting down WebDriver instances.

• It supports Appium for mobile test automation on Android and iOS.

Cons/weaknesses

• The supported browsers depend on the modules used, such as Selenium webdriver or Browserstack, which can result in slower interactions with them.

• Likewise, the video recording capabilities of the tests depend on the used modules and their configurations.

• It does not provide parallel execution capabilities by itself. Again, this feature depends on the modules used and their configurations (e.g. maven-failsafe-plugin or Cucumber-JUnit).

• Since it is implemented in Java, it only supports Java, Groovy or Kotlin (JVM languages) as programming languages.

• Advanced support, such as online courses and personalized support, is offered as a paid service.

Choosing a framework for test automation always boils down to a matter of needs to be met and available resources. For instance, if the application we want to test is built with Angular and the development team is highly specialized in this framework, Protractor may be the most suitable choice.

As for the protagonists of this article, both alternatives have strong points in their favor, and we could add other contenders to the fray such as Cypress, WebdriverIO, TestCafe, Puppeteer, and many others.

If we appreciate browser compatibility, execution speed, and tools that make development easier, Playwright could be the right choice. Playwright tends to be faster due to its efficiency-focused design and parallel execution.

If we are more interested in BDD, rich documentation, and ease of use for non-programmers, Serenity BDD may be more appropriate.

However, thanks to the extensibility and integration capabilities of both products, we can (albeit with the extra effort it may entail) further tailor them to our needs. For example, by adding Cucumber to Playwright to enable BDD functionalities.

Who knows, maybe in the future we might see SerenityBDD using Playwright instead of Selenium.

If you still have doubts about which tool is the most suitable for your project, you can contact us. We will review your project and help you choose the most appropriate framework according to your needs.