Playwright vs. Selenium: A Face-off of Open Source QA Test Automation Tools

Playwright vs. Selenium: A Face-off of Open Source QA Test Automation Tools

After many years of being the dominant open source tool for QA automation, is Selenium’s status being challenged by a sleek and powerful new tool built upon a radically different approach to automation in Playwright?  Let’s dive into a face off of this champion vs contender contest of QA automation tools.

Selenium is the entrenched incumbent which has grown to dominate the landscape of test automation since its birth in 2004. And for good reason, it has proven itself to be a flexible and effective tool with an ecosystem of third party extensions. 

Playwright is a relatively new tool which addresses two of the key challenges encountered by testers with Selenium: performance and usability. Originally created by Microsoft, Playwright was open sourced in January 2020. Its feature set is headlined by a vastly reduced test execution speed which can create a dramatic time-saving benefit when executing vast test suites on enterprise systems. This greater execution speed is paired with an intuitive and easy to use design. Simply put, Playwright delivers a more seamless and less labor intensive experience for testers in crafting automated tests. 

We’ve been evaluating and using both tools in our test automation work at RIA Solutions Group and this article is created to share the results of our analysis and experience. 

Note to readers: we are assuming that anyone familiar with test automation is likely intimately familiar with Selenium so we have avoided detailed analysis of that tool in favor of leaving more space for our Playwright analysis.

A Look under the Hood of the Playwright Tool and its Key Benefits

Playwright stands out due to its innovative and performance focused approach, which tackles the challenges and limitations faced by Selenium. This upstart tool not only overcomes obstacles but also enhances the efficiency and effectiveness of testing processes, resulting in significant time savings. 

Playwright Internals

Unlike Selenium, Playwright offers built-in support for managing browser drivers, eliminating the need for external libraries, and simplifying the handling of different browser versions. 

Playwright leverages WebSockets for greater efficiency of browser test execution, particularly in scenarios requiring real-time communication. While Selenium HTTP can work with WebSocket, handling WebSocket communication may necessitate additional third-party libraries or custom implementations. Selenium by default relies on the less efficient HTTP protocol for UI interactions.

Figure 1: Playwright vs Selenium Architecture

Easier Cross-Browser Compatibility

Cross-browser compatibility is built into Playwright, requiring no additional tools or configurations.  This contrasts with Selenium’s reliance on external libraries. Playwright’s “Any Browser / Any Platform / 1 API” feature adds versatility, making it a compelling choice for diverse testing environments. Additionally, Playwright simplifies API testing without external libraries, a task for which Selenium often depends on tools like REST Assured.

Code Generation

Playwright’s built-in Codegen feature sets it apart by automatically generating test scripts through recorded browser actions. This functionality proves invaluable for those new to automated testing, offering a user-friendly entry point that simplifies the automation process.

Figure 2: Playwright Code Generation

Auto-Waiting and Assertions

Integrated features, including auto-waiting and assertions, are standout aspects of Playwright.

Auto-waiting addresses a significant challenge in eliminating the inconsistency of test results seen with Selenium. With Selenium, waiting has to be explicitly coded into the tests. Even with coded in pauses, system lags can result in erroneously test failures.

A Short Selenium Code Example

Below is a short snippet of real-life Selenium code which highlights the explicit waiting that needs to be coded into many tests.

 

public void clickOnCommandIcon(String name) {

actOnHouseholdContentIframeAndReturnToDefaultContent(() -> {

waitForLoadingIconNotVisible();

toolsButton.waitUntilVisible().click();

getCommandIcon(name).waitUntilClickable().click();

waitABit(500);

waitForLoadingIconNotVisible();

});

}

Example 1: Selenium Explicitly Coded Waiting

The same code example but written in Playwright

Below is the same snippet of code written in Playwright (with Java) which highlights there is no need for the explicit waiting handling.

 

public void clickOnCommandIcon(String name) {

actOnHouseholdContentIframeAndReturnToDefaultContent(() -> {

toolsButton.click();

getCommandIcon(name).click();

loadingIcon.isHidden();

});

}

Example 2: Playwright Code Without Explicit Waiting

 

With Playwright, assertions cause functionality to be automatically retried until conditions are met, eliminating the need for coding explicit pauses.

Test Results with Screenshots and Charts Natively

Playwright not only provides its own test runner but also grants access to high-speed detailed reporting. The final report of testing results is easy to follow, and the built-in capabilities for taking screenshots and screen recording are notable advantages. In contrast, Selenium requires an external library, such as Serenity BDD, for generating reports with pie charts and screenshots. Notably, it also lacks the video recording capabilities offered by Playwright.

Here’s a report with a video, but there are no pictures in it. The good thing about this report is that you can watch the video right away without having to download it to your computer. It has its own video player built in.

Figure 3: Playwright Video Generation

Here’s another report with multiple screenshots. You can easily move through the different steps of the test by using the timeline bar at the top of the report. It helps you navigate visually.

Figure 4: Playwright Test Execution Timeline with Screenshots

Now, there’s a report with both multiple screenshots and a video recording. However, the downside is that you have to download the video to your computer before you can watch it.

Figure 5: Playwright Timeline with Screenshot and Video Link

A Balancing Act: Playwright’s Drawbacks

While Playwright boasts numerous advantages, it’s imperative to acknowledge that it comes with its share of drawbacks. 

Limited Community Support

From a technical standpoint, one notable challenge is the limited community support, owing to the framework’s relatively recent entry into the market.  Community support is critical for the long-term viability of any open source project which makes this a non-trivial concern.

Difficulty in Porting Selenium to Playwright

Porting an existing suite of test automation from Selenium to Playwright can be problematic. There are multiple reasons for this but the most notable is the limited code language support of Playwright.

It’s essential to weigh the above considerations carefully when evaluating the adoption of Playwright for automated testing. 

The Net Results of our Experience and Analysis

Choosing between Playwright and Selenium requires careful consideration of factors like the learning curve and community support. For new projects, opting for Playwright can expedite maturity, but switching from Selenium may not be practical, especially with tight project deadlines and existing Selenium expertise. Additionally, Playwright presents constraints in terms of handling fewer browsers and programming languages compared to its counterpart, Selenium.

Framework maintenance is a critical consideration in choosing between Playwright and Selenium. Playwright stands out for its long-term ease of maintenance, designed with a focus on built-in functionalities, reducing reliance on extensive third-party libraries. In contrast, Selenium, while offering core functionalities, may require additional third-party libraries or tools for advanced features like capturing screenshots or videos. 

The streamlined setup process of Playwright further enhances its maintenance efficiency, providing a single package for all dependencies with a simple installation command. Selenium users, on the other hand, often need to manage browser drivers separately, involving additional configurations even with tools like WebDriverManager, adding an extra step compared to the more straightforward setup offered by Playwright.

So which tool wins in the face-off of test automation tools?  The net of our experience and analysis is that both are great tools but the answer of which is better is highly context dependent.

 

Happy testing!