Software testing tools: Cypress vs Webdriverio

Web automation tools have come a long way since the days of pure Selenium, with better programmer APIs and overall improvements on the developer experience.

It all still fulfills the same basic function, which is to enable a programmer to simulate real user interactions with the browser, and in some cases we even find our same old friends beneath the hood. However, we can't deny that, as developers, our experience writing functional tests has become so much better.

Here at Redsauce Engineering we have gravitated towards two different tools, with their pros and cons: Cypress and WebdriverIO.

Javascript

You might notice that both these tools are written in Javascript. As a company, we've rotated among different automation tools (starting with Watir) and the choice to move to Javascript was not made lightly.

We often work with other businesses to kickstart their functional testing, and due to this we always try to use technologies that are as familiar as possible to them to make the transition as smooth as possible. This led to us realizing that web automation is not really complex from a programming standpoint, but that its difficulty rises from the browser itself and the knowledge of how to correctly test a functionality. So we found that we didn't really mind changing the language, as most language's syntax is learned quickly.

We took a step back to focus on which language made the most sense for us, and came to the conclusion that, since our goal is to have our client's teams learn to make their own functional tests, we should choose a language they were already using. And since we're talking about testing web applications, the one language you cannot escape knowledge of is Javascript.

So we then started looking for the best Javascript-based web automation framework.

WebdriverIO

After a brief time using Protractor, we found WebdriverIO and have been defaulting to it since. There are a few reasons we particularly like WebdriverIO among all its competitors:

• Functions easily as a stand-alone library

• Easily mixed with Mocha/Chai/Cucumber/etc.

• Full project structure flexibility

• Both synchronous and asynchronous code is painless

• Great documentation

• Works with a Selenium backend, which makes its use with Appium extremely simple

As an example, here is a simple test which goes to a certain page, fills in a search bar, clicks the search button and checks that the shown page contains elements with that search:

describe('Search Engine', () => {
    it('should be able to search for a word', () => {
        browser.url('https://duckduckgo.com/')
        $('#search_form_input_homepage').setValue("Potatoes");
        $('#search_button_homepage').click();
        expect($('#search_form_input')).toHaveValue("Potatoes");
        expect($$('.result')).toBeElementsArrayOfSize({ gte: 5 })
    })
})

These reasons have made it stand out from other alternatives, as we value the flexibility and adaptability of the framework which allows us to build code that perfectly meets our needs.

Recently, though, we've started to also use a different technology, which is amazing for a completely different set of reasons...

Cypress

Basically, Cypress has one big advantage over every alternative: the developer experience is amazing.

Cypress comes with its own interface for launching tests and gathering reports (but it also works through the commandline). This interface comes with some very useful features:

• Complete console logs

• Time travel: you can go back to any instant in the tests and examine the DOM's state at that point, both visually and via Javascript

• Video capture: you can get a video rendering of all your tests

• Selector sandbox: you can check what a specific selector will find within the browser, even when time-travelling to a specific moment.

It does also come with some hefty costs:

• Works mainly through the cypress command, and is hard to use as a library

• It's very difficult to make conditional tests (where the test diverges depending on some state unknown at the start) which is not infrequent when you don't control the code under test

• Has a somewhat rigid structure, which is fine when doing a standalone testing project, but not ideal when adding testing to an existing development

• Essentially impossible to use on mobile (but does work in responsive mode)

• Firefox support is iffy

Here is the same above example coded using Cypress:

describe('Search Engine', () => {
    it('should be able to search for a word', () => {
        cy.visit('https://duckduckgo.com/')
        cy.get('#search_form_input_homepage').type("Potatoes");
        cy.get('#search_button_homepage').click();
        cy.get('#search_form_input').should("have.value", "Potatoes");
        cy.get('.result').should("have.length.of.at.least", 5);
    })
})

Conclusion: Cypress or Webdriverio

Here at Redsauce Engineering we have opted to choose both these tools, with some caveats. We thoroughly analyze a project and what we will need for it and depending on whether we need more flexibility or a quicker and smoother development speed we opt for one tool or the other. There is no clear winner here, but both these tools are amazing and we will continue to use them to provide the best possible service to our clients.