Learn web automation in 6 actions

Posted by Kenan Rhoton

Web Automation is not a complex issue. Let me rephrase that. Using Web Automation tools is not a complex issue.

Maybe you want to get writing some E2E tests. Maybe you just want to alleviate some manual tasks. Maybe you're just curious. Whatever the reason, you've decided to give Web Automation a go.

And there's just so much stuff.

But actually, there isn't! I'm here to tell you that, in practice, you'll only ever use 6 actions in 99% of cases in Web Automation, and understanding how these work is the key to good test development. For the remaining 1% of cases, you'll have to read the docs for whichever automation tool you're using.

Navigating to a URL

Now this first one might seem obvious and unhelpful. In Webdriver.io (which we'll use for the examples in this article) you would simply do:

await browser.url("https://google.com")

Not exciting at all, right? Wrong.

See, the great thing about navigating through URLs is all the things you don't need to do. No clicking, no looking for IDs in web elements. Just go. There's two areas where you can gain mastery in the humble URL-navigation action:

Choosing a good starting spot: Starting your tests as close as possible to whatever you need to test will make writing them easier and execution much faster.
Skipping steps when appropriate: Recognizing when you can just browse through the URL instead of through web interaction can again simplify and speed up everything. It is important to understand that you should only ever skip actions you don't want to test in the current test and that you should skip as many of them as possible.

Finding Elements

![Jackie Chan meme nobody use ids](images/18 =100%x*)

If there ever was a cause for headaches, it's this one.

If you're lucky enough to work at a project where you are both working on Web Automation and have some influence on the development of new tasks, or one where developers are at least aware of the importance of classes and IDs, then this step is relatively simple:

const element = await $("#great-id")

If you're not one of the lucky 500, this can quickly devolve into:

const element = await $(".flex.container.list > ul > div > a > .w-50")

Or some other bizarre selector. Thankfully, there are some skills that we can develop to become better at selectors:

Clarity: Often it's more about finding the clearest selector that identifies what we want than simply finding one that "works".
JavaScript support: At times, we will have a much easier time if we just take some actions through JavaScript to get to our target element from a different one than if we attempt some arcane selector voodoo magic.
CSS selector and XPath knowledge: Knowing our options is important, and sometimes small features can make for a readable and precise selector.
Complaining!: If you're not part of the development team itself, it's often much better to classify hard-to-reach elements as a bug to be fixed (please add a class or ID, thanks!) than to, once again, reach for arcane selector voodoo magic.
Selecting multiple elements: Recognizing when it's best to find multiple elements at once instead of just the first match.

Clicking

Did I say selectors give you headaches? Nah, if there ever was something in Web Automation that was hard it's clicking.

Wait, what do you mean? Isn't clicking just:

await element.click()

You, sir/madam, are correct! And it's horribly complicated!

You see, sometimes when we click something, we're not clicking what we think we're clicking (clicked on that button? Nope! It's actually a

tag around the button!), which makes for some non-obvious errors when clicking on our beloved element doesn't work as expected.

But that's not even the bad part.

No, the bad, horrible, terrible part is overlapping elements.

You want to click on that login button? Nope! Cookies banner in the way! Want to click on that subscription link? Tough luck! You didn't scroll enough and it's behind the footer!

So clicking itself may be simple, but you'll find yourself having to make a number checks and preparations for that single click:

Overlaps: Check that some elements that we know can overlap our target element are not overlapping it (if they are, well, do something about it!)
Screen location: Check that the element's position doesn't hide it under somewhere else, and scroll if needed.
Confirm target: Make sure the element we're clicking is the element we actually want to click.

Typing Text

Finally a straightforward one! Unless the website uses some dark magic (in which case, note the "Complaining!" bullet point a few sections back) this action is extremely simple:

await element.setValue("potatoes")

Congratulations! You type "potatoes" and are now legally a Web Automation Expert (Disclaimer: you are most definitely not a Web Automation Expert. Web Automation Experts don't, in fact, exist).

Retrieving Text

This one is just as simple as the previous one:

const text = await element.getText()

And just like that, we have OH GOD, NO! WHERE DID ALL THIS EXTRA TEXT COME FROM?

Okay, so there's a minor hiccup: in most Web Automation libraries, getting an element's text fetches all its child element's text along with it. Which is generally what you want (if there's some text in there with some emphasis on it do you really want to drop it?), but you've got to learn to treat it correctly:

Containment: Generally when fetching texts we should test for containment rather than equality (so dust off your trusty text.includes()!)
Element selection: Sometimes we can simplify everything by just picking the right element to avoid said child text shenanigans.

Waiting

Now this is a skill Web Automators have perfected: waiting for your test suite to end while cracking conversation by the coffee machine. Also, code waits:

const elem = await $('#someText')
await elem.waitUntil(async function () {
    return (await this.getText()) === 'I am now different'
}, {
    timeout: 5000,
    timeoutMsg: 'expected text to be different after 5s'
});

Here's a secret: in Web Automation, we dont expect, we wait. Well, not quite, cause in many libraries expect is just secretly wait. But it's the concept that matters!

What's the difference, you ask?

When we expect, we're telling our test "This should be true at this exact instance and I don't care if it would be ready in 5ms: it needs to be there now!" Flips table.
When we wait, we're telling our test "Hey, we understand browsers do a lot of asynchronous stuff, so if within the next 5 seconds or so this happens, please do tell me and we can then continue with the test. Thank you and have a nice day!"

So basically, waits make our test code more robust and resistant to minor fluctuations such as small network delays or any other of the myriad of reasons a website might slow down. And they also make us specify what we consider an acceptable response time. To use them correctly we need to:

Learn to write good wait bodies: Understand what needs to be run everytime we want to check our condition (for example in Webdriver.io we don't need to search for the element every time unless we're waiting for its existence).
Conditions: These are the desired final state we are testing for, and writing them well is crucial

Honorable Mention: Deleting Cookies

Now this action doesn't really fit much into that "99% of uses" since you'll basically write it once:

await browser.deleteCookies()

However, that once will be in your beforeEach function to ensure that every test can run independently of each other!

Wrapping up

We hope this post has helped demystify Web Automation a little bit and made it less scary. It is, after all, quite simple!

And if you ever get stuck, you can always contact Redsauce.