Functional test, frontend or backend?

Have you ever wondered if instead of simulating the actions the user would take to perform the tests it would be better to make the API calls directly and analyze the response?

If so, in the following lines we will try to show you the advantages and disadvantages of both cases according to our experience.

What is frontend functional testing?

They are tests that are done by performing the same actions that the user would do automatically. They check the effects that they have at a visual and functional level. They can also be done to check if the process a user has to follow is intuitive.

public void loginAs(String user, String password) {
    loginPage.usernameTextbox.sendKeys(user);
    loginPage.passwordTextbox.sendKeys(password);
    loginPage.loginButton.click();
}

What is API testing?

In these type of tests the different actions that are carried out in the use cases are done against the API directly, sending the appropriate call, thus simulating that the user has performed a specific action.

@Step("Authenticate")
public void authenticateAs(String id, String secret) {
	/* set session variable used for json's template placeholders */
    Serenity.getCurrentSession().put("clientId", id);
    Serenity.getCurrentSession().put("clientSecret", secret);

    given()
	    .body(PebbleTemplateMethods.evaluate("testdata/authenticate.json", extractAuthenticateData()))
        .when()
        .post(ScenarioHooks.authUrl);
}

Comparing frontend and backend functional testing

One of the biggest pros in favor of API tests is the speed of execution of the tests which is notably faster given that we don't have to wait for the loading times of elements of the web page, however, sometimes it is necessary to wait for a response from the server that can take a few seconds. In that case when using Serenity we had to add an await().untilAsserted(), which meant having to import an extra library.

However, to be able to perform the API tests in a satisfactory way, it is a requirement to have a very good documentation with all the calls that can be performed, the configurations and the necessary parameters. This is not the case with the frontend tests, since by intention or by logic you can know the route to be followed without the help of any documentation.

API testing using Postman

With this in mind, we noticed that the tests were much faster to program, because you don't have to look for good identifiers for the particular elements of the page you want to interact with. In addition, performing actions on the frontend usually means more than one interaction with the page, which can be avoided by making the call directly.

This leads us to the point of not having to worry about whether the objects are well identified so that the tests are more robust and are not affected by the position updates of the page.

For example, in this case where the identifier depends on the position:

    @FindBy(css = "#root > section > div > div.core-section.col-12.col-lg-9 > div.post_list > div:nth-child(1) > div > a")
   WebElementFacade previewImage;

If in a new version of the page the object still has the same functionality but different location, the test would fail, but it would be a false negative, that's why the code would be more unstable and harder to maintain. However, an API call is usually unchanged, so we don't have to update the code frequently.

An aspect that we also think is an advantage of the API tests is that when you are running them you don't need to have the browser open, which can't be done with some of the existing frameworks, unless they have the headless option, in which case it would be a draw between the two options.

A disadvantage of these tests is that you cannot simulate 100% of all the interactions that the user would make against the page since there are times that not all the actions generate a request to the API. For example, in the purchase process, in the steps of filling in the data and selecting the shipping method, an API call is usually not made until the whole process is completed.

Frontend testing using Cypress

Another disadvantage to consider is that when you are not testing the interaction with the frontend you are trusting that the API calls are triggered correctly when the specific actions are performed. This is a big disadvantage for us because the API may work perfectly, but if the button that should trigger it is not working as it should, you would not detect this error with this kind of test.

This leads us to the next counterpoint, which concern the checks. Which we feel are more limited since sometimes an action causes multiple changes in the front that in the API are only shown with a parameter change. This can be solved in two ways: The first would be to simply check the parameter change in the response and hope that this correctly affects the frontend and the second would be to add some checking in the frontend.

For example in this case:

$ ~ http http://localhost:4000/tags
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 352
Content-Type: application/json; charset=utf-8
Date: Wed, 29 Jul 2020 14:03:28 GMT
ETag: W/"160-maNWQR6ptXu2EC2MrZrPjhs9CL4"
X-Powered-By: Express

[
    {
        "Tag": "Ruby",
        "Usages": 3
    },
    {
        "Tag": "Javascript",
        "Usages": 5
    },
    {
        "Tag": "Sonar",
        "Usages": 3
    }
]

With a test to the backend we cannot see how the parameter Usages visually affects to the front.

Another point that we would like to emphasize is that at the code level the body of the API calls is usually somewhat difficult to reuse, as each action has certain characteristics. This means that a file is needed for almost every action, and sometimes for the same action but with different scenarios. This causes many similar files to be stored, whereas with the front would be to simply click in different places. We tried to solve this problem using the Pebble library but we believe it made the code more illegible and difficult to maintain.

And finally, at report level we realized that Serenity only showed the names of the endpoints in the steps. Which in our case, when using json-rpc, was not very informative because all endpoints were the same, so we had to add more code to show some specific titles in the steps of the report. On the other hand, for the frontend tests, screenshots are saved in the report right at the moment of performing the action, so if there is a loading or visual error, it can be quickly detected.

Conclusion: Which functional tests should I choose?

For us, the conclusion would be that the best thing to do is to mix the two options. What we would do, the first time that we have to test an interaction with the web, would be to test interactions with the frontend and in the following tests, as we have already verified that the component sends the calls correctly to the API, send the calls directly without passing through the frontend to gain speed but without losing reliability.

For example, the first time that the login is made, it should be checked from the frontend, but once that functionality is tested, the following times that the web page has to be accessed, the login can be made directly via API, which is faster to access the page.

It is true that at first it may seem that the code becomes more complex as we have to implement the same functionality for web and API but this can be remedied with a good file organization.

We also want to emphasize that the need for a very good documentation to make the API tests is positive, since it will help in maintaining and further developing the project.

Finally, having compared the two options we believe that both have their pros and cons so it will depend on the requirements or objectives of the project to choose one or the other or, as we did in our case, choose a mix of both.

If you still don't know which of the two options is better for you, functional testing or API testing, write us a message and we will be happy to help you define the best strategy. Fill in the form of our contact page.