E2E & Integration Testing for any UI5 application using testcafe

Having stable automated integration and e2e tests greatly increase the stability and overall quality of your application. Still if we look on today’s SAP UI5/FIORI applications automated testing is – at best – a rare sight.

In my opinion the key cause for this is the high complexity of the available testing tools and their “non-conformity” to UI5. If you ever tried to work out writing stable tests using a combination of 50 npm packages and try to integrate them into the UI5 world, you will probably rethink your decision to work with automated tests at all. If you fought through the first setup, you probably saw that your test is unstable (fails every 10th time, UI is changing, testdata, ..) and you can not hand over the code to any other developer (far to complex) ==> Projects give up and just “let the customer do the manual tests”.

For me there is a clear vision. There should be a UI5 specialized framework which:

1. Is easy to learn and master for any junior developer
2. Allows to write stable e2e and integration tests
3. Is based on a modern and high-quality framework
4. Is specialized for UI5, both in getting element-selectors and the overall code syntax
5. Is easy to integrate in all common CI tools
6. Has embedded handling of code-coverage – even in E2E scenario

To achieve these goals, I’ve started a new npm package “ui5-testcafe-selector-utils”. As the package is slowly getting stable (and it is MIT license / OSS based), I want to present you the current status. I am happy about any feedback / critics (or whatever reaction you have). As there is a lot of information this will be blog-series. In this first part we will only implement a very simple test, to give you an idea how the code looks like and to motivate you to play around with it on your own.

The blog has the following parts:

1. Part 1: (This Blog) Overview and first test
2. Part 2: Enhanced Selection, Action and Assertion Possibilities.
3. Part 3: Structure your test and common best-practices
4. Part 4: Code-Coverage for Integration and E2E tests
5. Part 5: Integration in CI (Jenkins / Github Actions) / Authentification
6. Part 6: Use testcafe for SAP Analytics Cloud and SAP Lumira

Everything presented here is based on the testing-framework testcafe. Testcafe has introduced a “new world” of testing tools ~3 years ago, by not having the burden of using selenium / webdriver, but instead realizing test-automation by providing a nodejs based proxy-server, which is hosting your web-page to be tested. This architecture allows testcafe to know anything which is happening inside your page – no matter if it is a timeout, an HTTP-request or e.g. an animation, and wait for it accordingly. FPurthermore frameworks can specialize their selectors on their need, by injecting any kind of code into the “to-be-tested” webpage. While test-instabilities are not completely out of the world – they occur less often.

OK – enough talking, let’s get started and write out first end-to-end test-case. We will use the demokit application for it. We have to do three steps: Create a package.json, Create a test-file and run it.

1. Create a new folder to work on, and insert a new file “package.json” (to be used by nodejs – please see other blog posts, if you don’t know what nodejs is about)

{
    "name": "ui5-testcafe-samples",
    "description": "Most simplistic example",
    "dependencies": {
        "testcafe": "1.9.4",
        "typescript": "^4.0.3",
        "ui5-testcafe-selector-utils": "latest"
    }
}

2. Create a new file “first.ts” containing the following content

import { ui5Fixture, ui5Test, ui5 } from "ui5-testcafe-selector-utils";

ui5Fixture("Startseite", "https://sapui5.hana.ondemand.com/test-resources/sap/m/demokit/cart/webapp/index.html?sap-ui-theme=sap_fiori_3");

ui5Test('Product-Demo', async u => {
    await u.typeText(ui5().id("homeView--searchField"), "Flat Basic");
    await u.expect(ui5().id("homeView--productList")).tableLength().equal(1);
}); 

3. Run the following commands in the command-line:

npm i
testcafe "chrome --start-maximized" first.ts --selector-timeout 30000

Be amazed about the automated execution of your test in chrome (Please open the following gif into a new tab, to see it from beginning).

What is happening here?

• Using package.json we have maintained the dependency on both testcafe and ui5-testcafe-selector-utils
• The test is written in typescript, giving you full auto-complete possibilities. You don’t have to worry about wrong syntax (and much more) – just code. I highly recommend using VS-Code as the support for TS is great.
• The test is defining a fixture (think of it as category, which can bundle 0..n tests which are running on the same application) and one test-case.
• The test-case can perform 0..n actions and assertions (do something –> check afterwards if everything went as expected)
• The following line is instructing testcafe to type the text “Flat-Basic” into a UI5 Element with the ID “homeView–searchField”. Please note that testcafe itself is of course only aware of DOM elements. The ui5() API gives you all attributes, properties, aggregations (..) available inside UI5 to identify any item, and transforms it into something testcafe can understand (= DOM elements). In this case the ui5 API will search for any element having a SAPUI5 ID (not necessarily DOM-ID) ending with “homeView–searchField”.

await u.typeText(ui5().id("homeView--searchField"), "Flat Basic");

• The next line is now asserting / expecting that the List contains exactly one entry. Also note here that you are not hassling around with any DOM-Attributes or complex JSON structures – instead you are writing the clear instruction “tableLength”, which internally is accessing the finalLength attribute of the List-Binding.

await u.expect(ui5().id("homeView--productList")).tableLength().equal(1);

This is the most simplistic possible test-case. One of the biggest pain points in test development is the definition of the “selection-criteria” of an object. For this a colleague of mine and me have developed a supporting chrome extension. As the approval of version updates is currently pending, please install the extension manually. after cloning. A short overview of the tool was given at the last years ui5con.

Let’s assume you want to assert, that the currently searched element “Flat-Basic” has a value of 399,00€. The ID can’t be used here (as it is not stable inside a list binding). Instead we have to find a “smart-way” of identifying and asserting that item.

The test-recorder chrome plugin provides you will all the options “In a what you see is what you get” way. You are simply selecting the best properties which are available (here: the – not even visible – product-id of Flat-Basic, which is available in the context bound against the list-item) – afterwards you copy the code into the test and you are done. (Please open the following gif into a new tab, to see it from beginning).

Add the line to your test:

await u.expectProperty(ui5().element('sap.m.ObjectNumber').context('ProductId', 'HT-1035'), "number").equal("399,00")

After re-running your test, you will notice that this property was asserted on.

At the end of this blog your code should look like this:

import { ui5Fixture, ui5Test, ui5 } from "ui5-testcafe-selector-utils";

ui5Fixture("Startseite", "https://sapui5.hana.ondemand.com/test-resources/sap/m/demokit/cart/webapp/index.html?sap-ui-theme=sap_fiori_3");

ui5Test('Product-Demo', async u => {
    await u.typeText(ui5().id("homeView--searchField"), "Flat Basic").
        expect(ui5().id("homeView--productList")).tableLength().equal(1);

    await u.expectProperty(ui5().element('sap.m.ObjectNumber').context('ProductId', 'HT-1035'), "number").equal("399,00");

}); 

Next to end-to-end testcases, you can of course also run testcafe against your localhost server, provided by ui5-tooling.

ui5Fixture('Startseite', "SAP", "https://localhost:8443/index.html");

Using this blog post you should be able to perform very simplistic test-cases based on testcafe and UI5.