E2E Testing Part 2: Enhanced Selection, Actions and Assertions

This is the second part of the blog-series about automated testing (end to end and integration-tests) using testcafe. The blog-series has the following parts:

1. Part 1: Overview and first test
2. Part 2: (This Blog) Enhanced Selection, Action and Assertion Possibilities.
3. Part 3: Structure your test and common best-practices
4. Part 4: Code-Coverage for Integration and E2E tests
5. Part 5: Integration in CI (Jenkins / Github Actions) / Authentification
6. Part 6: Use testcafe for SAP Analytics Cloud and SAP Lumira

Inside this second part we will discuss ways to declare stable selectors and assertions, and how to handle and debug issues.

Selectors

What is a selector-overall? When working with integration and end-to-end tests there is always one big task: You have to interact with page elements. The big question is: How can you identify this page element? The framework must be informed about element-attributes with which it can identify the element in a stable way. The description of an element is named Selector. Unstable selectors are one of the root causes for breaking tests. Examples:

• Language / Time / Description dependent selectors (test fails in case of different descriptions / languages / ..)
• Selectors accessing generate / cloned framework IDs (e.g. button13), which can differ in every run
• Unspecific selectors which return more than one DOM element

Under normal cirumstances you should always select an Element by it’s unique identifier. Please respect that during test development and provide IDs inside your productive code, before making fancy and complex selectors. There are however a lot of situations where selecting elements via IDs is simply not possible. Examples are Aggregations (e.g. Tables, Lists, Combo-Boxes, [..]) or app coding which is not under your control (e.g. Fiori Elements). The definition of selectors can be utterly painful and difficult in such cases. Using the UI5 enhancements for testcafe you are getting a lot of support in writing these selectors. Generally you can use almost any piece of information available inside your UI5 control to identify it in a clear, unique and stable way.

In the following examples we will again use the SAP UI5 Shopping-Cart demo as example. I will simply show you a lot of code examples, which should give a better overview than an abstract reference.

Select by Id

We want to select the “Search-Input-Field” inside the Demo:

To access the element you can use the ui5() api, and afterwards select by id. This will make a pattern search over the “getId()” method.

let searchField= ui5().id('homeView--searchField');
await u.typeText(searchField, "Test");

Select inside List via Context

We want to select the first category:

Using other frameworks you would normally somehow go over the text (“Accessoires”). This is of course also possible using the ui5 enhancements for testcafe, as seen in the following example. Here we are using the ui5() API, and selecting all list-Item Elements (Control Type sap.m.ListItemBase and deriving classes) and property title with value “Accessoirs”. Please note that you can choose any property which is assigned to the element you want to click on.

const accListItem = ui5().listItem().property('title', 'Accessories');
await u.click(accListItem);

In my experience this is however not really perfect, as the title text is – most-probably – retrieved from the backend. Therefore the title might change, or might be translatable. Ideally we are therefore defining the selector by key. The key value is unfortunately not visible on frontend. As mentioned using the testcafe enhancements for ui5 you can select based on all attributes bound to an UI5 element – including of course the invisible ones. This includes the Binding Context, Binding Paths, Tabular Information and more.

In our scenario we will take the object behind the assigned binding context. Please note that the framework is automatically detecting (for lists / tables / ..) the binding context to be used. In the following example we are accessing the backend Category-Key instead of the text.

const accListItem = ui5().listItem().context('Category', 'AC');
await u.click(accListItem);

Alternatively it would also be possible to select based on the binding path (e.g. of the title). As in this scenario this is more fragile (change of Binding Entity-Name, change of attribute name, ..) as the context I would prefer the context object.

const accListItem = ui5().listItem().bindingPath('title', '/ProductCategories(\'AC\')/CategoryName');
await u.click(accListItem);

 

In the next example we want to select the first item which comes up. As we don’t want to depend on backend data we are actually not interested in the product-id at all.

 

To enable such a scenario the ui5 extensions allow to select via row or table column (for sap.m.Table / sap.ui.table.Table based controls). This is working for all aggregations (also e.g. for Combobox or Select entries).

const firstItem = ui5().listItem().row(0);
await u.click(firstItem);

Select via complex properties

In the next step we want to add this first item to our shopping-cart.

 

Unfortunately the button does not have a static ID (shame on you SAP standard app). Normally the solution would be a id based access. In this scenario we must find a different way. An example would be to select via the “Add to Cart” text.

const button = ui5().button().property('text', 'Add to Cart');
await u.click(button);

Again this is of course not very static. What happens if you login in German? The static element here is of course the i18n text id. As described above you can also select elements based on binding paths – this is utilized in the following example:

const button = ui5().button().bindingPath('text', 'i18n>addToCartShort');
await u.click(button);

This selector is selecting all elements of type Button, with a text i18n binding against “addToCartShort”, which should be pretty stable. In case of complex selection criteria it is a very good idea to at least add the ID of a parent-element, to make the selector more specific. The following example will restrict the current selector to items on the product page.

const button = ui5().button().bindingPath('text', 'i18n>addToCartShort').parentId('product--page');
await u.click(button);

Find Selectors

Let’s be honest: It is nice that you are able to define selectors in such a complex way – but it is horrible to find those elements. To support you in the selector definition there is a public open source chrome / firefox plugin . Please install the plugin via the official chrome webstore. If you now access the demo application you can open the plugin. The plugin will guide you through the selector process and output the selector code you can simply copy & paste inside your test-code. The plugin should be pretty self-explanatory. Still I will go into more details in one of the next blog-posts.

By the way: even if you decide to use uiveri5 or OPA5 – the plugin has support for it 🙂

 

Assertions

Again the question: What is a assertion? When reviewing tests I often notice that the test-developer is only writing an “action-based-test” (click here, type here, do that, …). What is missing is the actual test (the validation phase). An example of an assertion is that after a click on “add to cart”, you want to validate that the shopping-cart counter increases by one.

Of course also assertions are normally based on selectors. If you want to make an assertion based on an UI element, you have to identify the element, and afterwards describe what you want to validate.

Let’s have a few examples again.

The first most trivial example: After selecting the item, we want to ensure that the title in the Detail View is 10” Portable DVD Player.

This can now be done on two different ways:

1. Define a very specific selector which contains the text inside the selector definition, and assert that this selector actually exists
2. Define a less specific selector, which is accessing the title without text property, and is asserting the property later on.

Following are both examples. I personally like the second way more as you are actually asserting on what you want to check (console output will be “expect on 10′ Portable DVD Player, Actual: 10′ Portable DVD Player).

//use specific selector
const objectHeader = ui5().element('sap.m.ObjectHeader').property('title', '10" Portable DVD player');
await u.expectExists(objectHeader).ok();

//use unspecific selector and assert via property
const objectHeader = ui5().element('sap.m.ObjectHeader').parentId('product');
await u.expect(objectHeader).property('title').equal('10" Portable DVD player');

 

 

In the test before, we now want to validate that the element with Text 10” Portable DVD Player is actually available. Again we are using the test-recorder Chrome Plugin for simplicity. The result after clicking the assertion is the following code.

const objectListItem = ui5().listItem(). property('title', '10" Portable DVD player');
await u.expect(objectListItem).context("Status").equal('A');

Retrieve Element Information

Under certain circumstances, you want write very complex assertions / selectors, using program logic, which can not be expressed in a simple “pattern-based” assertion. To do this, you can retrieve all element-data available “behind” a control using the .data method of a selector.

Example: In the category list we want to select the second category which has a count higher than 10.

const data = await ui5().id('homeView--categoryList').data(); //get all attributes assigned to the table
let iHitCount = 0;
let iFoundRow = 0;
//loop over table data
for (var i = 0; i < data.tableData.data.length; i++) {
    const element = data.tableData.data[i];
    if (element.NumberOfProducts > 10) { //Number of products > 10?
        iHitCount++;
        if (iHitCount == 2) { //2nd item with number of products > 10?
            iFoundRow = i;
            break;
        }
    }
}

//select via row..
const definedItem = ui5().listItem().row(iFoundRow).parentId("homeView--categoryList");
await u.click(definedItem);

Please note that beside the table data there is much more information available you can work and assert on (using the already described APIs). The following screenshot is showing a minimal excerpt of the data available.

 

Debug & Trace

A lot of opportunities are causing a lot of problems. Even using the testrecorder plugin you will make tons of errors during the definition of selectors. The plugin is offering multiple ways to support you in this case.

Most important: in case of issues always work with the debug mode of VS-Code (in console window, press the “Create JavaScript Debug Terminal”, and execute the testcafe start instruction here). This allows you to fully debug the typescript code – especially in case you have more than just selectors and actions, this can really help a lot.

In case you are having issues with your selectors, you can use the u.traceSelector() API. As example, let’s take the first example of this blog (selecting category “AC”) and adjust it to select an non-existing category:

const accListItem = ui5().listItem().context('Category', 'AC1');
await u.click(accListItem);

This will of course fail with an error (selector not available). In the following blog-post I will describe how to enhance this error message – for the moment we are just seeing that the selector can’t be found –> very difficult to continue here.

Using the traceSelector API:

const accListItem = ui5().listItem().context('Category', 'AC1');
await u.traceSelector(accListItem, { timeout: 15000 });

Now the output is changing and showing you all UI5 elements scanned on the webpage, and the reason why the element wasn’t taken into account.

As this list is very big, let’s remove all items which are not at least having the correct control-type (list-Item).

const accListItem = ui5().listItem().context('Category', 'AC1');
await u.traceSelector(accListItem, { timeout: 15000, hideTypes: [ui5TraceMismatchType.CONTROL_TYPE] });

The output get’s very usable now, and as developer you will clearly notice that your chosen Categoy was apparently incorrect.

 

Using this blog post you should now be able to design and create more complex test-cases, selectors and assertions using testcafe and UI5. In the next blog-post I will write more about test-organization and configuration. Until than stay tuned. I am happy for any feedback.

Custom Selectors / Assertions

There is always a situation where the predefined API is simply not sufficient and you need to integrate custom code. In case you need a completely own selector please have a look on the official testcafe documentation. There are however of course situations where you need 2 selectors by the ui5 enhancements for testcafe, and just want to have an additional logic. To enable this the ui5 selector builder has the two methods “fnData” and “fnSelect“, which allows you to include callbacks into the selector logic. The following code snippet is giving an example:

    const ui5Sel = ui5().fnSelect(function (ui5Element, selDef, getElemInfo): boolean {
        if (ui5Element.getId().indexOf("homeView--searchField") !== -1) {
            return true;
        }
        return false;
    }).fnData((ui5Element, retData) => ui5Element.getId());

    await u.typeText(ui5Sel, "test");
    const data = await ui5Sel.data();

• fnSelect will be called for every single UI5 element on the page, and can return true (yes, this element is relevant) or false (no, this one is not relevant). It has 3 parameters:
  • ui5Element: reference to the ui5-element control
  • selDef: reference to the selector definition
  • getElemInfo: function which can be called to get all information available in the data() API method.
• fnData will be called once if you are using the data method of your selector. The function should return the data you want to extract from the ui5-control. The data will be stored inside “customData” attribute. The function has two parameters.
  • ui5Element: reference to the ui5-element control
  • retData: currently identified element data

Using the customData API function this can get even more complex, by giving you the ability to parameterize your custom selector callback. This way you can really define your own selector. The following code gives you an example, where you write a custom selector for css classes.

const cssSel = () => ui5().fnSelect(function (ui5Element, selDef) {
    return ui5Element.hasStyleClass && ui5Element.hasStyleClass(selDef.customData);
});

await u.typeText(cssSel().id("homeView--searchField").customData("sapMTBShrinkItem"), "test");

• cssSel is checking if the ui5Element passed has the property hasStyleClass and if the css style class provided in customData is available in the ui5Element
• Inside the selector definition (using cssSel(), instead of ui5()) the customData parameter is used to pass the css value.