Skip to Content
Technical Articles
Author's profile photo Timo Stark

E2E & Integration Testing for any UI5 application using testcafe

Having stable automated integration and e2e tests greatly increase the stability and overall quality of your application. Still if we look on today’s SAP UI5/FIORI applications automated testing is – at best – a rare sight.

In my opinion the key cause for this is the high complexity of the available testing tools and their “non-conformity” to UI5. If you ever tried to work out writing stable tests using a combination of 50 npm packages and try to integrate them into the UI5 world, you will probably rethink your decision to work with automated tests at all. If you fought through the first setup, you probably saw that your test is unstable (fails every 10th time, UI is changing, testdata, ..) and you can not hand over the code to any other developer (far to complex) ==> Projects give up and just “let the customer do the manual tests”.

For me there is a clear vision. There should be a UI5 specialized framework which:

  1. Is easy to learn and master for any junior developer
  2. Allows to write stable e2e and integration tests
  3. Is based on a modern and high-quality framework
  4. Is specialized for UI5, both in getting element-selectors and the overall code syntax
  5. Is easy to integrate in all common CI tools
  6. Has embedded handling of code-coverage – even in E2E scenario

To achieve these goals, I’ve started a new npm package “ui5-testcafe-selector-utils”. As the package is slowly getting stable (and it is MIT license / OSS based), I want to present you the current status. I am happy about any feedback / critics (or whatever reaction you have). As there is a lot of information this will be blog-series. In this first part we will only implement a very simple test, to give you an idea how the code looks like and to motivate you to play around with it on your own.

The blog has the following parts:

  1. Part 1: (This Blog) Overview and first test
  2. Part 2: Enhanced Selection, Action and Assertion Possibilities.
  3. Part 3: Structure your test and common best-practices
  4. Part 4: Code-Coverage for Integration and E2E tests
  5. Part 5: Integration in CI (Jenkins / Github Actions) / Authentification
  6. Part 6: Use testcafe for SAP Analytics Cloud and SAP Lumira

Everything presented here is based on the testing-framework testcafe. Testcafe has introduced a “new world” of testing tools ~3 years ago, by not having the burden of using selenium / webdriver, but instead realizing test-automation by providing a nodejs based proxy-server, which is hosting your web-page to be tested. This architecture allows testcafe to know anything which is happening inside your page – no matter if it is a timeout, an HTTP-request or e.g. an animation, and wait for it accordingly. FPurthermore frameworks can specialize their selectors on their need, by injecting any kind of code into the “to-be-tested” webpage. While test-instabilities are not completely out of the world – they occur less often.

 

OK – enough talking, let’s get started and write out first end-to-end test-case. We will use the demokit application for it. We have to do three steps: Create a package.json, Create a test-file and run it.

  1. Create a new folder to work on, and insert a new file “package.json” (to be used by nodejs – please see other blog posts, if you don’t know what nodejs is about)
{
    "name": "ui5-testcafe-samples",
    "description": "Most simplistic example",
    "dependencies": {
        "testcafe": "1.9.4",
        "typescript": "^4.0.3",
        "ui5-testcafe-selector-utils": "latest"
    }
}

2. Create a new file “first.ts” containing the following content

import { ui5Fixture, ui5Test, ui5 } from "ui5-testcafe-selector-utils";

ui5Fixture("Startseite", "https://sapui5.hana.ondemand.com/test-resources/sap/m/demokit/cart/webapp/index.html?sap-ui-theme=sap_fiori_3");

ui5Test('Product-Demo', async u => {
    await u.typeText(ui5().id("homeView--searchField"), "Flat Basic");
    await u.expect(ui5().id("homeView--productList")).tableLength().equal(1);
}); 

 

3. Run the following commands in the command-line:

npm i
testcafe "chrome --start-maximized" first.ts --selector-timeout 30000

Be amazed about the automated execution of your test in chrome (Please open the following gif into a new tab, to see it from beginning).

What is happening here?

  • Using package.json we have maintained the dependency on both testcafe and ui5-testcafe-selector-utils
  • The test is written in typescript, giving you full auto-complete possibilities. You don’t have to worry about wrong syntax (and much more) – just code. I highly recommend using VS-Code as the support for TS is great.
  • The test is defining a fixture (think of it as category, which can bundle 0..n tests which are running on the same application) and one test-case.
  • The test-case can perform 0..n actions and assertions (do something –> check afterwards if everything went as expected)
  • The following line is instructing testcafe to type the text “Flat-Basic” into a UI5 Element with the ID “homeView–searchField”. Please note that testcafe itself is of course only aware of DOM elements. The ui5() API gives you all attributes, properties, aggregations (..) available inside UI5 to identify any item, and transforms it into something testcafe can understand (= DOM elements). In this case the ui5 API will search for any element having a SAPUI5 ID (not necessarily DOM-ID) ending with “homeView–searchField”.
await u.typeText(ui5().id("homeView--searchField"), "Flat Basic");
  • The next line is now asserting / expecting that the List contains exactly one entry. Also note here that you are not hassling around with any DOM-Attributes or complex JSON structures – instead you are writing the clear instruction “tableLength”, which internally is accessing the finalLength attribute of the List-Binding.
await u.expect(ui5().id("homeView--productList")).tableLength().equal(1);

 

This is the most simplistic possible test-case. One of the biggest pain points in test development is the definition of the “selection-criteria” of an object. For this a colleague of mine and me have developed a supporting chrome extension. As the approval of version updates is currently pending, please install the extension manually. after cloning. A short overview of the tool was given at the last years ui5con.

Let’s assume you want to assert, that the currently searched element “Flat-Basic” has a value of 399,00€. The ID can’t be used here (as it is not stable inside a list binding). Instead we have to find a “smart-way” of identifying and asserting that item.

The test-recorder chrome plugin provides you will all the options “In a what you see is what you get” way. You are simply selecting the best properties which are available (here: the – not even visible – product-id of Flat-Basic, which is available in the context bound against the list-item) – afterwards you copy the code into the test and you are done. (Please open the following gif into a new tab, to see it from beginning).

 

Add the line to your test:

await u.expectProperty(ui5().element('sap.m.ObjectNumber').context('ProductId', 'HT-1035'), "number").equal("399,00")

After re-running your test, you will notice that this property was asserted on.

 

At the end of this blog your code should look like this:

import { ui5Fixture, ui5Test, ui5 } from "ui5-testcafe-selector-utils";

ui5Fixture("Startseite", "https://sapui5.hana.ondemand.com/test-resources/sap/m/demokit/cart/webapp/index.html?sap-ui-theme=sap_fiori_3");

ui5Test('Product-Demo', async u => {
    await u.typeText(ui5().id("homeView--searchField"), "Flat Basic").
        expect(ui5().id("homeView--productList")).tableLength().equal(1);

    await u.expectProperty(ui5().element('sap.m.ObjectNumber').context('ProductId', 'HT-1035'), "number").equal("399,00");

}); 

 

Next to end-to-end testcases, you can of course also run testcafe against your localhost server, provided by ui5-tooling.

ui5Fixture('Startseite', "SAP", "https://localhost:8443/index.html");

 

Using this blog post you should be able to perform very simplistic test-cases based on testcafe and UI5.

 

Assigned tags

      11 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Somnath Paul
      Somnath Paul

      Great!!! Thanks for sharing, will try to explore more in the coming weeks.

      Author's profile photo Timo Stark
      Timo Stark
      Blog Post Author

      Thanks for the feedback Paul. If you need any assistances / have any issues, let me know.

      Author's profile photo Dirk Raschke
      Dirk Raschke

      Hi Timo,

      I really appreciate your work! Tried to follow your sample, but was struggling with this command: testcafe “chrome -start-maximized” first.ts -selector-timeout 30000

      PS C:\git\ui5-testcafe-samples> testcafe “chrome -start-maximized” first.ts -selector-timeout 30000
      ERROR Unable to find the browser. “chrome –start-maximized” is not a browser alias or path to an executable file.

      Got it only to work, while using the whole path.

      testcafe “chrome:C:\Program Files (x86)\Google\Chrome\Application\chrome.exe:headless -start-maximized” first.ts -selector-timeout 30000

      Did I miss a step?

      But thanks a lot, because until now I wasn’t aware that there exists a chrome version – without an interface – explicite for running tests.

      Author's profile photo Timo Stark
      Timo Stark
      Blog Post Author

      Hi Dirk,

      Thank you very much for your Feedback. Regarding your question

      1. I’ve noticed that the SAP Blog “corrupted” the “–“. I’ve adjusted the blog, so now the start command is correct.
      testcafe "chrome --start-maximized" first.ts --selector-timeout 30000

      2. Regarding browser detection i really never had an issue. The official documentation of testcafe just says that the browsers are auto detected. Good thing: testcafe is open source.The browser detection code (platform independent) is here. For windows it is searching the registry at

      root => `${root}\Software\Clients\StartMenuInternet\*\shell\open\command` both in HKEY_LOCAL_MACHINE and HKEY_CURRENT_USER.

       

      For me at this place all my locally installed browsers are available.

       

      Therefore I assume the installation process of your chrome instances was somehow corrupted. There are also other windows users who reported that problem and simply solved it by adding the relevant keys manually. As described in blog-post-3 you can also simply debug the browser detection process in

      \node_modules\testcafe-browser-tools\lib\api\get-installations.js

      (line 63).

      Best Regards and have fun testing,

      Timo

      Author's profile photo Dirk Raschke
      Dirk Raschke

      Now it works. 🙂

      testcafe "chrome --start-maximized" first.ts --selector-timeout 30000

      Forgot to say, that not only the hint with the browser was very helpful, but also the hint with the quite popular project "testcafe" is also a great enrichment for me. Thank you very much!

      BR, Dirk

       

      Author's profile photo Dirk Raschke
      Dirk Raschke

      Hi Timo,

      I think I did something wrong, get this error msg:

      Property ‘expectProperty’ does not exist on type ‘ui5ActionDefIntf’.

      ui5Test('Product-Demo', async u => {
          await u.typeText(ui5().id("homeView--searchField"), "Flat Basic");
          await u.expect(ui5().id("homeView--productList")).tableLength().equal(1);
       
          const objectNumber = ui5().element('sap.m.ObjectNumber').context('ProductId', 'HT-1035');    
          await ui5Action.click(objectNumber);
          await u.expectProperty(ui5().element('sap.m.ObjectNumber').context('ProductId', 'HT-1035'), "number").equal("399,00")
      }); 
      Author's profile photo Timo Stark
      Timo Stark
      Blog Post Author

      Hi Dirk,

      Sorry my fault. Please update to ui5-testcafe-selector-utils version 0.0.217. Remarks regarding your code:

      1. While you can use ui5Action.click, it is always preferable to use "u.click". Reason: Testcafe can run multiple tests in paralell (concurrent run mode). "u" is a unique test-instance of the ui5 Actions (and the underlying testacfe framework) for this single test. ui5Action is actually a proxy object, which tries to identify the current test, based on the callstack. While this is normally working, it is simply slower.
      2. If you are defining objectNumber, you can also simply reuse this selector inside the expect.
      3. You can "chain" multiple actions, and at (maximum) one assertion. This (in my opinion) looks cleaner. But that's just a matter of coding style ofc..
      4. Your order is incorrect. After you click the objectNumber (at least for me), the object number is disappearing. You can not assert onto an non existing object.

      Your code would look like this:

      ui5Test('Product-Demo', async u => {
          await u.typeText(ui5().id("homeView--searchField"), "Flat Basic").
              expect(ui5().id("homeView--productList")).tableLength().equal(1);
      
          const objectNumber = ui5().element('sap.m.ObjectNumber').context('ProductId', 'HT-1035');
          await u.expectProperty(objectNumber, "number").equal("399,00");
          await u.click(objectNumber);
      }); 

      I've also added this one to the blog itself.

       

      Best regards,

      Timo

      Author's profile photo Dirk Raschke
      Dirk Raschke

      Nice, now it works! Thanks for your detailed explanation! Will now play something with it. 🙂

      Best Regards, Dirk

      Author's profile photo Mike Zaschka
      Mike Zaschka

      Hi Timo Stark,

      thank you for those great and interesting blog posts. I've been following the great stuff you're doing around Testcafé since UI5Con 2019 and I've also used Testcafé and your selectors in some of my projects. Sadly, we had some issues in a more complex scenario with Testcafé, TypeScript, the UI5Selectors and Cucumber and decided to replace Testcafe with Webdriver.IO and wdi5.

      It seems you did a restart of the project and I will definitely take a closer look at your work!

      One question out of curiosity:
      Did you have a look at UIVeri5 and wdi5. What are your thoughts on those E2E test frameworks?

      Kind Regards,

      Mike

      Author's profile photo Timo Stark
      Timo Stark
      Blog Post Author

      Hi Mike,

      Thanks a lot for the feedback. Yes it is a restart. As we are now starting those npm package productivly in real projects, it should be much more stable and has a nicer API. When you check please always ensure to have the latest version. E.g. I just added Caching support (respecting the HTTP Cache-Control header) – which makes testing much faster.

      Regarding webdriver.io && wdi5 (all ist a very big “in my opion” ofc – if you would ask Volker rgd. wdio or SAP rgd. uiveri5 they would probably give very different answers 🙂 )

      • What I love when using testcafe is the extremly tight integration of test-code and the page under test – made possible using the proxy based architecture. This makes a headstart extremley easy. Using webdriver / selenium based solutions there is always a big hassel (yes, yes it got much much better – still..).
      • Testcafe has native support for typescript. if you ever ever developed in typescript – you never want to go back.
      • Due to this architecture testcafe is very very stable. Check e.g. the description of testcafe themselfes. I have a test-set with 20 tests running now since 1 month every 30 minutes – and *none* of those tests failed (>1000 test executions) – and all of this without one single manual timeouts or hack. You can of course also realize that with selenium based architecture – but I had longer talks with colleagues using Nightwatch and webdriver.io – and this stability is really far from common. Normally every 20th test is giving a false-positive due to whatever instability is there.
      • Both uiveri5 and wdio are using the sap.ui.test library, which is (imo) to limited. There is no possibility to use IFrames (required for e.g. Lumira), “non-ui5 objects” (e.g. if you mix in SAPUI5 Web Components),  for older UI5 versions or for context access (e.g. you want to check the model). Additionally the library is not enhancible (as far as I know at least) – therefore you have to live with it as it is.
      • Testcafe is very popular (while honestly the commit activity is pretty low atm). There is an actual company behind that tool – which has testcafe as its main product. For me this gives me more trust than I will get help (either by the community or by the developers) compared to the some open source projects (even if it is SAP..). If I really have to, I can even pay DevExpress (company behind testcafe) to provide me support.
      • As testcafe is not at all UI5 specific, you can use that framework also for Angular, React, UI5 Web Components, […] – which is a big help if you want to establish testing cultur at your customer, and you don’t always have to say “we have to use another framework, because it is not the launchpad”. For testcafe (ofc also wdio) you can just use the same test and mix angular, react and ui5 apps.
      • Testcafe has a super open and enhanceable API. Using e.g. ClientFunctions you can emit any code very easily into the web page. That makes fancy stuff possible – like the custom-selectors here – but also much much more use cases. Example would be e.g. that you want to trigger a REST call inside the webpage under test.
      • Testcafe has a super easy way to mock requests (which I’ve used in the latest version of the plugin to enable caching)
      • Testcafe is very very fast and allows concurrent execution (if your tests are isolated). On our CI server I can run ~~4 tests in paralell. This way I can executed ~~10 tests in less than 2 minutes – which is super fast. When using better servers (e.g. the ones from GitHub Actions are crazy fast) you can even get better.
      • Last I really just love the simplified API of testcafe (and the addon I’ve developed 🙂 ).

      Especially for the stability reason, the overall trend of the industry is going to those proxy based test tools. Testcafe is actually the smaller one – the more popular one is Cypress. So for me the question would be more why to set on testcafe and not cypress – and the only real reason here is that FIORI is manipulating the testpage in a way cypress cant handle - as the cypress folk is appernantly now that big, that they are not responding to bug reports anymore I gave up 🙂

      Regard ,

      Timo

      Author's profile photo Mike Zaschka
      Mike Zaschka

      Thank you for this valuable feedback! I will definitely take a closer look into your libraries.

      I also played around with Cypress and was really impressed… until I struggled to get it running in combination with Fiori Launchpad. Glad to know that it wasn’t me failing… 😉