Technical Articles
E2E & Integration Testing for any UI5 application using testcafe
Having stable automated integration and e2e tests greatly increase the stability and overall quality of your application. Still if we look on today’s SAP UI5/FIORI applications automated testing is – at best – a rare sight.
In my opinion the key cause for this is the high complexity of the available testing tools and their “non-conformity” to UI5. If you ever tried to work out writing stable tests using a combination of 50 npm packages and try to integrate them into the UI5 world, you will probably rethink your decision to work with automated tests at all. If you fought through the first setup, you probably saw that your test is unstable (fails every 10th time, UI is changing, testdata, ..) and you can not hand over the code to any other developer (far to complex) ==> Projects give up and just “let the customer do the manual tests”.
For me there is a clear vision. There should be a UI5 specialized framework which:
- Is easy to learn and master for any junior developer
- Allows to write stable e2e and integration tests
- Is based on a modern and high-quality framework
- Is specialized for UI5, both in getting element-selectors and the overall code syntax
- Is easy to integrate in all common CI tools
- Has embedded handling of code-coverage – even in E2E scenario
To achieve these goals, I’ve started a new npm package “ui5-testcafe-selector-utils”. As the package is slowly getting stable (and it is MIT license / OSS based), I want to present you the current status. I am happy about any feedback / critics (or whatever reaction you have). As there is a lot of information this will be blog-series. In this first part we will only implement a very simple test, to give you an idea how the code looks like and to motivate you to play around with it on your own.
The blog has the following parts:
- Part 1: (This Blog) Overview and first test
- Part 2: Enhanced Selection, Action and Assertion Possibilities.
- Part 3: Structure your test and common best-practices
- Part 4: Code-Coverage for Integration and E2E tests
- Part 5: Integration in CI (Jenkins / Github Actions) / Authentification
- Part 6: Use testcafe for SAP Analytics Cloud and SAP Lumira
Everything presented here is based on the testing-framework testcafe. Testcafe has introduced a “new world” of testing tools ~3 years ago, by not having the burden of using selenium / webdriver, but instead realizing test-automation by providing a nodejs based proxy-server, which is hosting your web-page to be tested. This architecture allows testcafe to know anything which is happening inside your page – no matter if it is a timeout, an HTTP-request or e.g. an animation, and wait for it accordingly. FPurthermore frameworks can specialize their selectors on their need, by injecting any kind of code into the “to-be-tested” webpage. While test-instabilities are not completely out of the world – they occur less often.
OK – enough talking, let’s get started and write out first end-to-end test-case. We will use the demokit application for it. We have to do three steps: Create a package.json, Create a test-file and run it.
- Create a new folder to work on, and insert a new file “package.json” (to be used by nodejs – please see other blog posts, if you don’t know what nodejs is about)
{
"name": "ui5-testcafe-samples",
"description": "Most simplistic example",
"dependencies": {
"testcafe": "1.9.4",
"typescript": "^4.0.3",
"ui5-testcafe-selector-utils": "latest"
}
}
2. Create a new file “first.ts” containing the following content
import { ui5Fixture, ui5Test, ui5 } from "ui5-testcafe-selector-utils";
ui5Fixture("Startseite", "https://sapui5.hana.ondemand.com/test-resources/sap/m/demokit/cart/webapp/index.html?sap-ui-theme=sap_fiori_3");
ui5Test('Product-Demo', async u => {
await u.typeText(ui5().id("homeView--searchField"), "Flat Basic");
await u.expect(ui5().id("homeView--productList")).tableLength().equal(1);
});
3. Run the following commands in the command-line:
npm i
testcafe "chrome --start-maximized" first.ts --selector-timeout 30000
Be amazed about the automated execution of your test in chrome (Please open the following gif into a new tab, to see it from beginning).
What is happening here?
- Using package.json we have maintained the dependency on both testcafe and ui5-testcafe-selector-utils
- The test is written in typescript, giving you full auto-complete possibilities. You don’t have to worry about wrong syntax (and much more) – just code. I highly recommend using VS-Code as the support for TS is great.
- The test is defining a fixture (think of it as category, which can bundle 0..n tests which are running on the same application) and one test-case.
- The test-case can perform 0..n actions and assertions (do something –> check afterwards if everything went as expected)
- The following line is instructing testcafe to type the text “Flat-Basic” into a UI5 Element with the ID “homeView–searchField”. Please note that testcafe itself is of course only aware of DOM elements. The ui5() API gives you all attributes, properties, aggregations (..) available inside UI5 to identify any item, and transforms it into something testcafe can understand (= DOM elements). In this case the ui5 API will search for any element having a SAPUI5 ID (not necessarily DOM-ID) ending with “homeView–searchField”.
await u.typeText(ui5().id("homeView--searchField"), "Flat Basic");
- The next line is now asserting / expecting that the List contains exactly one entry. Also note here that you are not hassling around with any DOM-Attributes or complex JSON structures – instead you are writing the clear instruction “tableLength”, which internally is accessing the finalLength attribute of the List-Binding.
await u.expect(ui5().id("homeView--productList")).tableLength().equal(1);
This is the most simplistic possible test-case. One of the biggest pain points in test development is the definition of the “selection-criteria” of an object. For this a colleague of mine and me have developed a supporting chrome extension. As the approval of version updates is currently pending, please install the extension manually. after cloning. A short overview of the tool was given at the last years ui5con.
Let’s assume you want to assert, that the currently searched element “Flat-Basic” has a value of 399,00€. The ID can’t be used here (as it is not stable inside a list binding). Instead we have to find a “smart-way” of identifying and asserting that item.
The test-recorder chrome plugin provides you will all the options “In a what you see is what you get” way. You are simply selecting the best properties which are available (here: the – not even visible – product-id of Flat-Basic, which is available in the context bound against the list-item) – afterwards you copy the code into the test and you are done. (Please open the following gif into a new tab, to see it from beginning).
Add the line to your test:
await u.expectProperty(ui5().element('sap.m.ObjectNumber').context('ProductId', 'HT-1035'), "number").equal("399,00")
After re-running your test, you will notice that this property was asserted on.
At the end of this blog your code should look like this:
import { ui5Fixture, ui5Test, ui5 } from "ui5-testcafe-selector-utils";
ui5Fixture("Startseite", "https://sapui5.hana.ondemand.com/test-resources/sap/m/demokit/cart/webapp/index.html?sap-ui-theme=sap_fiori_3");
ui5Test('Product-Demo', async u => {
await u.typeText(ui5().id("homeView--searchField"), "Flat Basic").
expect(ui5().id("homeView--productList")).tableLength().equal(1);
await u.expectProperty(ui5().element('sap.m.ObjectNumber').context('ProductId', 'HT-1035'), "number").equal("399,00");
});
Next to end-to-end testcases, you can of course also run testcafe against your localhost server, provided by ui5-tooling.
ui5Fixture('Startseite', "SAP", "https://localhost:8443/index.html");
Using this blog post you should be able to perform very simplistic test-cases based on testcafe and UI5.
Great!!! Thanks for sharing, will try to explore more in the coming weeks.
Thanks for the feedback Paul. If you need any assistances / have any issues, let me know.
Hi Timo,
I really appreciate your work! Tried to follow your sample, but was struggling with this command: testcafe “chrome -start-maximized” first.ts -selector-timeout 30000
PS C:\git\ui5-testcafe-samples> testcafe “chrome -start-maximized” first.ts -selector-timeout 30000
ERROR Unable to find the browser. “chrome –start-maximized” is not a browser alias or path to an executable file.
Got it only to work, while using the whole path.
testcafe “chrome:C:\Program Files (x86)\Google\Chrome\Application\chrome.exe:headless -start-maximized” first.ts -selector-timeout 30000
Did I miss a step?
But thanks a lot, because until now I wasn’t aware that there exists a chrome version – without an interface – explicite for running tests.
Hi Dirk,
Thank you very much for your Feedback. Regarding your question
2. Regarding browser detection i really never had an issue. The official documentation of testcafe just says that the browsers are auto detected. Good thing: testcafe is open source.The browser detection code (platform independent) is here. For windows it is searching the registry at
For me at this place all my locally installed browsers are available.
Therefore I assume the installation process of your chrome instances was somehow corrupted. There are also other windows users who reported that problem and simply solved it by adding the relevant keys manually. As described in blog-post-3 you can also simply debug the browser detection process in
(line 63).
Best Regards and have fun testing,
Timo
Now it works. 🙂
Forgot to say, that not only the hint with the browser was very helpful, but also the hint with the quite popular project "testcafe" is also a great enrichment for me. Thank you very much!
BR, Dirk
Hi Timo,
I think I did something wrong, get this error msg:
Property ‘expectProperty’ does not exist on type ‘ui5ActionDefIntf’.
Hi Dirk,
Sorry my fault. Please update to ui5-testcafe-selector-utils version 0.0.217. Remarks regarding your code:
Your code would look like this:
I've also added this one to the blog itself.
Best regards,
Timo
Nice, now it works! Thanks for your detailed explanation! Will now play something with it. 🙂
Best Regards, Dirk
Hi Timo Stark,
thank you for those great and interesting blog posts. I've been following the great stuff you're doing around Testcafé since UI5Con 2019 and I've also used Testcafé and your selectors in some of my projects. Sadly, we had some issues in a more complex scenario with Testcafé, TypeScript, the UI5Selectors and Cucumber and decided to replace Testcafe with Webdriver.IO and wdi5.
It seems you did a restart of the project and I will definitely take a closer look at your work!
One question out of curiosity:
Did you have a look at UIVeri5 and wdi5. What are your thoughts on those E2E test frameworks?
Kind Regards,
Mike
Hi Mike,
Thanks a lot for the feedback. Yes it is a restart. As we are now starting those npm package productivly in real projects, it should be much more stable and has a nicer API. When you check please always ensure to have the latest version. E.g. I just added Caching support (respecting the HTTP Cache-Control header) – which makes testing much faster.
Regarding webdriver.io && wdi5 (all ist a very big “in my opion” ofc – if you would ask Volker rgd. wdio or SAP rgd. uiveri5 they would probably give very different answers
)
Especially for the stability reason, the overall trend of the industry is going to those proxy based test tools. Testcafe is actually the smaller one – the more popular one is Cypress. So for me the question would be more why to set on testcafe and not cypress – and the only real reason here is that FIORI is manipulating the testpage in a way cypress cant handle - as the cypress folk is appernantly now that big, that they are not responding to bug reports anymore I gave up 🙂
Regard ,
Timo
Thank you for this valuable feedback! I will definitely take a closer look into your libraries.
I also played around with Cypress and was really impressed… until I struggled to get it running in combination with Fiori Launchpad. Glad to know that it wasn’t me failing… 😉
Hello Timo Stark,,
I have created package.json and first.ts as you told above.
Then, I installed plugins (npm install).
There is the error in this line of the code: