Product Information
UIVeri5 for E2E testing of UI5 apps
Some of you might’ve already explored the youngest member of our UI5 family on GitHub: UIVeri5. So, it’s high time to introduce this new framework to you!
Motivation
There are many testing frameworks out there and some of them are targeting specific UI technology or a specific layer of the testing pyramid. For UI5, we suggest and heavily use internally QUnit as a unit testing framework. For app tests, we offer OPA5. But we were missing an end-to-end (E2E) framework. This has traditionally been in the domain of manual UI testing. Of course, there’s the highly popular Selenium and many other great browser-automation frameworks, such as Puppeteer and Cypress, or OS-level automation ones, like Sikuli. Unfortunately, using them with modern, JS-centric UI framework like UI5 can be harder than it should.
The major real-world belief is that automated UI testing is considered unreliable and it is avoided in favour of manual tests. However, manual tests cannot be used as regression testing. The ability to automate and reliably execute a core set of E2E scenarios could make a significant difference to the quality of the product.
The idea of a Selenium-based framework that makes UI testing reliable was the dream based on which UIVeri5 was born: the E2E test framework for OpenUI5 and SAPUI5 apps!
Originally developed for our internal UI5 testing activities, the framework quickly found new customers within SAP. The feedback we got from them confirmed that UIVeri5 delivers real benefits and motivated us to make it open source and available to the whole UI5 community.
Key Benefits
I can come up with a long list of capabilities and benefits but let me share the major points that differentiate UIVeri5 from the classical home-grown, Selenium-based framework.
Synchronization with UI5 Rendering
UIVeri5 synchronizes the test execution with the UI5 rendering in the browser. This avoids the need for manual coding waits and sleeps in the test. It leads to tests that are reliable by design, without requiring deep expertise to create them.
Tests in JavaScript
Tests in UIVeri5 are written in JavaScript, which nicely matches the UI5 app. Having the tests in the same language as the app makes is easy for the app developer to comprehend and convenient to maintain the tests, together with the main code base. Even further, this removes the technical obstacles and opens the door to agile development practices, like Test-Driven Development.
Synchronous Tests
UIVeri5 uses WebDriverJS as Selenium binding and builds on the flow handling that it provides. As a result, the test looks synchronous and does not need explicit callbacks or promise chaining. This becomes a huge benefit for the automation experts with Java background.
Control Selectors
A major issue in every UI test is the identification of the elements. Typically, this is achieved with ID, CSS or XPATH selectors that all work on the DOM level. This approach works well with typical web pages or UI frameworks that augment the DOM, but it has its limitations when working with frameworks like UI5 that encapsulate the DOM handling in JavaScript. To overcome such problems, we provide control locators that allow identifying the control based on its properties or relations.
Ecosystem
UIVeri5 is inspired by Protractor, the E2E testing framework for Angular apps. Protractor is a very popular testing tool that a lot of developers are familiar with. So, it’s good to hear that for most parts, the Protractor knowledge is directly applicable to UIVeri5.
UIVeri5 is also Node.js-based that makes it easily combined with our UI5 build and development tooling.
Summary
We are excited to finally release UIVeri5 under the Apache 2.0 license, just like OpenUI5, so that everyone in the UI5 community can utilize our terrific testing tool. And, as UIVeri5 is hosted on GitHub, it’s also a piece of cake to fork and extend it to support specific use cases.
We would like to hear your feedback! Leave a comment under this post or ask a question on SAP Q&A. Found an issue? You are welcome to report it on GitHub.
Hi,
Could you please provide an example of Gruntfile.js which shows how to use ui5veri5 within WebIDE build functions?
Thanks!
BR,
Ivan
Hi Ivan,
actually, I am not sure this is possible right now. Up to my knowledge, WebIDE does not yet support generic nodejs executions and so you couldn't add an arbitrary grunt task. I have only seen grunt builds in WebIDE that run a very limited, preconfigured set of grunt tasks.
Otherwise, it would be very easy to hook uiveri5 with grunt - just spawn a task with grunt-shell for example.
Regards,
Maxim
Hi Maxim,
Thanks for the quick response!
Unfortunately its not working with grunt-shell too. I receive the following exception:
It would be nice if ui5veri5 works in WebIDE too
Thank you!
Greetings,
Ivan
Hi Ivan,
this particular problem you see is due to missing java in the WebIDE runner pods/dockers. Actually java was needed for the selenium.jar that we were by default. Starting with uiveri5 1.36.1, selenium.jar is not used by default. For older version, it could be disabled with --useSelniumJar=false.
Regards,
Maxim
Hello Maxim,
Is ui5veri5 now supported in webIDE? Are there any updates?
MILIND.
Hi Milind,
UIVeri5 is supported in WebIDE as much as you can use it to create, edit and push changes to your E2E scripts. But there are no wizards to help you with setting up basic configurations and there is no runner support (like "right click" => "run" ). We plan to address those limitations in the SAP Business ApplicationStudio that is the next generation of WebIDE
Maxim
I wonder why there is not more attention on Cypress. It is so easy to set up and has a very flat learning curve and beautiful UI traceability of each of the steps of your test. I did some tests with public web sites like google.com, and they were a breeze to set up. However, as soon as you want to e.g. log into an S/4HANA system, you hit against a wall. It seems that our code (abap.js) manipulates the prototype of the XHR object in JavaScript. So does Cypress and – boom – things go wrong.
I would be willing to team up with people trying to fix this. Is anyone interested?
Hi Patrik,
I personally like Cypress as they implemented some of the synchronization ideas we use in UIVeri5. But IMHO having a simple test of google.com and having a reliable test of UI5 app are quite different stories. At the same time, in UI5 we have APIs that could help you get most of the advantages of UIVeri5 in any testing framework – WDIO,Cypress, etc. I plan to write a separate blog post on this.
How to setup test data for uiveri5 tests for a SAPUI5 application ? Are there any tearup and teardown methods available ?
Hi Rajesh,
UIVeri5 is a End-to-End testing tool so the test runs against a real system with real data in it. So you should think about how to add test content to your system. What we have seen is that most people use the test data they already have for their manual QA activities. If your system have a REST API and robust data import/export capabilities, you can consider the API testing feature in UIVeri5. Another option is to combine data import/export steps in the CI pipeline.
hi Maxim Naidenov ,
how to run in Internet explorer webdriver whenever i run it is taking up chrome and also also is there any further documents/blogs regarding this.
thank you in advance.
Hi Manohar,
Running IE is definitely possible but quite challenging. We have documented the settings in the browsers.md, please check and configure the system very carefully. Much easier is to use a browser cloud provider like SauceLabs, BrowserStack, etc.
Regards,
Maxim
Hi Maxim,
Great info...I'm curious would CBTA test tool not be able to achieve the above for SAP Fiori? We have had our share of challenges with Fiori scripts automation using CBTA, but that was 2 years back (we haven't tried automating Fiori scripts since). If you shed some light on the diff between the two (CBTA vs UIVeri5), that would be great. Thank you.
Thanks,
Sadhna
Hi Sadha,
Never heard of CBTA before. Just checked and seems it is part of SAP Solution manager but this is all I know so can't really compare.
Regards,
Maxim
Hi Maxim,
Started exploring UiVeri5 framework. I have few questions :
Hi Saurabh,
Thanks Maxim for the prompt response to the query.
We can write code in BAS like you said as its a Nodejs with VS editor. Then how should we perform the test execution ? We need E2E test cases as part of our testing framework. Any workaround you have for same ?
Typically you need to execute the tests many times while you develop them and as mentioned, currently this can't be done from BAS. Once you have the E2E tests, you would like to have them executed as part of your DevOps process and here you can use the Piper library: https://www.project-piper.io/steps/uiVeri5ExecuteTests/
Hi Maxim, Thanks again for the reply.. This becomes a challenging situation for team working on BAS model. we can only edit/push scripts but can't debug/execute same. Devops is a part which gets implemented when we are sure of running things which are working fine locally(E2E test cases).. Running in pipeline on a daily or hourly basis is not feasible as there are multiple times we write code and need to debug,also it works in headless mode.. when is SAP planning to include same in BAS model because E2E execution is a mandatory process and running in pipeline is a temporary solution. Please guide on same.
Thanks,
Saurabh
Hi Saurabh, I get your point and I fully agree that E2E execution (generally Selenium execution from workspace) is pretty important but BAS planning it beyond my responsibility. We have received this feedback from customers and we have addressed to BAS product management so I hope they will take it.
Fine Maxim.. Thanks for your input.. I saw a blog of yours which says some tasks to be implemented in BAS ..
https://github.com/SAP/ui5-uiveri5/blob/master/docs/todo.md
Is this related to runner for uiveri5 ?
Hi Saurabh, thanks for pointing this leftover, I just corrected it. Please note yeoman generators for both OPA and UIVeri5 tests run perfectly fine from BAS workspace console. But any further integrations like context actions or E2E test executions are now in hands of the BAS colleagues and I can't give any further details.
Thanks Maxim.. I believe yeoman generators can be used if our application(SAPUI5) is developed using same ? am i correct..
i was following Tsvetelina Aleksandrova blog and find sub generators for UIVeri5..
Yes, the yeoman generators can be used from BAS, just install and call from the BAS console.
Maxim is it mandatory that our application(SAPUI5) is developed using Yeoman? Then only we can use yeoman uiveri5 sub generators ?
it is generating its uiveri5 folder having its own package.json and node_modules. 1 project having 2 package.json is fine?
screenshot of uiveri5 as a
uiveri5 folder
Also getting below error related to chromedriver, this is by default picking the path :
DEBUG: Starting local chromedriver with executable: /home/openui5-sample-app/uiveri5/node_modules/@ui5/uiveri5/selenium/chromedriver-91.0.4472.101
[08:28:13] E/launcher - Server terminated early with status 127
Currently using TODO project of SAP github for exploring.
yeoman is only a scaffolding tool that you can use to startup the E2E test code, of course you can start from any other sample or from scratch.
Hi Maxim,
Since BAS does not support E2E testing tools like uiveri5. we are working on developing framework in VSCode. locally it works for us as we launch localhost index.html and then run uiveri5 TC(in config file we provide localhost url),however how do we access application being developed in BAS(as it is a complete different workspace).. To give an example : when code will be deployed to QA and we have to run functional suite how do we access same, as it is not a cloud URL which can be accessed via script(eg :https://workspaces-ws). what is the solution to such case ?
Hi,
I know it is possible to define an external URL/path for a server started from BAS workspace. But I have no precise idea how and if this requires some special configurations (like a cloud connector or something similar). Please check with BAS documentation and forums.
Hi Maxim,
Could you let me know if there is any blog you have for data driven testing approach using uiveri5. We wanted data in our application to pass through excel/csv files and use same.
Thanks,
Saurabh
Hi Shruti,
UIVeri5 is a "low-level" testing framework, think of it as "better selenium for UI5". It is not a full-blown environment with IDE and so on. So we do not have support for "data-driven testing" out of the box. At the same time, you should not forget that a UIVeri5 test is simply a nodejs module and you can code whatever you want there. So you can implement a test that reads a cvs file, parses it and uses the related data in the test.
Regards,
Maxim
Hi Maxim,
Thank you for a good introduction. We're planning our testing strategy and looking forward to start working with UIVeri5.
I have an important question around the framework:
Could you provide any recommendation on testing multiple SAPUI5 apps, which are stored on-premise? Let's say we have 5 different apps, each application implements its own set of e2e tests using UIVeri5. For example we would like to launch all the tests together (maybe with some common test suite) every time we update the UI5 library. Just to confirm everything still works as expected? I'm just wondering what would be the best option.
BR,
Adrian
Hi,
if there are only 5 (or a limited number) of test suites, why don't you start them sequentially, one after another? You can also define another test suite, that spans specs.js from multiple folders, of course you need to have them locally.
Regards,
Maxim
Hi,
Thanks for your response. At the moment it does not matter if they run in parallel or sequentially. The question is how to run a single test suite, which utilizes the tests from different apps (different base URLs), so we don't need to launch each one separately. In case we have e.g. 50 apps it would be very hard to launch the test manually one by one. We think of a solution when a non-technical person can launch a common test suite and verify if something goes wrong in the whole landscape (across all the apps). But it would be a bit problematic if you need to have all your apps locally. Probably it would be best to prepare some pipelines and integrations with Jenkins, but we're still in the process of building the expertise in that area.
BR,
Adrian
Hi,
pipeline to script this all is the best choice.
Regards,
Maxim
Hello,
We are planning the intergration of Uiveri5 with SAP Web IDE :
we are always getting the error below INFO: Executing 1 specs [15:02:08] I/launcher - Running 1 instances of WebDriver INFO: Opening webdriver connection with capabilities: {"browserName":"chrome","platform":"LINUX","chromeOptions":{"args":["--no-sandbox","--disable-dev-shm-usage","--disable-gpu","disable-infobars"]},"count":1,"loggingPrefs":{"browser":"SEVERE"}} [15:02:09] E/launcher - Server terminated early with status 127 [15:02:09] E/launcher - Error: Server terminated early with status 127 at /home/vcap/app/node_modules/selenium-webdriver/remote/index.js:252:52 at processTicksAndRejections (node:internal/process/task_queues:96:5) From: Task: WebDriver.createSession() at Function.createSession (/home/vcap/app/node_modules/selenium-webdriver/lib/webdriver.js:769:24) at Function.createSession (/home/vcap/app/node_modules/selenium-webdriver/chrome.js:761:15) at DirectDriverProvider.getNewDriver (/home/vcap/app/node_modules/@ui5/uiveri5/src/connection/directConnectionProvider.js:522:43) at Runner.createBrowser (/home/vcap/app/node_modules/protractor/built/runner.js:195:43) at /home/vcap/app/node_modules/protractor/built/runner.js:339:29 at _fulfilled (/home/vcap/app/node_modules/protractor/node_modules/q/q.js:834:54) at /home/vcap/app/node_modules/protractor/node_modules/q/q.js:863:30 at Promise.promise.promiseDispatch (/home/vcap/app/node_modules/protractor/node_modules/q/q.js:796:13) at /home/vcap/app/node_modules/protractor/node_modules/q/q.js:604:44 at runSingle (/home/vcap/app/node_modules/protractor/node_modules/q/q.js:137:13) we tried with several configuration without success. Hope that you could help to solve the issue. Thanks! Best regards.Hi Sirine,
WebIDE supports an explicit, very limited set of nodejs modules to be run at its backend. And this list does not include UIVeri5. Similar for BAS. So UIVeri5 test could not be executed directly in WebIDE/BAS. You can only use WebIDE/BAS as an editor but then you need to execute the tests locally or in your CI/CD system.
Regards,
Maxim