This is the third part of the blog-series about automated testing (end to end and integration-tests) using testcafe. The blog-series has the following parts:
Inside this third part we will get the current know-how "together" and make a real end to end test with nice code structering and reusable methods.
We are again using the SAP UI5 Shopping-Cart demo as example. We want to completly test the "standard" workflow:
Beforehand see the result of the automatic test-execution in the following gif. All code described in this blog post is maintained in Github. Simply clone the repository and run the test using the already described CLI (testcafe) to check the code.
Until now we have maintained selectors and test execution directly within the test file. This is not recommended. See the selector as part of your code. If you are acting on the same page element multiple times (and you will!) you will have to duplicate your selector, which will cause problems while maintaing your test-code (short: as always avoid duplicate code). The same is of course also valid for sets of test-actions (like e.g. maintaing payment or delivery information). Also this code should be reused.
To enable you can use the page model. pragma (which is not really testcafe specific, but plain typescript). Technically you are maintaing one typescript file per component. In this file you can collect your activites and selectors of this component. For the new test we are therefore using the following folder structure:
The pages productDemo.ts file contains a plain typescript class, in which you are defining your selectors and actions. A reduced example is shown in the following code snippet.
A lot is happening in this short snippet - let's have a look in detail:
A page-model file itself is never called by testcafe (only tests are). You can however use the selector and functions inside the test, as shown in the following code-snippet of the test file:
If you have multiple tests you can safely reuse this code now (this reduced the maintenance costs in case of changes inside the app - as you only have to adjust the code at one place). See the example of multiple tests:
A more complicated example of reuse is shown in the next code snippet, where we are maintaing the payment information:
You might ask - why the hassel? Just enter the values. This kind of structure allows us to implement the following test code, which is at first checking an invalid credit cart number, and afterwards continuing the test with a correct number:
Per default testcafe almost outputs nothing (only test start and end). This was enhanced within the ui5 addon for testcafe. In the enhanced version testcafe is automatically logging all activities (no matter if assertions or actions) inside the spool log.
Next to this spool output testcafe itself can also output junit/xunit files. With the help of the plugin you can also generate a code-coverage report. I will show this in the next part of the blog-post.
There are two main JSON configuration files to configure the test execution.
Please see the testcafe documentation for the full description of all configuration possibilities. In the following code snippet I am shopping a typical configuration I am using:
The .ui5-testcafe.json allows to configure mostly code-coverage.
Especially in CI scenarios, where e.g. jenkins is executing your test-code it can be very annoying to understand the Command-Line output. Testcafe allows to record your test code execution as mp4 video. This allows to implement a CI workflow, which is pushing this video to e.g. a JIRA bug and makes the life of a developer much easier.
To activate video handling you have to download and install ffmpeg. Ideally you are doing that installing the npm package @ffmpeg-installer/ffmpeg, which automatically downloads the ffmpeg version and installs it.
Afterwards adjust the testcafe-configuration file and add the following lines:
This will now automatically record a video of every test (as isolated file) and put it into the artifcats/videos subfolder, as you can see in the following gif:
In the next blog we will talk about running testcafe as integration test tool, code coverage and usage of XUNIT.xml format. Until than stay tuned. As always I am happy for feedback.
- Part 1: Overview and first test
- Part 2: Enhanced Selection, Action and Assertion Possibilities.
- Part 3: (this blog) Structure your test and common best-practices
- Part 4: Code-Coverage E2E tests, Integration Tests
- Part 5: Integration in CI (Jenkins / Github Actions) / Authentification
- Part 6: Use testcafe for SAP Analytics Cloud and SAP Lumira
Inside this third part we will get the current know-how "together" and make a real end to end test with nice code structering and reusable methods.
Test-Scenario
We are again using the SAP UI5 Shopping-Cart demo as example. We want to completly test the "standard" workflow:
- Search Item in Master-Page
- Add to Cart
- Maintain Payment and Delivery Information
- Order the Cart
Beforehand see the result of the automatic test-execution in the following gif. All code described in this blog post is maintained in Github. Simply clone the repository and run the test using the already described CLI (testcafe) to check the code.
Structure your test
Until now we have maintained selectors and test execution directly within the test file. This is not recommended. See the selector as part of your code. If you are acting on the same page element multiple times (and you will!) you will have to duplicate your selector, which will cause problems while maintaing your test-code (short: as always avoid duplicate code). The same is of course also valid for sets of test-actions (like e.g. maintaing payment or delivery information). Also this code should be reused.
To enable you can use the page model. pragma (which is not really testcafe specific, but plain typescript). Technically you are maintaing one typescript file per component. In this file you can collect your activites and selectors of this component. For the new test we are therefore using the following folder structure:
- Folder tests is containg all tests you want to execute
- Folder pages is containg all pages / components you want to work on
The pages productDemo.ts file contains a plain typescript class, in which you are defining your selectors and actions. A reduced example is shown in the following code snippet.
import { ui5, ui5Action } from "ui5-testcafe-selector-utils";
class ProductDemo {
readonly masterPage = {
searchfield: ui5().id("homeView--searchField"),
productList: ui5().id("homeView--productList"),
productCatList: ui5().id("homeView--categoryList"),
listEntry(productId: string) {
return ui5().listItem().context("ProductId", productId)
}
}
async addToShoppingCart(productDescr: string, productId: string) {
await ui5Action.expect(this.masterPage.productCatList).tableLength().greater(0, "Initially the product-list must be filled");
await ui5Action.typeText(this.masterPage.searchfield, productDescr, { replace: true }).
click(this.masterPage.listEntry(productId));
}
};
export default new ProductDemo();A lot is happening in this short snippet - let's have a look in detail:
- ui5Action is imported from ui5-testcafe-selector-utils. This is a singelton instance class which allows you to access the actions API (e.g. typeText etc.), without having to pass the "u"-object from the test to all methods.
- The class ProductDemo should define all selectors and action-sets inside the Application. We will only have one Page File for the overall application. Depending on the scenario you can also have more than one page file per app - this really depends on how you want to structure your code and tests.
- The class is having one readonly Attribute per page (shortend code-snipped is only containg the master page), which is defining the selectors of all relevant page elements of the master page. Node that you can of course also define "dynamic selectors", by generating the selectors inside a function. Here this is used to create a list-entry selector depending on the product-id.
- The asynchronous method "addToShoppingCart" is accessing the selectors to maintain text inside the search field and clicking the found entry in the result-list.
- As you see witin the last two instructions (typeText and click) you can chain actions on multiple page elements, without having to repeat the boiler-plate-code "ui5Action.".
A page-model file itself is never called by testcafe (only tests are). You can however use the selector and functions inside the test, as shown in the following code-snippet of the test file:
import productDemo from "../pages/productDemo";
ui5Fixture("Shop-App", "https://sapui5.hana.ondemand.com/test-resources/sap/m/demokit/cart/webapp/index.html?sap-ui-theme=sap_fiori_3");
ui5Test('Create and Order Shopping-Cart', async u => {
//add HT-1035 to shopping-cart
await productDemo.addToShoppingCart("Flat Basic", "HT-1035");
//...
await u.click(productDemo.detailPage.navtoCartButton);
});- The page-model class is imported from the pages folder
- The addToShoppingCart method is reused inside the test
- The navToCartButton selector is reused inside the test click action
If you have multiple tests you can safely reuse this code now (this reduced the maintenance costs in case of changes inside the app - as you only have to adjust the code at one place). See the example of multiple tests:
import productDemo from "../pages/productDemo";
ui5Fixture("Shop-App", "https://sapui5.hana.ondemand.com/test-resources/sap/m/demokit/cart/webapp/index.html?sap-ui-theme=sap_fiori_3");
ui5Test('Create and Order Shopping-Cart', async u => {
//add HT-1035 to shopping-cart
await productDemo.addToShoppingCart("Flat Basic", "HT-1035");
//...
});
ui5Test('Order and Cancel Shopping-Cart', async u => {
//add HT-1036 to shopping-cart
await productDemo.addToShoppingCart("Flat", "HT-1036");
//...
});A more complicated example of reuse is shown in the next code snippet, where we are maintaing the payment information:
interface CreditPaymentDetails {
holder?: string,
number?: string //...
}
class ProductDemo {
//...
async maintainCreditCart(details: CreditPaymentDetails) {
if (details.holder) {
await ui5Action.typeText(this.creditCartView.holder, details.holder, { replace: true, confirm: true });
}
if (details.number) {
await ui5Action.typeText(this.creditCartView.number, details.number, { replace: true, confirm: true });
}
//...
}
};
export default new ProductDemo();- The selectors are very similar - For the input itself we are using an interface, which has the paramters to be filled (shortend in the snipped).
- Depending if an attribute value is passed, the field is filled or not.
- Note that the type options replace and confirm are set to "true". This will overwrite a possibly existing test and press enter after typing the text.
You might ask - why the hassel? Just enter the values. This kind of structure allows us to implement the following test code, which is at first checking an invalid credit cart number, and afterwards continuing the test with a correct number:
//maintain wrong credit cart data
await productDemo.maintainCreditCart({ holder: "Max Mustermann", number: "1234", cvn: "123", expiration: "10/2022" });
//check that number input field has an error
await u.expect(productDemo.creditCartView.number).property('valueState').equal('Error');
//enter correct value
await productDemo.maintainCreditCart({ number: "1234567891234567" });
//value state should be OK now
await u.expect(productDemo.creditCartView.number).property('valueState').equal('None');Output of the test
Per default testcafe almost outputs nothing (only test start and end). This was enhanced within the ui5 addon for testcafe. In the enhanced version testcafe is automatically logging all activities (no matter if assertions or actions) inside the spool log.
Next to this spool output testcafe itself can also output junit/xunit files. With the help of the plugin you can also generate a code-coverage report. I will show this in the next part of the blog-post.
Configure your test
There are two main JSON configuration files to configure the test execution.
- testcafe can be maintained both via file (.testcaferc.json) and command line arguments (see testcafe documentation)
- the ui5 addon can be maintained via file (.ui5-testcafe.json) and enviroment variables (see upcoming blog-posts)
Please see the testcafe documentation for the full description of all configuration possibilities. In the following code snippet I am shopping a typical configuration I am using:
{
"browsers": "chrome --start-maximized", //can also be an array
"selectorTimeout": 60000, //increase timeout to let ui5 more time during complex actions
"pageLoadTimeout": 20000, //see above
"assertionTimeout": 20000, //see above
"disableMultipleWindows": true, //testcafe in theory allows the handling of multiple windows. as this feature is still in beta stage, we are not using it
"stopOnFirstFail": false, //i want that a test is executed even though a predecessor test failed
"skipJsErrors": true, //i want to ignore possible unhandeld js exceptions. It is just too common too see those JS errors inside especially Launchpad applications. So better ignore them
"screenshots": { //screenshot of failures are automatically generated and stored inside that path
"path": "artifacts/screenshots"
},
//reporters are reporting about the test execution. There are quite a lot of reporters (Xunit, Coverage, ..). The easiest scenario is only using the "spec" reporter, which is writing the log to the command line
"reporter": [
{
"name": "spec"
}
]
}The .ui5-testcafe.json allows to configure mostly code-coverage.
{
"firstSelectorTimeout": 90000, //the first ui5 selector waits 90 seconds for the initialization (overwrites the testcafe settings). intenended for slow application startups
"traceSelectorOnFailure": true //in case of a failure (dom element can't be found), the last selector is automatically traced (see last blog)
}Activate Videos of your test
Especially in CI scenarios, where e.g. jenkins is executing your test-code it can be very annoying to understand the Command-Line output. Testcafe allows to record your test code execution as mp4 video. This allows to implement a CI workflow, which is pushing this video to e.g. a JIRA bug and makes the life of a developer much easier.
To activate video handling you have to download and install ffmpeg. Ideally you are doing that installing the npm package @ffmpeg-installer/ffmpeg, which automatically downloads the ffmpeg version and installs it.
npm i @ffmpeg-installer/ffmpeg --save-devAfterwards adjust the testcafe-configuration file and add the following lines:
{
//...
"videoPath": "artifacts/videos",
"videoOptions": {
"pathPattern": "${TEST}_${FILE_INDEX}", //filename of the video - see testcafe documentation
"ffmpegPath": "C:/Tools/ffmpeg/bin/ffmpeg.exe" //installation folder - can also be passed via enviroment variable
}
//...
}This will now automatically record a video of every test (as isolated file) and put it into the artifcats/videos subfolder, as you can see in the following gif:
In the next blog we will talk about running testcafe as integration test tool, code coverage and usage of XUNIT.xml format. Until than stay tuned. As always I am happy for feedback.
- SAP Managed Tags:
