Skip to Content

In this tutorial, we give an overview of how to test applications which are written using the SAP S/4HANA Cloud SDK.

Note: This post is part of a series. For a complete overview visit the SAP S/4HANA Cloud SDK Overview.

Testing Pyramid

In general, there are multiple kinds of tests differentiating mainly in the granularity of testing. They all have their advantages and disadvantages. A common visualization is the testing pyramid. Based on the costs of the tests it visualizes that you should have much more unit tests than integration tests than E2E (End-To-End) tests. The costs for creating, running and maintaining increase while you move the pyramid up.

For example, End-To-End (E2E) tests are running on the user interface level. The test executor automatically clicks through the user interface asserting that applications behave in the desired way. Testing on this level shows that the features of the application are working. You could have, e.g., one E2E test per feature or user story. However, E2E tests are expensive to write and execute. E2E tests interact with the browser. This is expensive, because it takes time to start the browser and the browsers loads and renders a lot of resources. In addition, it is usually difficult to specify the interaction with the browser in a way that it can deal with smaller changes in the application user interface structure. Furthermore, E2E tests are sometimes flaky. Imitating the interaction with the browser is error-prone. The browser with changing loading times and asynchronous behaviour creates a quite unstable environment, which is a bad condition for tests. Thus, you should limit their number to a reasonable subset of functionality not tested otherwise. At least the majority of functionality should be already tested by unit and integration tests.

Integration tests have a reduced complexity. They skip the user interface and work directly on the defined backend APIs. They test the integration between software modules or systems. In our example, we mainly use them to test the integration between backend services and the integration to SAP S/4HANA systems.
Although they have a reduced complexity, they still have medium costs. They still have an overhead, e.g., for network communications or spawning a small server to make the backend APIs available. You should use them to verify that your backend services can communicate with your SAP S4HANA system and to test that the services behave as the user interface expects it.

Unit tests have the smallest granularity. They can be defined directly on the programming level, e.g., calling methods of your java classes. Dependencies to other modules or systems may be mocked to ensure that they run very quickly and only test the portion under consideration. These tests are usually very cheap. You should use them to verify that your software modules, such as classes, behave as expected.

Goal of this Blog Post

In this tutorial, we show how to write tests for your application. We will write the tests in a way that an automated build pipeline can execute them. At the end you should be able the write the following kind of tests for your application:

  1. Backend Unit Tests
  2. Integration Tests
  3. Frontend Unit Tests with Jasmine and Karma.
  4. End-To-End Tests with Nightwatch-Cucumber.

Unit-Tests Backend

For the modules of your backend services we suggest writing JUnit tests. The archetype already contains a submodule called unit-tests. Please place your unit tests there. They are in a separate module to make it easier for the pipeline to execute them separately. Furthermore, it avoids that test dependencies become part of the application by accident. This module already contains the JUnit dependency.

Unit tests should be very light weight. They should test the modules, e.g. a class, in a isolated manner. Other depended modules or even calls to external destinations, such as an ERP system, may be mocked. For mocking, the SAP S/4HANA Cloud SDK provides mocking facilities with the class MockUtil. Furthermore, we recommend using mocking frameworks, such as mockito or powermock.

Integration Tests Backend

In the integration tests you can tests your backend services without the frontend application. In the steps 2,3 and 4, we already introduced the integration tests and showed how to set them up. In general, we recommend to use Arquillian to spawn a small server containing only the resources for the specific backend services you want to test. This is faster compared to deploying it to the SAP Cloud Platform first and then testing against the deployed version. Furthermore, you still have the possibility to influence the test execution, e.g. with mocking or to collect test coverage data. For spring, we recommend to use the SpringRunner. For both, there is an example test already included in the corresponding archetype.

Mocking Basics

In this tutorial, we want to focus on the mocking facilities provided by the SAP S4HANA Cloud SDK. When you execute the integration tests locally, they run in a different environment than later on in the SAP Cloud Platform. Therefore, many services, such as the destination service, are not available, as the SAP Cloud Platform provides a much richer environment in terms of services and runtime libraries than the local test environment. This target environment has to be, partially, replicated in the local environment where the mocking facilities allow to test your business application without relying on the provided services locally. Since the SAP S/4HANA Cloud SDK already comes with abstractions for these services, it also offers to mock them in a local environment.

As example, we take the CostCenterServiceTest from Step 4 with SAP S/4HANA Cloud SDK: Calling an OData Service:

package com.sap.cloud.sdk.tutorial;

import com.jayway.restassured.RestAssured;
import com.jayway.restassured.http.ContentType;
import io.restassured.module.jsv.JsonSchemaValidator;
import org.jboss.arquillian.container.test.api.Deployment;
import org.jboss.arquillian.junit.Arquillian;
import org.jboss.arquillian.test.api.ArquillianResource;
import org.jboss.shrinkwrap.api.spec.WebArchive;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.slf4j.Logger;
 
import java.net.URL;
import java.net.URISyntaxException;

import com.sap.cloud.sdk.cloudplatform.logging.CloudLoggerFactory;
import com.sap.cloud.sdk.testutil.MockUtil;
 
import static com.jayway.restassured.RestAssured.given;
 
@RunWith( Arquillian.class )
public class CostCenterServiceTest
{
    private static final MockUtil mockUtil = new MockUtil();
    private static final Logger logger = CloudLoggerFactory.getLogger(CostCenterServiceTest.class);
 
    @ArquillianResource
    private URL baseUrl;
 
    @Deployment
    public static WebArchive createDeployment()
    {
        return TestUtil.createDeployment(CostCenterServlet.class);
    }
 
    @BeforeClass
    public static void beforeClass() throws URISyntaxException
    {
        mockUtil.mockDefaults();
        mockUtil.mockErpDestination();
    }
 
    @Before
    public void before()
    {
        RestAssured.baseURI = baseUrl.toExternalForm();
    }
 
    @Test
    public void testService()
    {
        // JSON schema validation from resource definition
        final JsonSchemaValidator jsonValidator = JsonSchemaValidator.matchesJsonSchemaInClasspath("costcenters-schema.json");
 
        // HTTP GET response OK, JSON header and valid schema
        given().
                get("/costcenters").
            then().
            assertThat().
                statusCode(200).
                contentType(ContentType.JSON).
                body(jsonValidator);
    }
}

Everything around mocking is located in the class MockUtil in the SAP S/4HANA Cloud SDK. Information about the interface and the available methods can be found in the JavaDoc.

In this example, we use the class MockUtil in the beforeClass method to mock common resources using mockUtil.mockDefaults() and to mock the connection to the S/4HANA system. The mockDefaults method calls several methods from the MockUtil to mock:

mockCurrentCloudPlatform() Mocks the SAP Cloud Platform environment, i.e. the application name. The Default application name is testapp.
mockCurrentLocales() Mocks the current locale to en-US
mockCurrentTenant() Mocks the tenant id to 00000000-0000-0000-0000-000000000000
mockCurrentUser() Mocks the user name to MockedUser
mockAuditLog() Makes the audit logger locally available not to cause run-time problems. The default behavior is to forward the logs to the standard logger.

As result, you can get platform specific information such as tenant or user information although you are not in a SAP Cloud Platform environment. Furthermore, this is the basis to mock destination such as the connection to an ERP system or other external services.

The SAP Cloud Platform has a destination concept. A destination has a name and a configuration. It specifies an URI and authentication parameters, such as basic authentication with username and password. MockUtil provides a way to mock a destination configuration together with a name. Thus, your application code can be the same in the tests and in the productive environment. Destinations can be mocked using the following ways:

//Mock system without authentication
String name = "myDestinationName";
URI uri = new URI("http://path/to/destination/");
mockUtil.mockDestination(name, uri);
 
//Authentication with be taken from credentials configuration for this alias
String alias = "myAlias";
TestSystem<GenericSystem> testSystem = new GenericSystem(alias, uri);
mockUtil.mockDestination(name, testSystem);
 
//System configuration and authentication configuration for this alias 
//from command line or file.
mockUtil.mockDestination(name, alias);

To configure the credentials you can either create a local file or pass the credentials over command line. A valid file is named credential.yml and is located in integration-tests/src/resources. The file contains a list of credentials referencing an alias. Possible formats are json and yaml format:

credentials:
  - alias: 'myAlias'
    username: 'username'
    password: 'password'

To pass the credentials over the command line you can use the command line parameter -Dtest.credentials. The value of should be a json string in the same format as in the file. Please be aware that, if you pass the credentials over the command line, the credentials might be persisted in the history and others might see it. If you decide for a file, you don’t have to enter it again for every test. However, be aware, that you should restrict the access and add the file to your gitignore.

In the third case, we mock the destination only by referencing an alias. In this case, you also have to configure the system in the file integration-tests/src/resources/systems.yml:

systems:
  - alias: "myAlias"
    uri: "http://path/to/destination/"

Mock ERP Destination

Mocking an ERP system uses the same mechanism. However, in addition you also have to provide a system id and a client id in addition to providing the systems as above. You can use either SapClient.EMPTY if the default client of a host is fine, or you use the convenience method ErpSystem.builder() that does not require a client id.

//Authentication with be taken from credentials configuration for this alias
String alias = "myAlias";
String systemId = "AAB";
SapClient sapClient = new SapClient("SAPCLIENT-NUMBER");
ErpSystem erpSystem = new ErpSystem(alias, uri, systemId, sapClient);
mockUtil.mockErpDestination(name, erpSystem);
 
//System configuration and authentication configuration 
//for alias from command line or file.
mockUtil.mockErpDestination(name, alias);

In accordance to this, the system.yml with ERP systems looks as follows. In cloud system do not need to specify systemId and sapClient.

erp:
  default: "myAlias"
  systems:
    - alias: "myAlias"
      systemId: "AAB"
      uri: "http://path/to/destination/"
      sapClient: "SAPCLINET-NUMBER"

There is even a short form of the code, because the destination name and the alias are optional. If you don’t provide a destination name, the destination name will be taken from ErpDestination.getDefaultName(). The default destination name is ErpQueryEndpoint. The alias will be taken from the default system in your systems.yml.

//Uses ErpDestination.getDefaultName() as destination name and
// the default system defined in the systems.yml
mockUtil.mockErpDestination();

Unit-Tests Frontend

In this tutorial, we will use Jasmine and Karma to implement the unit tests of javascript frontend code written in SAP UI5. As an example we take the application developed in the previous steps. Jasmine is a framework to write unit tests for JavaScript code. Karma is a tool to run the tests from console or in a build pipeline.

In order to make it testable, we refactor our application from step 9. We extract loading the cost centers from the SAP UI5 controller into a service class. We assume that your front end code is located in webapp. Place your service class in webapp/service/costcenters.js:

sap.ui.define([], function () {
    "use strict";
 
    return {
        getCostCenters: function (sapClient) {
            return jQuery.get("/costcenters")
        }
    }  
});

The code uses jQuery to send a GET request to a backend service of your application. It returns a promise for the list of cost centers.  If you already did step 11, please also add the CSRF token handling.

After the refactoring the imports in the controller in webapp/controller/View1.controller.js would look as follows:

sap.ui.define([
    "sap/ui/core/mvc/Controller",
    "sap/ui/model/json/JSONModel",
    "sap/m/MessageToast",
    "blog-tutorial/service/costcenters"
], function(Controller, JSONModel, MessageToast, CostCenterService) {

Replace the jQuery.get(...) with: CostCenterService.getCostCenters().

A test for this service class looks as follows:

"use strict";
/* eslint-disable max-nested-callbacks*/
sap.ui.define([
        "blog-tutorial/service/costcenters"
    ],
    function (CostCentersService) {

        //Create test data used for mocking and in the assertion
        var testCostCenters = [{
            "controllingArea": "A000",
            "name": "Cost Center 01"
        }]

        //Helper method to mock the service response as a jQuery Promise.
        function getCostCentersPromise() {
            var jQueryPromise = new $.Deferred();
            return jQueryPromise.resolve(testCostCenters);
        }

        describe("Cost Center Service", function () {
            it("gets cost centers", function (done) {
                //Mock the response for the real backend request
                spyOn(jQuery, "ajax").and.returnValue(getCostCentersPromise());
                CostCentersService.getCostCenters("001").then(function (costCenters) {
                    expect(costCenters).toEqual(testCostCenters);
                    expect(jQuery.ajax).toHaveBeenCalled();
                    done();
                });

            });

        });
    });

In Jasmine each test is wrapped into an it block. In this case it is an asynchronous test. First we mock the call to the backend in the spyOn method by overwriting the method jQuery.ajax. Then we call the CostCenterService implemented in the file before. In the callback we evaluate the assertions and mark the asynchronous test execution is completed. We place the test here frontend-unit-tests/tests/service/costcenters.jasmine.js.

In order to execute this test you can either create a new html page which executes the tests or you can use karma. We will use karma to be able to run the test from the command line. To run karma you have to install nodejs and create a package.json in the root of your project.

The package.json could look like:

{
  "name": "costcenter-controller-cf",
  "version": "1.0.0",
  "description": "Frontend Tests",
  "scripts": {
    "ci-test": "karma start frontend-unit-tests/karma.conf.js --watch=false --single-run=true"
  },
  "author": "SAP",
  "private": true,
  "devDependencies": {
    "yargs": "^6.6.0",
    "jasmine": "^2.6.0",
    "karma": "^1.7.0",
    "karma-chrome-launcher": "^2.1.1",
    "karma-coverage": "^1.1.1",
    "karma-jasmine": "^1.1.0",
    "karma-junit-reporter": "^1.2.0",
    "karma-openui5": "^0.2.2"
  }
}

In the scripts section you define the command ci-test which is used to run the unit tests with karma. In the section devDependencies you specify the dependencies needed to execute the tests. You can install them with running npm install in a terminal window.

The only piece missing is the configuration of karma. To configure karma you create a file called karma.conf.js in the folder frontend-unit-tests. This file looks as follows:

const argv = require("yargs").argv;

module.exports = function (config) {
    config.set({
        basePath: '../',

        frameworks: ['openui5', 'jasmine'],

        openui5: {
            path: 'https://sapui5.hana.ondemand.com/1.42.9/resources/sap-ui-cachebuster/sap-ui-core.js',
            useMockServer: false
        },

        client: {
            openui5: {
                config: {
                    theme: 'sap_bluecrystal',
                    resourceroots: {
                        'blog-tutorial': './base/webapp/',
                    }
                }
            }
        },

        files: [{
            pattern: 'webapp/**',
            served: true,
            included: false
        },
            './frontend-unit-tests/tests/**/*.jasmine.js'
        ],

        browsers: [argv.headless?"ChromeHeadless" :"Chrome"],

        reporters: ['junit', 'progress', 'coverage'],

        preprocessors: {
            'webapp/**/*.js': ['coverage'],
        },

        junitReporter: {
            outputDir: 's4hana_pipeline/reports/frontend-unit', // results will be saved as $outputDir/$browserName.xml
            outputFile: 'Test.frontend.unit.xml', // if included, results will be saved as $outputDir/$browserName/$outputFile
            suite: '', // suite will become the package name attribute in xml testsuite element
            useBrowserName: true, // add browser name to report and classes names
            nameFormatter: undefined, // function (browser, result) to customize the name attribute in xml testcase element
            classNameFormatter: undefined // function (browser, result) to customize the classname attribute in xml testcase element
        },

        coverageReporter: {
            // specify a common output directory
            dir: 's4hana_pipeline/reports/frontend-unit/coverage',
            reporters: [
                {
                    type: 'html',
                    subdir: 'report-html/ut'
                }, {
                    type: 'lcov',
                    subdir: 'report-lcov/ut'
                }, {
                    type: 'cobertura',
                    subdir: '.',
                    file: 'cobertura.frontend.unit.xml'
                }
            ],
            instrumenterOptions: {
                istanbul: {
                    noCompact: true
                }
            }
        },
    });
};

First, we define the base path, which is in our case the project root folder. In the openUi5 block, we can define where the resources for SAP UI5 are located and can be downloaded from. In the client block you can define the resource roots for your application components, as you also do it in your index.html of your SAP UI5 application. Please note that the path has to have ./base/ in front and then points to the folder where your frondend code is located. It is not in the actual path on the file system. In the files section you specify the location of the files that you need for testing. By default all files are loaded to execute the tests. Thus, we specify for the application code that it is not included but served when required in a test.

In the reporters section, we specify that we also want the results to be stored in the junit format and that we also want the code coverage data to be stored. These reporters are defined in the remaining sections. In preprocessors we define that all js files should be preprocessed to allow computing the code coverage. The format and the location of the junit and the code coverage reporters are specified in the sections coverageReporter and junitReporter.

The final folder structure should look as follows:

 

Now you can run the following commands in the root folder of the project to execute your tests. It will start a chrome instance and execute the tests.

npm install
npm run ci-test

It should run the tests and shows the results in the terminal:

In an delivery pipeline everything is usually executed without a user interface. Thus, opening a normal browser would not work. However, most browsers also offer a headless mode. The browser is started without a user interface. Note that you can run the script also with a headless browser: npm run ci-test -- --headless.

End-To-End Tests

For the E2E tests we use nightwatch-cucumber. It runs your tests using the chrome driver in the browser against a running version of your application. The version can run locally or in the SAP Cloud Platform.

From the previous steps we have an application which can list and create cost centers. Our first test is to load the page and check that it was loaded successfully. Tests can be defined in the cucumber format:

Feature: Cost Center Manage App

Scenario: Cost Center App opens
	Given I open the Cost Center home page
	Then the title is "Cost Center Explorer"
	And the Create button exists

We place that file in e2e-tests/features/costcenter-app.feature.

In the same folder we create a new folder for the steps. The steps define what exactly happens in each line of the feature file. A step basically implements the single lines in the feature file above. The file e2e-tests/features/step_definitions/steps.js looks like as follows:

const { client } = require('nightwatch-cucumber')
const { defineSupportCode } = require('cucumber')

defineSupportCode(({ Given, Then, When }) => {
  Given(/^I open the Cost Center home page$/, () => {
    const costcenter = client.page.costcenter()
	console.info (costcenter.url())
    return costcenter
      .navigate()
      .waitForElementVisible('@body')
  })

  Then(/^the title is "(.*?)"$/, (titleValue) => {
    const costcenter = client.page.costcenter()
    costcenter.assert.visible('@title')
    return costcenter.expect.element('@title').text.to.equal(titleValue)
  })

  Then(/^the Create button exists$/, () => {
    const costcenter = client.page.costcenter()
    return costcenter.assert.visible('@create')
  })

})

In the code we check multiple times for the title. To have the retrieving strategy for this element in a central place, we can use page objects. @title is defined in the page object e2e-tests/page_objects/costcenter.js and should make navigation easier:

module.exports = {
    url: function () {
        return this.api.launchUrl;
    },
    elements: {
        body: 'body',
        create: 
        {
           selector : "//button//*[text()='Create']",
           locateStrategy: "xpath"
        },
        title: 
        {
           selector : "//div[contains(@class, 'sapMTitle')]",
           locateStrategy: "xpath"
        }
    }
};

To configure nightwach we create the following file in e2e-tests/nightwatch.conf.js:

const argv = require("yargs").argv;

require('nightwatch-cucumber')({
    cucumberArgs: ['--require', 'timeout.js', '--require', 'features/step_definitions', '--format', 'json:../s4hana_pipeline/reports/e2e/cucumber.json', 'features']
})

var chromeOptionsArgs = ["--no-sandbox" , "window-size=1400,1000"]
if(argv.headless){
    chromeOptionsArgs.push("--headless")
    chromeOptionsArgs.push("--disable-gpu")
}

const options = {
    output_folder: '../s4hana_pipeline/reports/e2e',
    custom_assertions_path: '',
    page_objects_path: 'page_objects',
    live_output: false,
    disable_colors: false,
    globals_path: "external.globals.js",
    selenium: {
        start_process: false,
    },
    test_settings: {
        default: {
            launch_url : argv.launchUrl,
            selenium_port: 9515,
            selenium_host: '127.0.0.1',
            default_path_prefix: "",
            desiredCapabilities: {
                browserName: "chrome",
                javascriptEnabled: true,
                acceptSslCerts: true,
                chromeOptions: {
                    args: chromeOptionsArgs
                }
            },
            globals: {
                abortOnAssertionFailure : true,
                retryAssertionTimeout: 10000,
                waitForConditionTimeout: 10000,
                asyncHookTimeout : 10000
            },
            screenshots : {
                enabled : true,
                on_failure : true,
                on_error : true,
                path: '../s4hana_pipeline/reports/e2e/screenshots'
            }
        }
    }
}

module.exports = options;

This configuration file defines some constants and places where the reports are located. Please note, that we use the folder s4hana_pipeline/reports to store all reports. This path is later required by our pipeline. In launch_url you can see that that the URL tested against can be defined as command line argument.

An additional point is the configuration globals_path. It references the following file. This file starts the chromedriver server before the test execution and ends it afterwards:

e2e-tests/external.globals.js:

"use strict";

const chromedriver = require("chromedriver");

module.exports = {
    before: function (done) {
        console.log(chromedriver.path);
        chromedriver.start();
        done();
    },
    after: function (done) {
        chromedriver.stop();
        done();
    }
};

Furthermore, in the cucumber arguments we import the timeout.js which is located in the folder e2e-tests. It defines the default timeout of the tests.

const {defineSupportCode} = require('cucumber')

defineSupportCode(({setDefaultTimeout}) => {
  setDefaultTimeout(30 * 1000)
})

We also have to update the package.json to add an additional start script and additional dependencies:

{
  "name": "costcenter-controller-cf",
  "version": "1.0.0",
  "description": "Frontend Tests",
  "scripts": {
    "ci-e2e": "cd e2e-tests && nightwatch --skiptags=ignore",
    "ci-test": "karma start frontend-unit-tests/karma.conf.js --watch=false --single-run=true"
  },
  "author": "SAP",
  "private": true,
  "devDependencies": {
    "yargs": "^6.6.0",
    "chromedriver": "^2.29.0",
    "cucumber": "^3.0.3",
    "cucumber-html-report": "^0.6.0",
    "cucumber-junit": "^1.7.0",
    "jasmine": "^2.6.0",
    "karma": "^1.7.0",
    "karma-chrome-launcher": "^2.1.1",
    "karma-coverage": "^1.1.1",
    "karma-jasmine": "^1.1.0",
    "karma-junit-reporter": "^1.2.0",
    "karma-openui5": "^0.2.2",
    "nightwatch": "^0.9.15",
    "nightwatch-cucumber": "^8.2.2"
  }
}

The final folder structure looks as follows:

Now you can run the following commands to execute your tests. Note that the launch URL must be unprotected.

npm install
npm run ci-e2e -- --launchUrl=https://path/to/your/running/application

You should see the following result:

Continuous Integration and Delivery

Now you know how to write various kind of tests for your application. In one of our next tutorials about the SDK we will show you how to execute these tests in a build pipeline before you deploy your application to the SAP Cloud Platform. Thus, writing tests is very important to ensure a high quality of your application. Please already note that the pipeline will enforce that most of your productive code is covered by unit or integration tests. It will check the amount of productive code tested with unit or integration tests.

To report this post you need to login first.

Be the first to leave a comment

You must be Logged on to comment or reply to a post.

Leave a Reply