Skip to Content

The following blog post will explain how you can start using the S/4HANA Cloud SDK Continuous Integration and Continuous Delivery server to build, test, and deploy your project.

Note: This post is part of a series. For a complete overview, visit the SAP S/4HANA Cloud SDK Overview.

Introduction

The SAP S/4HANA Cloud SDK enables you to develop and release your application in short cycles according to the software engineering best practices continuous integration (CI) and continuous delivery (CD).

To achieve this, the SAP S/4HANA Cloud SDK makes use of a set of well attuned components:

  • Maven project archetypes for the rapid creation of a new S/4HANA extension application.
  • A pre-configured Jenkins CI server, available via Docker Hub.
  • A SAP S/4HANA Cloud SDK specific CI & CD pipeline and library for building, testing, quality checking, and deploying applications that conform to the structure of the SAP S/4HANA Cloud SDK archetypes.

Prerequisites

For getting started with Cx for your project, you need to assure the following prerequisites.

  • Linux host with Docker: For instantiating the SAP S/4HANA Cloud SDK Cx Server, you need to provide a suitable host with a linux operating system and Docker installed. Please also ensure that the user with whom you start the Cx Server belongs to the docker group.
  • Project sources in git or github: Your project source files need to be available on a git or github server, which is accessible from the Cx Server host.

The S/4HANA Cloud SDK Pipeline uses Docker images for each individual build step. Accordingly, you do not need to take care of installing any further dependencies on your host machine. All tools required for building, testing, quality checking, and deploying your applications, are dynamically retrieved in the form of Docker images.

Installing and Managing the Cx Server

The lifecycle of the Cx Server is maintained by a script called cx-server. It can be found in the same named folder on the root of each SAP S/4HANA Cloud SDK project archetype. Together with the server.cfg file, this is all you need for starting your instance of the SAP S/4HANA Cloud SDK Cx Server.

Copying the Cx Server lifecycle management script to the target host

First, “cx-server” folder needs to be copied to the future host on which the Cx Server is intended to run. For this, copy the folder from your archetype to a place on your linux server, for example, via SCP:

scp -r cx-server username@cx-host.corp:/~/cx-server

Note: If you are copying the cx-server from a Microsoft Windows machine, the target files might use Windows line endings, which makes them unusable on the linux host. To correct this, you can use dos2unix or similar tools to convert the line endings of the cx-server script and its configuration in server.cfg. Also assure that the cx-server script is executable by the current user.

Example:

$ dos2unix ./cx-server/*
dos2unix: converting file ./cx-server/cx-server to Unix format...
dos2unix: converting file ./cx-server/server.cfg to Unix format...
$ chmod +x cx-server/cx-server

Next, login to your linux host and navigate to the cx-server folder. After that, you are ready to start your SAP S/4HANA Cloud SDK Cx Server instance.

Starting the SAP S/4HANA Cloud SDK Cx Server

Starting the Cx Server is very simple. All you need to do, is to change to the cx-server directory and execute the cx-server start command. This will start the Cx Server with its default settings on port 80 of your host machine.

If not done before, the script downloads the newest version of the SAP S/4HANA Cloud SDK Cx Server from Docker Hub and instantiates a container from it. Because of the initial download, the first invocation might take a few minutes. In case, the Cx Server has been started and stopped earlier, the existing container will be reused.

The state of Jenkins will be stored in a persistent Docker volume named jenkins_home_volume. This volume contains all critical data, comprising the Jenkins configuration, plugins, and build related artifacts. Therefore, even if your container crashes, no data of your Cx Server will be lost.

If everything goes well, you should be able to see output, which is similar to the following:

$ cd cx-server
$ ./cx-server start
Checking for newer version of this script on github...
no newer version detected.

Starting docker container for S/4HANA Cloud SDK Cx Server.
Parameters:
   - http_port=80
   - docker_image=s4sdk/jenkins-master:latest
   - jenkins_home=jenkins_home_volume

>> docker pull s4sdk/jenkins-master:latest
latest: Pulling from s4sdk/jenkins-master
...
e7ad665b2ea1: Already exists 
9094d1930c09: Pull complete 
1ae0fd7ec49a: Downloading [==>        ]  9.191MB/221MB
...
Status: Image is up to date for s4sdk/jenkins-master:latest
>> docker run -u 1000:115 --name s4sdk-jenkins-master -d -p 80:8080
 -v /var/run/docker.sock:/var/run/docker.sock
 -v jenkins_home_volume:/var/jenkins_home
 s4sdk/jenkins-master-sap:latest
0e214f9dcf3d9e1d2839638901d60f3067edb635067c640fe16b7c18a6b65607
Waiting for the S/4HANA Cloud SDK Cx server to start......... success.

Congratulations! Your Cx Server is now up and running. Give it a visit by entering the IP or domain name of your host machine in your favorite browser. Jenkins should welcome you with the following screen:

Next, you can continue with the basic setup and start building your project by adding your source code repository.

Stopping the SAP S/4HANA Cloud SDK Cx Server

For stopping your Cx Server instance, you can use the command cx-server stop. It will stop the Cx Server container in a safe manner by waiting for all jobs to finish before shutdown.

Note: Before the shutdown can be performed, you have to provide the credentials of a (technical) user who is enabled to perform a safe shutdown of the server. If you did not secure the server yet, you can skip entering credentials by leaving the user name empty.

$ ./cx-server stop 
Checking for newer version of this script on github...
no newer version detected.

Jenkins username (leave empty for unprotected server): user
Password for user: 

Initiating safe shutdown...
>> docker exec 0e214f9dcf3d curl
 -w '%{http_code}' -u 'user:******' -o /dev/null
 -s -X POST 'http://localhost:8080/safeExit'
200
Waiting for running jobs to finish...... success.

Creating a new SAP S/4HANA Cloud SDK Cx Server Container

In certain cases, you will need to create a new Cx Server container, for example, for:

  • updating the Cx Server to a new version
  • effectively applying configuration changes
  • troubleshooting

For this, you can stop the server and call cx-server remove to delete its corresponding docker container. To create a new instance, you can subsequently call cx-server start. This will instantiate a new container based on the current configuration. As long as you do not change the home directory of Jenkins, the new container will reuse the previous Cx Server configuration and all other persisted state information.

Configuring the SAP S/4HANA Cloud SDK Cx Server

You can use the cx-server/server.cfg file to customize your instance of the SAP S/4HANA Cloud SDK Cx Server. To get an overview of the available options, you can open the file and have a look at the explanatory comments of each option. Changes to the configuration will become effective after creating a new container. This means that, if you are already running an instance of the Cx Server, you have to stop and remove it, before the new settings will become effective.

As of today, the following options can be customized:

  • docker_registry and docker_image: The docker registry, respectively docker image to be used for instantiating your Cx Server. When no docker_registry is specified, the docker engine will use docker hub as a default and pull the s4sdk/jenkins-master image in its latest version.
  • http_port: The port on which your Cx Server will be reachable.
  • jenkins_home: The resource to be used as jenkins home directory. You can use an absolute path pointing to a folder on your host machine, for example, jenkins_home="/var/jenkins_home".  Alternatively, you can use a named docker data volume such as the default value jenkins_home="jenkins_home_volume".
  • x_java_opts: Custom java options to be passed to the Jenkins Java VM. You can use it to adapt the maximum heap size of its Java VM or to set Java’s proxy settings, for example, java_opts="-Xmx1024m -Dhttp.proxyHost=proxy -Dhttp.proxyPort=8080 -Dhttps.proxyHost=proxy -Dhttps.proxyPort=8080 -Dhttp.nonProxyHosts=localhost|*.corp".
  • http_proxy, https_proxy, no_proxy: Proxy settings to be passed as environment variables to the docker container, for example: http_proxy="http://proxy:8080", http_proxy="http://proxy:8080",
    no_proxy="localhost,.corp"

Note: If you choose to store Jenkins home in a local folder, you have to ensure that the Jenkins user within the Cx Server container, which is fixed to uid=1000, is able to read and write within the specified folder. To avoid permission related issues, we recommend to use named docker volumes.

Building your first project

After configuring and starting your Cx Server instance, you are ready to build, test, and deploy your first S/4HANA extension application. This application needs to be created from one of the S/4HANA Cloud SDK archetypes.

Before creating your build job, you need to configure your SCM system of choice. In our example, we will assume that your project is located in GitHub Enterprise.

Note: Jenkins also supports other SCM systems such as GitLab or plain git. You are free to install the corresponding plugin to enable your custom SCM setup.

Setup GitHub Enterprise

Go to your Jenkins main page and navigate to “Manage Jenkins » Configure System”. On the configuration page, search for “GitHub Enterprise Servers”. Here, add the API endpoint of your GitHub Enterprise Server and assign a name to it. Now you are ready to create a build job for your project.

Create a Jenkins job for your project

To create a build job for your project, navigate to “New Item” in the Jenkins main menu. On the following screen, choose “Multibranch Pipeline” and specify a name for your project’s build job.

Next, you need to setup the SCM source of your project’s build job. Look for “Branch Sources” and add one by specifying your GitHub Enterprise endpoint, organization, and repository. If your repository is set to private, please also create and use a pair of suitable credentials and choose them for your repository.

Congratulations! Jenkins will now scan your repository and trigger a build.

The best way to get an overview of the build status is to open the job, navigate to the currently running build, and opening it in the blue ocean UI of Jenkins.

For enabling Jenkins to automatically run builds in the future, you should open GitHub Enterprise and navigate to your “GitHub organization » Settings » Hooks” and create a webhook pointing to your freshly created Cx Server, for example, http://cx-server.corp/github-webhook/. Depending on your SCM system, there might be similar ways to realize the notification.

This is also a good point in time to setup Jenkins to know its own URL. Navigate to “Manage Jenkins » Configure System”, search for “Jenkins Location”, and specify “Jenkins URL” .

Jenkins will use this value to create links to its resources, for example, when creating quality check entries in context of pull requests that are initiated by developers within the project.

Working with the SAP S/4HANA Cloud SDK Pipeline

In the following, we will introduce the SAP S/4HANA Cloud SDK Continuous Delivery Pipeline with its stages and key characteristics. Moreover, we show you, how you can configure it for use in your individual context.

Pipeline stages

The following screenshot shows the stages of the SAP S/4HANA Cloud SDK Pipeline. The text below briefly introduces each stage and its tasks.

  • Declarative Checkout SCM: In this stage, your project’s source files are retrieved from the configured SCM URL.
  • Init: The init stage initializes the SAP S/4HANA Cloud SDK pipeline by, for example, loading your individual pipeline configuration.
  • Build: Parallelized build of frontend and backend components 
    • Backend: The “Build Backend” sub-stage executes the maven goal install in the root of your project as defined in your pom.xml. It will not compile or run any tests.
    • Frontend: The “Build Frontend” sub-stage executes npm install in the root of your project. This will download frontend dependencies and test tools as defined in your project.json.
  • Local Tests: Parallelized tests of frontend and backend components.
    • Backend Unit Tests: Executes the maven goal test in the unit-test module of your project and collects code coverage information.
    • Backend Integration Tests: Executes the maven goal test in the integration-test module of your project and collects code coverage information.
    • Frontend Unit Tests: Executes the npm script ci-test from your project’s package.json.
    • Static Code Checks: Executes a set of static code checks against your codebase and adds the results to your Jenkins build. The code checks are a composition of custom and sap-specific rules.
  • Remote Tests: Parallelized tests against deployments of your project.
    • End to End Test: Executes the npm script ci-e2e against a test deployment of your application and collects the results.
    • Performance Tests: Executes your JMeter Test Suite and collects the results.
  • Quality Checks: Collects report data of previous stages and applies SAP-specific quality checks.
  • Production Deployment: Deploys your application to your SAP Cloud Platform targets.

Design principles

This section introduces the design principles that shaped the design of the S/4HANA Cloud SDK Pipeline.

Pipeline as Code

The S/4HANA Cloud SDK Pipeline is distributed as code. This code describes the orchestration of build steps and is available in form of a Open Source project on SAP’s GitHub repository. From there, every S/4HANA Cloud SDK will by default retrieve the most recent pipeline code. This enables your project to rapidly leverage future improvements of the S/4HANA Cloud SDK pipeline.

Build logic in Global Pipeline Library

The actual logic of each pipeline stage is externalized to a reusable Jenkins Pipeline Library which can also be found on GitHub. The SAP S/4HANA Cloud SDK Cx Server is already configured to make the library available to pipeline-builds running inside it.

Strict isolation between build runs

Unexpected side effects between builds are a major problem in many of today’s CI landscapes. Therefore, we decided to conduct each S/4HANA Cloud SDK Pipeline build in strict isolation. None of the build tools are operated in a way that allows side effects between multiple builds. In consequence, no persistent local repositories are used for maven or npm.

Note: Today’s software projects typically contain dozens of dependencies and downloading them for each build can notably increase build time. To address this challenge, it is planned to augment the SAP S/4HANA Cloud SDK Cx Server with a download cache, which will locally store dependencies of your project and significantly increase the speed of retrieving them during a build.

Proper isolation of individual build steps

To foster modularity and a clear separation of concerns, the Cx Server does not deliver any specific build tools. All dependencies are externalized to specific docker images in which the actual work is executed. These containers are instantiated and subsequently disposed for each build step. This further assures that no unintended state is leaked between build steps or other build jobs. Relevant data is explicitly maintained in Jenkins stashes. There is no implicit data passing between stages via workspace artifacts. Each stage only stashes artifacts which are relevant for follow-up stages. Furthermore, each stage only unstashes files of relevance for its own execution.

Parallelization

To reduce the total build time, the S/4HANA Cloud SDK Pipeline makes strong use of parallel build stages.

Checked qualities

To assist you in achieving high quality within your application, the S/4HANA Cloud SDK Pipeline applies a set of quality rules to your project. At the time of writing this, the following checks are applied.

Exclusive use of whitelisted APIs

The pipeline is able to detect the use of non-whitelisted APIs which are prohibited for S/4HANA Cloud side-by-side extension applications.

Resilient invocation of 3rd party services

For the resilient invocation of 3rd party services, we recommend to use Hystrix, which implements Circuit Breaker and Bulkhead semantics. The pipeline will inform you about non-resilient invocations of downstream services that might cause cascading errors in the production deployment of your application.

Proper error handling

Proper error handling is crucial for rapidly diagnosing and fixing a malfunctioning application. The pipeline is able to warn you about unsafe ways of error handling and will help you to establish the prerequisites of good post-mortem-analyzability.

Minimum code coverage

Tests are important to assure the functional correctness of your application. If your tests only cover a small fraction of your code, the pipeline will indicate this.

Platform abstractions

The SAP S/4HANA Cloud SDK offers abstractions for many cloud environment specific APIs. This allows you to easily migrate your application between cloud environments, for example, from SAP SCP Neo to SAP SCP CloudFoundry. To assure that you are not unnecessarily binding against specific APIs, the pipeline will inform you about places in your code that should use corresponding S/4HANA Cloud SDK abstractions.

Pipeline configuration

To adjust the SAP S/4HANA Cloud SDK Pipeline to your project’s needs, it can be customized on multiple levels. Currently, this comprises a general configuration on project level, a step configuration to set default values for steps, and a stage level to set configuration values for a specific stage. If a property is configured on step as well as stage level, the stage level value takes precedence.

General configuration

Property Default Value Description
productiveBranch master The name of your default branch. This branch will be used for deploying your application. Other branches will skip deployment.
nonErpDestinations List of destination names that do not refer to ERP systems. Use this parameter to exclude specific destinations from being checked in context of ERP API whitelists.

Stage configuration

buildBackend

Property Default Value Description
dockerImage maven:3.5-jdk-7-alpine

The docker image to be used for building the application backend.

Note: This will only change the docker image used for building the backend. Tests and other maven based stages will still use their individual default values. For switching all maven based steps to a different maven or JDK version, you should configure the dockerImage via the executeMaven step.

buildFrontend

Property Default Value Description
dockerImage s4sdk/docker-node-chromium

The docker image to be used for building the application frontend.

Note: This will only change the docker image used for building the frontend. End to end tests and other npm based stages will still use their individual default values. For switching all npm based steps to a different npm or chromium version, you should configure the dockerImage via the executeNpm step.

staticCodeChecks

Property Description
pmdExcludes A comma separated list of exclusions expressed as an Ant style pattern relative to the application folder. Example: src/main/java/generated/**
findbugsExcludesFile Path to a FindBugs XML exclusion file relative to the application folder.

unitTests

Property Default Value Description
dockerImage maven:3.5-jdk-7-alpine

The docker image to be used for running unit tests.

Note: This will only change the docker image used for executing the unit tests. For switching all maven based steps to a different maven or JDK version, you should configure the dockerImage via the executeMaven step.

integrationTests

Property Default Value Description
dockerImage maven:3.5-jdk-7-alpine

The docker image to be used for running integration tests.

Note: This will only change the docker image used for executing the integration tests. For switching all maven based steps to a different maven or JDK version, you should configure the dockerImage via the executeMaven step.

credentials The list of system credentials to be injected during integration tests. The following example will provision the username and password for the systems with the aliases ERP and SFSF. For this, it will use the Jenkins credentials entries erp-credentials and successfactors-credentials. You have to ensure that corresponding credential entries exist in your Jenkins configuration.

credentials:
  - alias: 'ERP'
    credentialId: 'erp-credentials'
  - alias: 'SFSF'
    credentialId: 'successfactors-credentials'

frontendUnitTests

Property Default Value Description
dockerImage s4sdk/docker-node-chromium

The docker image to be used for running frontend unit tests.

Note: This will only change the docker image used for unit testing in the frontend. For switching all npm based steps to a different npm or chromium version, you should configure the dockerImage via the executeNpm step.

endToEndTests

Property Description
cfTarget The CloudFoundry deployment target to be used for running the end to end tests. Example:

cfTarget:
  org: 'MyOrg'
  space: 'Test'
  apiEndpoint: 'https://api.cf.sap.hana.ondemand.com'
  appName: 'testapp'
  manifest: 'manifest-test.yml'
  credentialsId: 'deploy-test'
  deploymentType: 'standard'
neoTarget The Neo deployment target to be used for running the end to end tests. Example:

neoTargets:
  - host: 'int.sap.hana.ondemand.com'
    account: 'x5e5e111a'
    application: 'testapp'
    credentialsId: 'deploy-test'
      ev:
        - 'STAGE=Production'
      vmArguments: '-Dargument1=value1 -Dargument2=value2'
      runtime: 'neo-javaee6-wp'
      runtimeVersion: '2'
appUrl The URL under which the app is available after deployment.

performanceTests

Property Description
cfTargets The list of CloudFoundry deployment targets required for the performance test stage.
neoTargets The list of Neo deployment targets required for the performance test stage.

productionDeployment

Property Description
cfTargets The list of productive CloudFoundry deployment targets to be deployed when a build of your productive branch succeeds.
neoTargets The list of productive Neo deployment targets to be deployed when a build of your productive branch succeeds.

Step configuration

executeMaven

The executeMaven step is used for all invocations of the mvn build tool. It is either used directly for executing specific maven phases such as test, or indirectly for steps that execute maven plugins such as checkPmd.

Property Default Value Description
dockerImage maven:3.5-jdk-7-alpine The image to be used for executing maven commands.
globalSettingsFile   The global settings.xml to be used for maven builds. You can specify a relative path to your project root or a URL starting with http or https.
projectSettingsFile   The project settings.xml to be used for maven builds. You can specify a relative path to your project root or a URL starting with http or https.

executeNpm

The executeNpm step is used for all invocations of the npm build tool. It is, for example, used for building the frontend and for executing end to end tests.

Property Default Value Description
dockerImage s4sdk/docker-node-chromium The image to be used for executing npm commands.

Troubleshooting

In the future, we will use this section to address common challenges and how they can be resolved. For example, how you can access the Cx Server log files, how you can get shell access to your Cx Server, and how to customize it for your environment when you are using self signed certificates.

Security Notes

The Cx Server is built with care and based on well established technologies. However, as with most tools, their are certain risks involved that you should properly manage within your setup. Therefore, please carefully read the following security notes before starting to use your Cx Server productively.

Setup Jenkins security and disable anonymous access

Initially, the Jenkins server within the docker image is not secured in any manner. Anyone with network access to the Cx Server host can access and modify its settings. In your Jenkins, navigate to “Manage Jenkins » Configure Global Security” and apply a suitable security setup for your organization and its users.

Take care who can modify the pipeline

Jenkins automatically executes your project’s pipeline definition after it is notified about changes. This can be leveraged by attackers that are able to inject code that is then executed on your build server. Carefully manage which users are able to make changes to your pipeline definition, for example, by maintaining an explicit list of allowed committers in GitHub.

Take care who can access the Docker socket

To be able to start other docker containers, the Cx Server has access to Docker’s control socket under /var/run/docker.sock. Attackers that manage to intrude your Cx Server container, will be able to launch other containers that then might interfere with other resources on your system. This is a general security risk in setups where docker containers need to be able to launch other containers. Accordingly, we advice you to carefully control who can directly or indirectly access your Cx Server. To avoid unnecessary risks, you should avoid running any other business critical applications on the same host.

Additional Resources

Jenkins Documentation

Docker Documentation

To report this post you need to login first.

Be the first to leave a comment

You must be Logged on to comment or reply to a post.

Leave a Reply