Deep Dive 4 with SAP Cloud SDK: Continuous Delivery Pipeline for SAP Cloud SDK Based Projects

[former_member257185]

Disclaimer: The SAP Cloud SDK for Continuous Delivery and its features have been merged into project "Piper". Therefore we recommend to use the General Purpose Pipeline of project "Piper" instead of the SAP Cloud SDK pipeline. The reasoning for this change, as well as further information on how to adopt the General Purpose Pipeline are described in this guide.

This blog post introduces the SAP Cloud SDK Continuous Delivery Pipeline. It introduces the pipeline structure and the principles behind its design. Finally, it introduces the set of enterprise software qualities which are checked as part of the Quality Checks stage in each pipeline execution.

Note: This post is part of a series. For a complete overview visit the SAP Cloud SDK Overview.

Goal of this blog post

This blog post is your starting point for learning more about the SAP Cloud SDK Continuous Delivery Pipeline. Moreover, the post should help you to understand how you can make optimal use of the pipeline in your projects.

The SAP Cloud SDK Pipeline

In the following, we will introduce the SAP Cloud SDK (fka SAP S/4HANA Cloud SDK) Continuous Delivery Pipeline with its stages and key characteristics. Moreover, we show you, how you can configure it for use in your individual context.

Pipeline Stages

The following screenshot shows the stages of the SAP Cloud SDK Pipeline. The text below briefly introduces each stage and its tasks.

• Declarative Checkout SCM: In this stage, your project's source files are retrieved from the configured SCM URL.


• Init: The init stage initializes the SAP Cloud SDK Pipeline by, for example, loading your individual pipeline configuration.


• Build: Parallelized build of frontend and backend components 
  
  
  
  • Backend: The "Build Backend" sub-stage executes the maven goal install in the root of your project as defined in your pom.xml. It will not compile or run any tests.
  
  
  • Frontend: The "Build Frontend" sub-stage executes npm install in the root of your project. This will download frontend dependencies and test tools as defined in your project.json.
  
  
  
  


• Local Tests: Parallelized tests of frontend and backend components.
  
  
  
  • Backend Unit Tests: Executes the maven goal test in the unit-test module of your project and collects code coverage information.
  
  
  • Backend Integration Tests: Executes the maven goal test in the integration-test module of your project and collects code coverage information.
  
  
  • Frontend Unit Tests: Executes the npm script ci-test from your project's package.json.
  
  
  • Static Code Checks: Executes a set of static code checks against your codebase and adds the results to your Jenkins build. The code checks are a composition of custom and sap-specific rules.
  
  
  
  


• Remote Tests: Parallelized tests against deployments of your project.
  
  
  
  • End to End Test: Executes the npm script ci-e2e against a test deployment of your application and collects the results.
  
  
  • Performance Tests: Executes your JMeter Test Suite and collects the results.
  
  
  
  


• Quality Checks: Collects report data of previous stages and applies SAP-specific quality checks.


• Production Deployment: Deploys your application to your SAP Cloud Platform targets.

Design Principles

This section introduces the design principles that shaped the design of the Cloud SDK Pipeline.

Pipeline as Code

The SAP Cloud SDK Pipeline is distributed as code. This code describes the orchestration of build steps and is available in form of an Open Source project on SAP's GitHub repository. From there, every SAP Cloud SDK project will by default retrieve the most recent pipeline code. This enables your project to rapidly leverage future improvements of the SAP Cloud SDK pipeline.

Build logic in Global Pipeline Library

The actual logic of each pipeline stage is externalized to a reusable Jenkins Pipeline Library which can also be found on GitHub. The SAP Cloud SDK Cx Server is already configured to make the library available to pipeline-builds running inside it.

Strict isolation between build runs

Unexpected side effects between builds are a major problem in many of today's CI landscapes. Therefore, we decided to conduct each SAP Cloud SDK Pipeline build in strict isolation. None of the build tools are operated in a way that allows side effects between multiple builds. In consequence, no persistent local repositories are used for maven or npm.

Note: Today's software projects typically contain dozens of dependencies and downloading them for each build can notably increase build time. To address this challenge, it is planned to augment the SAP Cloud SDK Cx Server with a download cache, which will locally store dependencies of your project and significantly increase the speed of retrieving them during a build.

Proper isolation of individual build steps

To foster modularity and a clear separation of concerns, the Cx Server does not deliver any specific build tools. All dependencies are externalized to specific docker images in which the actual work is executed. These containers are instantiated and subsequently disposed for each build step. This further assures that no unintended state is leaked between build steps or other build jobs. Relevant data is explicitly maintained in Jenkins stashes. There is no implicit data passing between stages via workspace artifacts. Each stage only stashes artifacts which are relevant for follow-up stages. Furthermore, each stage only unstashes files of relevance for its own execution.

Parallelization

To reduce the total build time, the SAP Cloud SDK Pipeline makes strong use of parallel build stages.

Selective Stashing

Each stage is structured according the following high level workflow:

1. Unstash relevant files


2. Do work


3. Stash relevant files

To avoid communication overhead and unnecessary parallelization bottlenecks only files that are needed per stage are unstashed. Furthermore, after the stage completed its work, only relevant results are written back to the build master.

Checked Qualities

To assist you in achieving high quality within your application, theSAP Cloud SDK Pipeline applies a set of quality rules to your project. At the time of writing this, the following checks are applied.

Exclusive use of whitelisted APIs

The pipeline is able to detect the use of non-whitelisted APIs which are prohibited for side-by-side extension applications.

Resilient invocation of 3rd party services

For the resilient invocation of 3rd party services, we recommend to use Hystrix, which implements Circuit Breaker and Bulkhead semantics. The pipeline will inform you about non-resilient invocations of downstream services that might cause cascading errors in the production deployment of your application.

Proper error handling

Proper error handling is crucial for rapidly diagnosing and fixing a malfunctioning application. The pipeline is able to warn you about unsafe ways of error handling and will help you to establish the prerequisites of good post-mortem-analyzability.

Minimum code coverage

Tests are important to assure the functional correctness of your application. If your tests only cover a small fraction of your code, the pipeline will indicate this.

Platform abstractions

The SAP Cloud SDK offers abstractions for many cloud environment specific APIs. This allows you to easily migrate your application between cloud environments, for example, from SAP SCP Neo to SAP SCP CloudFoundry. To assure that you are not unnecessarily binding against specific APIs, the pipeline will inform you about places in your code that should use corresponding SAP Cloud SDK abstractions.

Troubleshooting and Questions

If you are interested in more information on the integrated quality checks, visit our detailed article on quality checks and their troubleshooting.

Are you facing a development question? Then check out StackOverflow, the main place for SAP  Cloud SDK related questions. If you do not find an answer, feel free to post it and make sure to attach the tag ‘s4sdk’. Our team, as well as the whole StackOverflow community, are at your service and will quickly react to your question.

For an overview of SAP Cloud SDK related questions, go to https://stackoverflow.com/questions/tagged/s4sdk.

You think that you found a bug in one of our Continuous Delivery artifacts? Feel free to open an issue in our Pipeline GitHub repository on https://github.com/SAP/cloud-s4-sdk-pipeline/issues.