In the on-premise ABAP world, lifecycle and quality processes can be controlled by the ABAP systems themselves. Think of transport routes and jobs to execute different checks. In the SAP BTP ABAP Environment aka Steampunk, this is no longer possible. Steampunk systems are independent of each other, and source code is “transported” with git-based solutions: gCTS and abapGit. While different systems in one global account are connected to the same repositories (read: “software components”), they do not know of each other. Due to this, the driver of lifecycle and quality processes needs to shift from the ABAP system itself to an external driver – for example: a classic Continuous Integration / Continuous Delivery (CI/CD) server like Jenkins. Of course, the Steampunk systems need to expose the functionality via APIs so that a CI/CD server can control those processes.

Components of Jenkins driven Steampunk CI/CD processes

In the next sections, I want to introduce the released APIs regarding gCTS and ABAP Test Cockpit (ATC), as well as all the functionality we built on top of these APIs to make your life easier: reusable functions using these endpoints and a predefined pipeline which lets you easily create CI/CD processes for Steampunk.

To make the consumption of our pipeline and functions (or “steps” in the context of a pipeline) as convenient as possible, we decided to contribute them to the Project Piper. This open-source project is the home of different steps and pipelines related to the SAP Ecosystem.

Let’s start with the easiest way to create a pipeline and drill our way down to the APIs.

ABAP Environment Pipeline

The “ABAP Environment Pipeline” is a predefined pipeline usable on a Jenkins server. It covers most use cases and is ready to use: no programming is required – only the configuration of the pipeline is necessary. The pipeline consists of several stages. A stage groups a set of steps together, which are related in content. One stage, for example, is called “Prepare System”. As the name indicates, this stage takes care of preparing a system that is usable within the pipeline. It includes creating a Steampunk system and configuring the communication.

Layers of a predefined Jenkins Pipeline

The ABAP Environment Pipeline supports various use cases. A simple scenario that can be realized with this predefined pipeline is running nightly ATC checks on the released source code. (Of course, you will benefit most from such a pipeline if you release your Transport Requests early and often.) This pipeline would consist of the following stages:

• Prepare System: This stage creates the Steampunk system and configures the communication to the system
• Clone Repositories: This stage clones your repositories (software components) to the system
• ATC: This stage executes ATC checks on the configured repositories
• Post: This stage is automatically executed in the end and will delete the Steampunk system

Diagram of the ABAP Environment Pipeline. The Post stage is not displayed. Stages that are not configured are skipped.

While the “ABAP Environment Pipeline” itself is predefined, you still have the option to customize it via configuration and extension. In principle, a stage of this pipeline will only be executed if it is configured. This means that, if I do not want to create a new Steampunk instance for every pipeline execution, I just do not configure the “Prepare System” stage. In case a specific stage does not quite fit my needs, I can create an extension file for this stage and add missing functionality (or even rewrite the whole stage). A popular example is to extend the ATC stage with the “CheckStyle” Plugin, which creates helpful diagrams out of the XML output of the ATC API.

Example set of CheckStyle diagrams

As you can imagine, there is a lot of room for creativity. You can head to the documentation to learn more about the pipeline or check out some examples in this repository.

If this pipeline does not quite fit your needs, you still have the option to reuse our steps to create your own pipeline. Let’s have a look in the next section.

Pipeline Steps

I want to start this section by highlighting the pipeline steps’ design pattern in Project Piper. Some years ago, Piper’s focus was on the CI server Jenkins: The pipeline steps were written in the language “Groovy” and could only run on this technology. Nowadays, while Jenkins is still widespread, other technologies are gaining popularity. Therefore, the decision was made to switch the implementation of pipeline steps over to “Go” (or “Golang”). This language compiles the source code to a binary, which can be executed on different CI servers. Here is an example of how the “clone” step can be called via the piper binary:

abapEnvironmentCloneGitRepo --host host.com --username myUser --password myPassword --repositoryName  “/DMO/GIT_REPOSITORY” --branchName master

Of course, for users of a Jenkins CI server, a Jenkins step implementation is built on top of the Golang implementation. So, the full picture is as follows: several WebAPI endpoints of Steampunk are called by the Golang implementation and the resulting binary is then used by the Jenkins step.

Layers of a Piper step

Before looking at the WebAPI, let me show you an example of a Jenkinsfile defining a custom pipeline and reusing the Jenkins step “abapEnvironmentCloneGitRepo”.

@Library(['piper- lib-os']) _
	
pipeline {
  agent any
  stages{
    stage("Pull Git Repository"){
      steps {
        abapEnvironmentCloneGitRepo(script: this) 
      }
    }
  }
}

The annotation “@Library([‘piper-lib-os’]) _” is a reference to the Project Piper jenkins-library (https://github.com/SAP/jenkins-library/), which would have been configured on the Jenkins server with the ID ‘piper-lib-os’. The reused step is part of a pipeline stage and simply called with the statement “abapEnvironmentCloneGitRepo(script: this)”. The parameters for this step (e.g. endpoints, credential IDs etc.) would be provided in a separate configuration file.

The documentation of our example step for the action “Clone” can be found here. Other available steps can then be found in the documentation menu under “Library steps”.

Now that you know how to use the steps in Jenkins or via the Golang binary, what are they actually doing? Let’s have a look at our example step: the action of cloning a repository. At first, the step authenticates on the system (and fetches the X-Csrf-Token, which is required for a POST request). After triggering the “clone” action, the step polls the status of the action to find out, when the repository is finally cloned successfully. As you can see, the steps implement a simple series of API requests to achieve a single task (“clone a git repository”).

Of course, if you really want to control every detail of your pipeline (or want to run it on a CI server that is not able to support the Golang binary), you can use the released APIs for gCTS and ABAP Test Cockpit directly. Let’s have a look at them in the next section.

APIs

The goal of the gCTS API is to enable you to automate the most crucial functions that you may know from the “Manage Software Components” app: cloning a repository, pulling a repository, and checking out a branch. The service, including those endpoints, is documented here.

For example, the following request would clone the software component “/DMO/GIT_REPOSITORY” to the system, with the branch “master” being checked out.

POST/sap/opu/odata/sap/MANAGE_GIT_REPOSITORY/Clones HTTP/1.1
Content-Type: application/json
Accept: application/json

{
        "sc_name": "/DMO/GIT_REPOSITORY"
        "branch_name": "master"
}

A second service would then allow you to execute a check run with the ABAP Test Cockpit (ATC). You can specify the check variant and the software component or package to be checked in the request body. Here is an example:

POST https://<host.com>:<port>/sap/bc/adt/api/atc/runs?clientWait=false
Content-Type: application/vnd.sap.atc.run.parameters.v1+xml

<?xml version="1.0" encoding="UTF-8"?>
<atc:runparameters xmlns:atc="http://www.sap.com/adt/atc" xmlns:obj="http://www.sap.com/adt/objectset">
  <obj:objectSet>
      <obj:softwarecomponents>      
        <obj:softwarecomponent value="/DMO/GIT_REPOSITORY"/>
      </obj:softwarecomponents>
   </obj:objectSet>
</atc:runparameters>

The full documentation can be found here.

What’s next?

In the future, the ABAP Environment Pipeline will support the build process of ABAP Add-ons for Steampunk. This scenario is intended for SAP partners who want to build Software as a Service (SaaS) solutions for the SAP BTP. Some stages and steps are already contributed and part of the ABAP Environment Pipeline. While this is currently our focus, we are still continuously improving the existing pipeline and steps, and we are hoping to support more tools in the future.

Please let me know if you have feedback or questions and feel free to post in the comment section below (or head to the Answer Hub for general questions).

Notice: images are my own.