Skip to Content
Technical Articles

Project “Piper” – Jenkins on Docker

This post forms a part of our overall series SAP Business Technology Platform – Software Lifecycle with elements of Continuous Integration, Delivery and Transport Management. If you’d like to follow the overall use case, the Overview post provides a good starting point.

This series and the associated blog posts are a collaboration between Jacek Klatt and Cameron Swift on behalf of the SAP HANA Database & Analytics, Cross Solution Management team. You will find a video walkthrough at the bottom of the blog post

 

Introduction

 

To support elements of Continuous Delivery across our scenario, we want to set up a tool to help us configure pipelines.

SAP maintains Project “Piper” to simplify the setting up of Continuous Delivery. Our end to end scenario uses the Project Piper Jenkins Image. At the time of writing, Jenkins is the only tool available. However, Project Piper is being revamped with the aim of making it accessible with the tool of your choice, so this is worth keeping an eye on as more tools become available over time

Jenkins allows us to create pipelines for automating our build, testing and release process. In our scenario, we’ll be committing changes to GitHub. These commits will then trigger the testing and transport of our artifacts between the Development and Production environments of our simplified landscape

 

Preparing Our Docker Image

Before we deploy our Jenkins container, we’ll need to make a change to the provided container image. This is because in order to automate the migration of artifacts for SAP Data Intelligence, we’ll need to ensure the container image also has one important tool – the SAP Data Intelligence System Management Command-Line Client (also known as the helpfully shorter vctl)

At the time of writing, the most recent version is 1.0. Before you prepare your own image, you should check the SAP Software Center for the most up-to-date version (you will find the tool under the name DATA INTELLIGENCE-SYS MGMT CLI)

Before we continue with our next steps, we have two prerequisites to complete. Our first step is to install Docker on your machine to allow us to create our container images. The next step is to create an account on DockerHub, an online repository for container images – those with privacy concerns should investigate DockerHub Organizations, this is out of scope for this scenario

Docker allows us to use something called a Dockerfile to build our final container images. While these Dockerfiles can become quite involved, the instructions required for our image are reasonably straightforward

# Uses the DockerHub image ppiper/jenkins-master as a base
FROM ppiper/jenkins-master

# Copy SAP Data Intelligence command line tool (vctl) to the image and set 
# PATH variable to expose it
RUN mkdir $HOME/vctl
RUN chmod 777 $HOME/vctl
COPY vctl $HOME/vctl/vctl
ENV PATH="$HOME/vctl:${PATH}"

 

The above code should be put into a file named Dockerfile (no extension). This, when placed in the same folder as the vctl tool we provided earlier will give us everything we need to build our Jenkins container image

The first line of code sets the “base image”, the starting point for our container. The line refers to the most recent version of the container image located at ppiper/jenkins-master on DockerHub. From this base image, we then have the necessary code to integrate the vctl tool

 

Building our Docker Image

Before we build the container image and push it to Dockerhub, please note that we performed these actions on a Windows machine using either Command Prompt or Windows Powershell. For other operating systems including considerations around the need for sudo please consult the official documentation

First we want to navigate to the folder which contains our prepared Dockerfile and the vctl tool

Next, we can build the image locally using a single command. Two things are important to note: first, the image name we choose will be the final name of the image in DockerHub. Secondly, the full stop is important – please don’t forget it.

This process may take a while – please use this time as an opportunity to grab a coffee, catch up on emails or enjoy some sunlight

docker build -t <dockerhub username>/<image name> .

After we build our Docker Image, we can view it in the Docker Desktop application

Our%20Jenkins%20Image%20in%20Docker%20Desktop

Our Jenkins Image in Docker Desktop

Before we upload our Container Image to DockerHub, we want to test it locally first. Run your Container Image, assigning port 8080 in the optional settings so that you can test for the Welcome Page at localhost:8080

If%20you%20can%20access%20the%20Welcome%20Page%2C%20then%20we%20should%20be%20ready%20to%20go

If you can access the Welcome Page, then we should be ready to go

 

Uploading to DockerHub

Once we’ve verified our image locally, we want to push the image to DockerHub. This means that the exact container we’ve just created will be backed up online, and can be pulled down from there to be used in our setup.

First, we will need to use our login credentials. Using the below command and then providing your username and password will store our credentials for Docker to use when pushing images

docker login

 

Docker will use your stored credentials to push the container image in a single command.

docker push <dockerhub username>/<image name>

 

Once your container image has been successfully pushed to DockerHub, we’re almost ready to start building our pipelines.

 

Hosting Your Docker Container

For our scenario, we’re hosting our setup on a VM on Azure. However, the question of where to host your own setup is important, and you will want to decide for yourself on a case-by-case basis.

At the time of writing, Windows is not supported for productive use. The minimum recommendations for hosting are as below:

  • OS: Ubuntu 16.04.4 LTS
  • Docker: 18.06.1-ce
  • Memory: 4GB reserved for Docker
  • Available Disk Space: 4GB

On your VM, you will want to run the following command to generate cx-server and server.cfg, both of which will be helpful for your environment.

docker run -it --rm -u $(id -u):$(id -g) -v "${PWD}":/cx-server/mount/ ppiper/cx-server-companion:latest init-cx-server

 

Once this has run, you’ll want to edit the server.cfg file to point to our customized Docker Image.

Under the heading Build Server Configuration, we need to uncomment the docker_image line, and use it to point towards our custom docker image as below

 

 

Save the server.cfg file, then it’s time to start our server.

 

Initializing our Server

Once we’ve pointed the config towards our custom image, it’s time to start our server

chmod +x ./cx-server
./cx-server start

The server will be started using our custom image, and initial admin credentials will be generated. We’ll need to retrieve the initial credentials

./cx-server initial-credentials

We should now be able to access the Jenkins Dashboard through Port 80 of your host (this can be changed in server.cfg).

Jenkins%20Dashboard%20Running

We can reach the Jenkins Dashboard

 

As a troubleshooting tip here, we can run the following command to get information on which images are running, which ports are in use and the container ids

./cx-server status

 

Before we’re ready to use our Jenkins instance, we want to log on with the admin credentials we retrieved earlier and change the initial admin password. We can do this through Manage Jenkins->Manage Users->admin->Configure

We will also need to install one plugin to ensure that our pipelines can run correctly. We can do this by visiting Manage Jenkins->Manage Plugins->Available then navigating to the Workspace Cleanup Plugin. We’ll tick the check box to the left, and then click Download now and install after restart

This plugin can be called to clean workspaces, handy for clearing files before a run

Wrap Up

Throughout this blog we’ve covered a lot of ground. First, we created a Jenkins Container Image capable of connecting to SAP Data Intelligence using the vctl tool

We then stored our image on DockerHub. Finally, have used the Project Piper cx-server utility to run our custom Container Image on a VM

I hope that you’ve found this blog helpful, and I welcome your comments and questions below. To get an understanding of our end to end scenario, you can view our Overview blog

 

Video Walkthrough

1 Comment
You must be Logged on to comment or reply to a post.
  • Hi Cameron,

    I tried to set up the Project Piper as described. But when I come to the './cx-server start' command, I am facing an issue. Containter is failing to start and when I am going threw the logs I found an error telling me 'Error while retrieving list of tags from https://registry.hub.docker.com/v2/repositories/<username>/ <imagename>/tags'.
    When I visit the website manually, I first got an error but when I log in I can see the needed information. So I think this is an authorization problem.

    I did not find any solution on the internet by now, so maybe you can help me.
    Thanks for the effort.

    Kind regards
    Simon