Technical Articles
CI / CD for SAPUI5 on SCP NEO with GitLab
Hi all,
I posted a blog earlier about using Gitlab CI for testing, building and deploying UI5 apps to an ABAP system: https://blogs.sap.com/2018/08/01/ci-cd-for-sapui5-on-abap-with-gitlab/ . Now, we had a use case where we wanted a similar setup, but we needed to deploy our app to SCP NEO instead of an ABAP system. Besides the target system there is another difference, this time I’m using a windows server for my own GitLab Runner.
Deploying via a Gitlab runner to ABAP or SCP is a significant difference. For configuring this setup, I started from the best-practice guide of SAP: https://developers.sap.com/belgie/tutorials/ci-best-practices-fiori-sapcp.html . In this tutorial, SAP describes how you have to configure Jenkins for deploying to SCP. This is very similar for GitLab as both systems just run OS scripts on your server. The most import part of this tutorial is the script for deploying to SCP:
# install the MTA archive builder
mkdir -p ${WORKSPACE}/tmp/mta
cd ${WORKSPACE}/tmp/mta
wget --output-document=mta.jar '<URL from where to download the MTA archive builder>'
# install neo command line client
mkdir -p ${WORKSPACE}/tmp/neo-java-web-sdk
cd ${WORKSPACE}/tmp/neo-java-web-sdk
wget 'http://central.maven.org/maven2/com/sap/cloud/neo-java-web-sdk/1.127.11/neo-java-web-sdk-1.127.11.zip'
unzip -o neo-java-web-sdk-1.127.11.zip
rm neo-java-web-sdk-1.127.11.zip
# create local npmrc file
cd ${WORKSPACE}/src
cat <<EOF > .npmrc
registry=https://registry.npmjs.org/
@sap:registry=https://npm.sap.com/
EOF
# extract artifact name
cd ${WORKSPACE}/src
mtaName=`awk -F: '$1 ~ /^ID/ { gsub(/\s/,"", $2)
gsub(/\"/,"", $2)
print $2 }' mta.yaml`
# replace timestamp placeholder
sed -ie "s/\${timestamp}/`date +%Y%m%d%H%M%S`/g" mta.yaml
# execute MTA build
java -jar ${WORKSPACE}/tmp/mta/mta.jar --mtar ${mtaName}.mtar --build-target=NEO build
# deploy to SAP Cloud Platform
${WORKSPACE}/tmp/neo-java-web-sdk/tools/neo.sh deploy-mta --user ${CI_DEPLOY_USER} --host ${DEPLOY_HOST} --source ${mtaName}.mtar --account ${CI_DEPLOY_ACCOUNT} --password ${CI_DEPLOY_PASSWORD} --synchronous
I just had to convert the script to powershell (from the best-practice guide of SAP) because I’m working on a windows server. Besides converting the script, I also installed the required tools on my server. Otherwise the GitLab Runner would have to download it every time we want to deploy. As I have access to the server, I can just install the required tools. This will also speed up the build and deployment process.
Prepare the GitLab runner
Prepare the gitlab runner by installing the tools on the runner.
Download the following tools:
- MTA builder:
- NEO SDK:
I copied both tools to a newly created “tools” folder in the GitLab Runner folder on my windows server:
Here I have my mta builder jar file and all the NEO files for the deployment:
The NEO CLI which we use for the deployment is part of the NEO SDK which you can download here: https://tools.hana.ondemand.com/#cloud
The MTA builder can also be downloaded at the bottom of the same page.
Next to that, I’ve installed npm on the server.
By installing the tools on the runner upfront, will speed up the CI process and simplify the CI script.
Prepare the UI5 app
Next to the tools, we also need to prepare our project for the CI process. The CI process will deploy our project as an MTAR file on SCP. For this, we need to add a mta.yml file to our project. The mta.yml file has all the required configuration that the MTA builder needs to build and generate the MTAR file.
The mta.yml file is very basic for a UI5 app only and looks like this:
_schema-version: "2.0.0"
ID: "<appid>"
version: 1.0.1
parameters:
hcp-deployer-version: "1.0.0"
modules:
- name: "<appname>"
type: html5
path: .
parameters:
version: 1.0.1-${timestamp}
build-parameters:
builder: grunt
build-result: dist
The file can have different parameters depending on the target platform. Our target platform is NEO.
We just added this yml file in our main UI5 project. This means that we can use the path “.” for the UI5 app.
The “${timestamp}” will be replaced by the CI process to always have a unique version of the app.
Configure the CI/CD Process
We need to configure the CI/CD process and therefore we need to add and configure the GitLab CI yml “.gitlab-ci.yml” file to our UI5 project. Compared to my blog post about CI/CD for UI5 on an ABAP system, the steps are very similar: https://blogs.sap.com/2018/08/01/ci-cd-for-sapui5-on-abap-with-gitlab/ . The steps are similar but the technology behind is different. I replaced the build grunt task with the MTA build. The deployment grunt task is replaced with the NEO CLI. The CI/CD script for the MTA build and NEO CLI is based on the script from the SAP best practices guide. I just made it easier by downloading the required tools upfront.
I also left out the step for unit testing, all the others stay the same like in my previous blog:
- Initialization
- This will load the required npm modules and store them for the other steps
- For storing these npm modules, I’m using the syntax “artifacts”
- Linting tests
- This will check for errors in the project, it will stop the flow in case of any error
- It’s based on the same linting that’s being used in the SAP Web IDE
- Calls grunt task lint. Normally this task doesn’t stop in case of errors. I wrapped a task around it to check the result of the lint task. If I find an error in this result of the lint task, it will stop the process
- Build
- This time the build step won’t be a grunt task, it will trigger the mta builder which will run the grunt build task on its turn. The MTA builder will do more than only build the app, it will also wrap it into an mtar file.
- The mta builder triggers the grunt build task because this is defined in the mta.yml file in the build parameters “builder: grunt”. It will trigger the default task which is configured to be the build task.
- Before we run the MTA builder, we get the current timestamp and store it as a variable in a temporary file. We use the timestamp as part of the name for the generated mtar file. This file will be deployed to SCP and that’s why we need to store it.
- The MTA builder requires the target platform, we use NEO for this but could also be CF.
- Deploy
- This step is completely grunt free ? It’s using the NEO CLI for deploying the mtar file to SCP NEO.
- It will first read the timestamp from the stored location.
- Trigger the deployment with the correct parameters
- The deploy command requires your SCP user, password, host (eu, us, .. ) and account.
Here you have an example of the full GitLab CI yml file:
image: node:latest
stages:
- init
- validation
- build
- deploy
before_script:
- echo $env:CI_BUILD_REPO
- echo $env:CI_BUILD_NAME
- echo $env:CI_PIPELINE_ID
- echo $env:CI_PIPELINE_IID
- echo $env:CI_COMMIT_REF_SLUG
- echo $env:CI_PROJECT_PATH
- echo $env:CI_PROJECT_DIR
- whoami
init-ci:
stage: init
script:
- npm install
- grunt --verbose clean
artifacts:
paths:
- node_modules/
code-validation:
stage: validation
script:
- grunt -d -v fiori-test
dependencies:
- init-ci
build-app:
stage: build
script:
- $date = Get-date -UFormat '%Y%m%d%H%M%S'
- echo $date
- if (Test-Path build\variables) { Remove-Item build\variables}
- New-Item -ItemType Directory -Force -Path build
- New-Item build\variables -ItemType file
- echo $date >> build\variables
- (Get-Content mta.yaml).replace('${timestamp}', $date) | Set-Content mta.yaml
- java -jar C:\gitlabrunner\tools\mta.jar --mtar build\tmp-$($date).mtar --build-target=NEO build
artifacts:
paths:
- build/
deploy-scp:
stage: deploy
script:
- $date = Get-Content build\variables | Out-String
- $date = $date.Trim()
- echo $date
- C:\gitlabrunner\tools\neo.bat deploy-mta --user %SCP_USER% --host %SCP_HOST% --source build\tmp-$($date).mtar --account %SCP_ACCOUNT% --password %SCP_PWD% --synchronous
dependencies:
- build-app
only:
- master
Grunt
The grunt script contains the tasks that we need for the code validation and build steps. The build task is defined as the default task and will be used by the MTA builder.
It’s basically the same grunt script as it’s been generated by the SAP Web IDE with the SAPUI5 best practice task. I only added a task that will check the eslint result to stop the build in case of eslint errors.
Here you have the grunt script:
/* global process:true */
"use strict";
module.exports = function (grunt) {
// Variables from environment
// Project properties
var webAppDir = "webapp";
var targetDir = "dist";
// Project configuration.
// grunt.initConfig({
var config = {
eslint: {
options: {
configFile: ".eslintrc.js"
},
target: [webAppDir + "/**/*.js"]
}
}; //);
grunt.loadNpmTasks("grunt-eslint");
grunt.loadNpmTasks("@sap/grunt-sapui5-bestpractice-build");
grunt.loadNpmTasks('grunt-openui5');
grunt.config.merge(config);
grunt.registerTask("check-lint", "Check validation", function () {
var validation = grunt.file.readJSON(targetDir + "/di.code-validation.core_issues.json"),
hasErrors = false;
for (var check in validation.results) {
grunt.log.writeln("Result for: " + check);
for (var file in validation.results[check].issues) {
validation.results[check].issues[file].forEach(function (error) {
if (error.severity === "error") {
grunt.log.error(error.path + "(" + error.line + "," + error.column + ") : " + error.message).error();
hasErrors = true;
}
});
}
}
if (hasErrors) {
grunt.fail.warn('Errors found during code validation');
}
});
grunt.registerTask("fiori-test", ["lint", "check-lint"]);
grunt.registerTask("buildapp", ["build"]);
grunt.registerTask("default", ["build"]);
};
Config
The ESLint config and CI/CD Global variables can be configured the same way like in my other CI/CD blog: https://blogs.sap.com/2018/08/01/ci-cd-for-sapui5-on-abap-with-gitlab/
CI/CD Global variables are just a bit different:
Recap
You should have added at least the “mta.yaml” file and “.gitlab-ci.yml” to your project (marked in red). If you also want to use the eslint check in your pipline, you also need to add the eslint config and modify the gruntfile (marked in orange).
Result
Deploying the app as an MTAR file means that it won’t be deployed the same way as you’re used to with UI5 apps. Instead of a new HTML5 app, it will be deployed as a solution.
If the CI/CD Process goes well, you see the following pipeline in GitLab:
When all steps are finished, you can find the app in SCP.
Go to solutions, the UI5 app will show up when the deployment is finished:
And that’s how you can use GitLab for your CI/CD setup in SCP!
Enjoy!
Great stuff! I've been wanting to look into the city setup needed for scp for quite a while. Just didn't have a reason for it yet as my client is using on prem Fiori.
I've added an auto readme generator that takes the jsdoc from all is files and summarise in the readme. Maybe that's worth a blog on its own.
Thanks for laying out the steps for us.
great blog. ist there any way to get the login token in the ci to run test with backend oData?
You could use the credentials of the technical user (that's being used for the deployment) to test your service..
Kr, Wouter
Hi Wouter,
I am a newbie in CI/CD, could you help me in understanding the external_url which we need to change while installing GitLab?
While searching, it looks it should be a kind of reverse proxy that will be used to access Installed GitLab, But it’s not clearly mentioned anywhere.
Thanks,
Manish
The setup of Gitlab is mentioned in an earlier blogpost: https://blogs.sap.com/2018/08/01/ci-cd-for-sapui5-on-abap-with-gitlab/ -> section setup
Those are the only steps I had to follow.
Kr, Wouter
Thanks for your post.
After our deployment, the SCP HTML5 cache isn't cleared, so we have to do it manually every time.
Do you have any suggestion?
Thanks for posting such a instructive blog !
I configured all the important elements for my CI/CD on a folder.
Which part is going to trigger the process of devOps?
Should I commit all thoses files into my Gitlab instance on google cloud?
Thank you so much for sharing this information as POC. It really helped me a lot. But Is it possible to shed more details on integrating Gitlab with SAP Commerce Cloud based on Azure for Hybris (CCV2). Because I would like to clarify some of the doubts since you implemented build and deployment automation already
I know I posted so many doubts relevant to CCV2 but it would be nice to know for everyone if we get clarified it for better understanding
Thank you