I think CPIlint from 7a519509aed84a2c9e6f627841825b5a is a great tool that enables users to validate their SAP Cloud Integration/CPI iFlows. But how do you run it on only the iFlows that changes
I have had for a long time tried to find a way to embed it into our build process, so users could become aware of iFlows that did not follow the standards.
We could not find a good way to embed it within the Figaf DevOps flow for handling transport. It would be nice to get validation that an iFlows is correct before transport.
In the Figaf DevOps Tool for SAP Cloud Integration (or find us in SAP Store), we synchronize all changes into a Git repository. The synchronization can easily be scheduled each hour. The main idea is to make it easier for developers to work on their complex CPI iFlows. I wanted to explore GitHub Actions and see if they should help add some custom validation to the process. It was quite interesting to see the result of what is possible.
In the following video you can see more about the process
Git repository
The structure we have in our repository is package name->iFlow name and then we have the extracted payload there. This makes it a lot easier for you as a user to work on your iFlows. You can see an example of how the repository looks here.
We have created some open source Gradle plugins that allow you to upload/download/deploy iFlow artifacts directly from your favorite IDE. So you can work much faster on your Groovy scripts and then see how they work in your iFlow. We also added support for Script Collections and Message Mappings artifacts recently (I'll need another blog on that topic later).
CPIlint
CPIlint is a tool designed for running in a command prompt to validate the compliance of your iFlows. You can create rules to perform validation if they comply with your governance.
Normally you can schedule CPILint to run each night on your system, but that could mean you would not get potential problems until the next day.
One problem with using CPIlint in our case is that it works on iFlows that are zipped. So we need to zip the iFlows to zip files and then run lint on the folder of the Zip file.
Github Actions
Actions are something that can be triggered each time someone performs an action on a repository. Other platforms have similar options that can be triggered in the same way. So the idea can easily be used for Azure DevOps or Bitbucket.
Each time a commit is pushed then the script is triggered.
Here you see our script for handling the processing.
# This workflow will build a Java project with Ant
# For more information see: https://help.github.com/actions/language-and-framework-guides/building-and-testing-java-with-ant.
name: CPILint
on:
push:
branches: [ irtsync ]
pull_request:
branches: [ irtsync ]
jobs:
build:
if: "! contains(github.event.head_commit.message, 'skip:tests')"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Tests
uses: actions/setup-java@v3
with:
java-version: '11'
distribution: 'temurin'
- name: Git Branch
run: git pull
- name: java home
run: echo $JAVA_HOME
- name: current directory
run: pwd
- name: current directory content
run: ls -lrt
- name: create artifacts folder
run: mkdir -p artifacts
- name: find iflow changes
run: |
chmod +x ./.github/workflows/zipiflows.sh
./.github/workflows/zipiflows.sh
- name: List the iFlows going to tested.
run: ls -lah artifacts/
- name: get the cplink test script test cases
run: |
wget https://github.com/mwittrock/cpilint/releases/download/v1.0.4/cpilint-1.0.4.zip
unzip cpilint-1.0.4.zip
chmod +x ./cpilint-1.0.4/bin/cpilint
- name: Executing tests on iFlow
run: ./cpilint-1.0.4/bin/cpilint -rules cpilint.xml -directory artifacts/
To handle the zipping of iFlows we have created the following flow. This will look at the commit changes and see which folders with MANIFEST have been changed.
The script then traverses thru each changed and checks if the folder contains an iFlow and if it does then zips the content to an artifacts folder.
You can see the bash script here
#!/bin/bash
#git checkout irtsync
git diff --name-only --relative --diff-filter AMR HEAD^ HEAD ./ | grep MANIFEST.MF | awk -F "/" '{print $1,$2}' > ../filetoProcess
cat ../filetoProcess
echo "before cycle:"
cat ../filetoProcess | while read line
do
# do something with $line here
echo "line $line"
ldirectory=$(echo $line | awk -F " " '{print $1 "'/'" $2}')
echo "ldirectory $ldirectory"
liflow=$(echo $line | awk -F " " '{print $2}')
echo "liflow $liflow"
cd $ldirectory/ ;
echo "current $ldirectory:"
ls -lrt
if grep IntegrationFlow META-INF/MANIFEST.MF ; then
zip -r ../../artifacts/$liflow.zip .
fi
cd ../../
done
Then CPILint will be downloaded and extracted. This is probably not the most ideal way to handle the download.
Then CPIlint is run on the folder with a rules file defined in the repository.
If there is any problem build fails and you can see what the problem is.
You will also get information in your email for each action. You then need to check what the error message is.
Improvements
There are some ways to improve the validations.
Inform the user that created/updated the iFlow and what the problem is
Have exceptions for some iFlows that should not be checked because they are allowed to break some of the rules.
Update CPILint to also work with extracted iFlow objects
Include a Groovy code analyzer for problems.
Conclusion
I hope you enjoyed this and you can see some ways you can bundle CPIlint in your development process. I did learn a lot about creating the Actions, they are now way perfect so please let me know how I can improve them.
You will be able to use the script without using Figaf and just leverage our open-source plugins. I do hope you want to see how Figaf can simplify a lot of your SAP integration like testing, transport, and monitoring of SAP Cloud Integration.
- SAP Managed Tags:
3 Comments
