| Part 2 of this series sheds light on automated unit testing and code coverage with continous integration. |
Dear community,
I got you covered with some nice integration content today. We will connect what was meant to be connected for a long time. Oh boy, hopefully this will not end up being a cheesy love story 😄 I will do my very best.
I know you always wanted to consume some nice Microsoft Teams chats, load bunches of Outlook E-Mails, peak into calendars or stalk some user profiles on M365 with your applications on SAP Cloud Platform (SCP)?. However, all of that is already covered with SAP Open Connectors and the Microsoft Graph in a managed way.
What about LogicApps or Microsoft Flow, though?
Possible integration scenarios range from business related events like purchase orders to administrative actions like approvals or exception handling in integration flows. Since there is nothing much in terms of DevOps for SAP CPI, I created a prototype to showcase a modern approach for your iFlows. Excited yet?
I chose LogicApps over Microsoft Flow due to the technical nature of my integration project. Microsoft Flow is more targeted at business processes rather than technical integrations.
Fig.1 Integration process overview
Imagine a problem in your precious synchronization flow, that harmonizes your CRM with S4. In case you are using an agile approach like SCRUM with Azure Boards or Jira it would be nice to create an issue in your backlog automatically, right? That way you can keep track of the problem and associate the fix on your version control system (e.g. GitHub) and track the delivery through the stages (dev/test to prod).
In addition to that, it would be great to have online links from your Azure Boards Issue to the CPI log entry and vice versa to quickly identify the involved sources.
Finally, you would want to release the fix automatically from your IDE and deploy the changed iFlow.
That is quite something. Let’s walk through it step-by-step.
The moving parts to make it happen
Every iFlow should have an exception sub-process, to handle possible errors. To start using my prototype you need to add a “content modifier” and a “process direct” call to your existing iFlows. ProcessDirect makes sure the message is sent internally within CPI with no overhead. Down below is an example that throws an error due to a mal-configured http target URL with the described modification of the exception subprocess.
Fig.2 iFlow to trigger error
The content modifier “Map props” stores the necessary metadata to create a meaningful Issue in Azure Boards and passes them as headers through the Process Direct link. My shared iFlow “CreateIssueInAzureDevOps” picks up the message with the ProcessDirect receive link.
Fig.3 iFlow to create Issue in Azure Boards
Often iFlows receive multiple messages in a short time. To avoid flooding Azure Boards with redundant issues I added a “fail safe” mechanism using global variables. It makes sure that only one message a day is processed for that iFlow. Going forward you can get more sophisticated with your failsafe. For my purposes this will suffice.
The http call on the right targets my Azure LogicApp. You can call LogicApps with access key (shared access signature URLs), put them on Azure VNets and deploy networking components in front, or simply use an Azure API Management component with a whitelist entry on Azure LogicApps for private access.
On Azure API Management you can get crazy with subscription keys, basic auth, OAuth and the likes. I chose API Management, because I am exposing it on the internet.
On the “Generate Payload” step I create a JSON payload containing the metadata from the sending iFlow, that will be passed to the Azure LogicApp.
Fig.4 Azure LogicApp to create Issue in Azure Boards
The API connector for Azure DevOps allows quick and easy ui-based integration for the REST interface. The placeholders from the JSON payload from SAP CPI are parsed automatically. Any citizen developer can create such an integration effortlessly.
Speaking of effort: you could come up with solutions to map severe errors to higher priorities and effort for instance (check lines below description box). For instance, an http 500 error could possibly be a more complex problem and should be ranked higher than a http 404. To achieve that, add a new step to evaluate the httpCode and create a variable for “effort”. Reference that instead of my constant “10” and you are good to go.
So, if I trigger the first iFlow now, we will see an error message and a completed one for the creation of the Issue in Azure Boards.
Fig.5 Postman request to first iFlow
That gives us an error message, because the target https://postman-echo.com/sth does not exist.
Fig.6 Message Processing view on CPI
Fig.7 Message Processing details view on CPI
As expected, the problematic iFlow shows a failed message and the iFlow to create the Issue in Azure Boards completed successfully. The result of the HTML encoded message from SAP CPI in Azure Boards looks like this:
Fig.8 Screenshot of Issue in Azure Boards
You can see that all info from the Azure LogicApp has been applied. The deployment section on the right will highlight on which stage (dev/test or prod) the change is already applied. That’s a nice way to get insights and transparency of your development efforts.
The link next to the entry “CPI-Log” leads back to the text-based message monitor in SAP CPI:
Fig.9 Screenshot of text-based message view in SAP CPI
Once the CPI API allows navigation to the detailed UI, I am going to update the call. Until then the text-based version is all we can get.
For the fully-cycle you can navigate from the error message back to Azure Boards:
Fig.10 Screenshot of Message view to link back to Azure Boards
Fig.11 Link back to Azure Boards from SAP CPI message
So far attachments on CPI messages are still treated as plain text. The change for html is supposed to be on the road map at SAP. Until then you need to select the url manually.
Great, now on to fixing things and applying CI/CD!
Imagine your error originates from a groovy script. SAP CPI offers two types of scripting for complex integration tasks – JavaScript and Groovy. Often the scripting tasks perform the heavy lifting of your integration process and can become quite complex. Therefore, it is very important to apply good coding practices like DevOps.
The web-based UI of SAP CPI is great for online mapping of properties and connection configuration. But when it comes to coding not so much. Due to that many integration developers use local IDEs like Eclipse on their desktops. I shared a blog on how to setup a local dev environment some time ago in case you want to dig deeper.
In my example I am using Visual Studio Code. To get the contents of your integration flow you have multiple options:
- Download a zip file through the CPI web-ui: Click actions on the iFlow overview -> Download
- SAP API Business Hub: /IntegrationDesigntimeArtifacts(Id='{Id}',Version='{Version}')/$value
- Use my python script: https://github.com/MartinPankraz/SAPCPI-Az-DevOps/blob/master/Templates/download-package.py
My script uses the SAP API Business Hub to download all the iFlows from my package and updates the local git repos. You can adapt that easily to your environment.
Fig.12 DevOps process with focus on CI/CD
The continuous integration (CI) trigger on Azure pipelines automatically picks up your groovy script change and prepares the content for further processing. During the next iteration of this blog post I am going to add unit testing for groovy for a more complete CI step.
Fig.13 Screenshot from build pipe in Azure DevOps
Once the deployable is created the release pipe takes over:
Fig.14 Screenshot from release pipe in Azure DevOps
I applied a simple approval process and created service connections for two CPI trial accounts in the CloudFoundry environment to simulate the staging process. For most SAP Cloud Platform subscriptions, you get two stages: one for dev/test and for production. That’s another big reason for good DevOps practices to ensure the quality of your releases.
Fig.15 Screenshot from release pipe details in Azure DevOps
I created a python script that iterates all groovy scripts of my iFlow and sends the update to CPI via the SAP API Business Hub. Meaning the update happens on a file by file basis. In case you want to expand the approach to the other file types of integration flows you will need to enhance the python script further.
You can find the API Business Hub docs here.
Interesting 3rd party implementation for the CI/CD practice
There is a nice video by daniel.graversen for an integrated build and testing environment for CPI with similar ideas. Daniel’s video will give you another view on what is possible with CPI today.
The final mile to an end-to-end process
Since the Issue in Azure Boards is associated with Git it can automatically close. There are various ways you can achieve that. Find the specifics here ? That’s what I call visibility and control over your iFlow development and deployment!
Fig.16 Screenshot from complete Issue in Az Boards
Hints on configuration to replicate my prototype
Guide on how to setup your Cloud Foundry trial environment: https://developers.sap.com/tutorials/cp-starter-integration-cpi-onboard-subscribe.html
GitHub Repos with example integration flow and logic app template: https://github.com/MartinPankraz/SAPCPI-Az-DevOps
To upload my iFlows to your tenant you can simply zip the folders here individually and upload through the UI on SAP CPI. Within your package click edit -> Actions -> View Metadata -> Edit -> Browse -> Upload zip. Make sure to replace the http call to Azure DevOps in case you are not using Azure API Management.
Also, I put a placeholder “<<<>>>” on the ARM template to mask my subscription id. You can either modify the JSON code or mix & match the config and re-create a new instance of Azure LogicApps.
Azure DevOps project: https://dev.azure.com/mapankra/SAP-CPI-Integration-Az-DevOps
Final Words
Both Logic apps and SAP CPI are strong integration solutions with different strengths regarding their connector portfolio. SAP targets their system landscape with great standard integration content out-of-the-box and “Open Connectors” for APIs such as Microsoft Graph and the likes, that you can easily extend. LogicApps is a more generic solution on serverless infrastructure, that allows pay-per-use. That is not the case for CPI. Like CPI’s integration content LogicApps offer building blocks like the one you saw in fig.4.
Another big part of my post was enablement of DevOps practices for your SAP CPI iFlows. Leveraging the existing REST APIs you can bridge the gap of CI/CD capabilities. I focused on groovy scripts because they are the most complex integration component. Since they are “code” it is a no-brainer to apply coding practices to them. During the next iteration of this post I am going to add unit tests for an even more complete solution.
As you can see there is merit to combine SAP CPI with Azure DevOps and even Azure LogicApps to make use of the best of both worlds.
As always feel free to leave or ask lots of follow-up questions.
Best Regards
Martin
Opinions expressed are solely my own and do not express the views or opinions of my employer.
- SAP Managed Tags:
7 Comments
