Technical Articles

Eng Swee Yeoh

July 5, 2021 2 minute read

Want to sync your IFlows to GitHub? FlashPipe it!

Did you say cpiGit?

One of the most requested feature on the Customer Influence Project for SAP Integration Suite is native Git support for Cloud Integration.

While waiting for SAP to provide such capability, this gap has been filled in various manners by the community, largely utilising the public API available to download the integration artifacts from the Cloud Integration tenant.

When FlashPipe was in its development phase, I was reluctant to include such functionality in it. It was my conviction that SAP should be delivering such a vital and much-requested functionality, especially in light of the fact that any self-respecting development platform these days would have such functionality natively. Additionally, I didn’t feel the urgency to reinvent the wheel in view of the existing ‘workarounds’ already available.

It has been more than a year since the above request was submitted and as time goes by, it is becoming uncertain if this will be delivered despite the huge demand for it from the partner and customer ecosystem.

Since the release of FlashPipe, I have revisited my position on this, and I have realised otherwise that I cannot wait any longer, and it is time to take it into my own hands. Coincidentally, whilst working on the GitHub platform, I also realised that such a functionality is a “low-hanging fruit” (since most of the functions required for it is already available in FlashPipe), but the benefits are aplenty.

All you need is love… FlashPipe!

I am happy to announce that since release 2.1.0, FlashPipe now supports syncing of Integration Flows from a Cloud Integration tenant to a Git repository.

In particular, if you are already using GitHub, you can get it up and running in a ⚡flash⚡(pardon the pun!)

Once you get the GitHub Actions workflow in place, you can run the workflow, and let FlashPipe do the heavy lifting for you – download the content from the tenant, compare it against the repository, and commit any differences.

Voila, and it’s done ✅!

Look mom, no tools to install!

Even if you do not use FlashPipe for deployment of your IFlows, it can come in handy to back up your precious IFlow development work to GitHub. And you can even schedule it on a periodic basis and have the peace of mind that your work is always safe! And all this is achieved without requiring any additional tool or programs to be installed on your local computer.

If you haven’t tried FlashPipe, head over to its documentation page and try it for yourself. The following page provides you the steps to get started.

Sync IFlows to GitHub with FlashPipe on GitHub Actions

Psst.. FlashPipe is open-source

Oh, and did I mention…. FlashPipe is open-source and licensed under Apache 2.0, so you don’t have to worry about giving it a go!

30 Comments

You must be Logged on to comment or reply to a post.

prachetas singh

July 5, 2021 at 10:52 am

Thanks a lot for this wonderful blog.

I tried to replicate but at Sync step getting 401 error.

Client ID, Client Secret and environmental variables seems to be correct.

nv:
	HOST_TMN: abc.it-cpitrial02.cfapps.eu10-001.hana.ondemand.com
	HOST_OAUTH: abc.authentication.eu10.hana.ondemand.com
	OAUTH_CLIENTID: ${{ secrets.DEV_CLIENT_ID }}
	OAUTH_CLIENTSECRET: ${{ secrets.DEV_CLIENT_SECRET }}

[INFO] Using Basic Authentication for https://abc.authentication.eu10.hana.ondemand.com:443
[INFO] Using OAuth 2.0 Authentication for https://abc.it-cpitrial02.cfapps.eu10-001.hana.ondemand.com:443
[INFO] Getting artifacts in integration package PackageForTransportDemo

[ERROR]  Response body = 
Exception in thread "main" io.github.engswee.flashpipe.http.HTTPExecuterException: Get designtime artifacts of IntegrationPackages call failed with response code = 401
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:83)
	at org.codehaus.groovy.reflection.CachedConstructor.doConstructorInvoke(CachedConstructor.java:77)
	at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrap.callConstructor(ConstructorSite.java:84)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:59)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:238)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:250)
	at io.github.engswee.flashpipe.http.HTTPExecuter.logError(HTTPExecuter.groovy:40)
	at io.github.engswee.flashpipe.http.HTTPExecuter$logError$0.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:128)
	at io.github.engswee.flashpipe.cpi.api.IntegrationPackage.getIFlowsWithDraftState(IntegrationPackage.groovy:49)
	at io.github.engswee.flashpipe.cpi.api.IntegrationPackage$getIFlowsWithDraftState.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:128)
	at io.github.engswee.flashpipe.cpi.exec.DownloadIntegrationPackageContent.execute(DownloadIntegrationPackageContent.groovy:48)
	at io.github.engswee.flashpipe.cpi.exec.DownloadIntegrationPackageContent$execute.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
	at io.github.engswee.flashpipe.cpi.exec.DownloadIntegrationPackageContent.main(DownloadIntegrationPackageContent.groovy:20)
Error:   Execution of java command failed
Error: Process completed with exit code 1.

Eng Swee Yeoh

Blog Post Author

July 5, 2021 at 11:10 am

I'd suggest the following:

Go through the OAuth client setup page (below) to make sure your service instance is set up correctly

https://engswee.github.io/flashpipe/oauth_client.html

Try with Basic auth to check if it is also getting the same issue

If it still does not work, please raise an issue on the GitHub (https://github.com/engswee/flashpipe/issues) so that we can look into this further.

prachetas singh

July 5, 2021 at 1:01 pm

Completely missed the setup of Oauth client.

I had "ESBMessaging.Send" role assigned to my user, that was the problem.

Thanks for the setup instructions.

It is working now.

🙂

Eng Swee Yeoh

Blog Post Author

July 5, 2021 at 1:48 pm

Fantastic. Glad to hear it is working now 😉

Philippe Addor

August 24, 2022 at 1:08 pm

Hi Prachetas

I see from the error message that you have set this up on a trial account. I Did you successfully create a service key with plan "API" on it? When I try to create a new key for API access I get the error described here: https://answers.sap.com/questions/13005478/scp-abap-trial-service-broker-error.html

I however can create keys for plan "integration-flow" without problems. Maybe you or somebody can let me know if that's still supposed to work.

Best regards,

Philippe

Rob Hofman

July 6, 2021 at 11:57 am

I truly believe this will be a pivotal monent in de SAP Integration timeline! Thanks for developing this, can’t wait to get my hands dirty with this!

Eng Swee Yeoh

Blog Post Author

July 8, 2021 at 9:48 am

Thanks Rob. I see that you are already getting your hands dirty with it on GitHub. Hope you like it so far! 🙂

Rob Hofman

July 10, 2021 at 6:29 pm

It really is a joy to use 😁! Is the sync with a git repository on azure devops possible using azure devops pipelines?

Eng Swee Yeoh

Blog Post Author

July 11, 2021 at 7:06 pm

Good to hear that, Rob. About syncing to Azure Git Repo, I haven't tried that myself but I would think it would be possible.

Why don't you give it a try? For GitHub Actions, there is an action available that does the "push to Git", so you'll have to find if there is an available task in Azure to do the same, or if not use the command line.

Rob Hofman

July 14, 2021 at 3:37 pm

Hi Eng, indeed had to use the command line. I had some issues (mainly because this is the first time I create a pipeline) but got it working 😉 thanks for the great tool!

Eng Swee Yeoh

Blog Post Author

July 25, 2021 at 5:46 pm

Hi Rob,

I was away for a while, but thanks for the update. It's great to hear that you got it working, and you learned something along the way 😉

Would you kindly share what you had to do for it to work with Azure repository, for the sake of those who might be interested also in it?

Regards

Eng Swee

Rob Hofman

July 26, 2021 at 10:22 am

I was stuck a long time on this line:

echo "GIT_SRC_DIR=$GITHUB_WORKSPACE/${{ github.event.inputs.directory }}" >> $GITHUB_ENV

I the end it appeared I could just add the following to the env variables.

GIT_SRC_DIR : $(System.DefaultWorkingDirectory)/iflows

The other thing is as you said I had to push using a command.

Jacques Otto

August 25, 2021 at 9:01 am

Hi Rob,

Any chance you could share an example of how you created this for azure?

Pieter Janssens

December 21, 2022 at 1:46 pm

Hi Rob,

Would you be willing to contribute your ADO pipeline (.YML) to sync with git over here https://github.com/engswee/flashpipe-demo?

Best regards,

Pieter

Fatih Pense

July 8, 2021 at 12:14 am

Thanks for sharing this, Eng Swee! Using GitHub actions is a great idea.

Regards,
Fatih

Eng Swee Yeoh

Blog Post Author

July 8, 2021 at 9:49 am

Thanks Fatih. Indeed GitHub Actions is quite nice - there are tonnes of stuff you could do with it.

cpi learner

July 14, 2021 at 2:28 pm

Wow. it works great. I am able to add the package to GitHub.

Eng Swee Yeoh

Blog Post Author

July 14, 2021 at 7:21 pm

That's fantastic! Glad you like it 😉

Cveto Ljubic

August 16, 2021 at 12:09 pm

Hello. Sorry to write you here. I attend your course of FlashPipe Simplifying CI/CD. I was just wondering where can I get a video of course. Thank you for info. Best Regards, Cveto

Eng Swee Yeoh

Blog Post Author

August 16, 2021 at 2:02 pm

Hi Cveto

Our team will be doing some editing on it first. Once it's done, you will get an email with a link to it. Please do be patient while our team works on it.

Thanks

Eng Swee

Cveto Ljubic

August 17, 2021 at 6:04 am

Hello, thank you for the reply. For good things it is worth the wait 🙂

Best Regards,

Cveto

Eng Swee Yeoh

Blog Post Author

August 20, 2021 at 1:46 pm

Hi Cveto

You should have received the email with the recording already. Hope your patience is well rewarded 🙂

Cveto Ljubic

August 24, 2021 at 12:25 pm

Hello Eng, I have received everything. You are the best. Best Regards Cveto

Sam Hepworth

August 18, 2021 at 1:52 pm

Hi Eng Swee,

Thanks for releasing Flashpipe, version control is something I have been missing for a long time in CPI, and this is an excellent addition.

I have got it working with Cloud Foundry, however I am having an issue with getting the OAuth to work with Neo.

To begin with I was using HOST_OAUTH: oauthasservices-xxx.eu2.hana.ondemand.com without the token path, however I was getting a 404 error. I suspect I need to add the full path so added /oauth2/api/v1/token, however I get a 400 error. I have tested the OAuth token using Postman and the only difference I can see is that the port is included, so not sure if this is the issue.

[INFO] Using Basic Authentication for https://oauthasservices-xxx.eu2.hana.ondemand.com/oauth2/api/v1/token:443
Exception in thread "main" io.github.engswee.flashpipe.http.HTTPExecuterException: Get OAuth token call failed with response code = 404

Do I have the OAuth host correct?

Thanks,

Sam

Eng Swee Yeoh

Blog Post Author

August 18, 2021 at 5:55 pm

Hi Sam,

Glad to hear that you find FlashPipe useful. For OAuth in Neo, the path differs from CF, so you need to set the additional parameter (whilst keeping HOST_OAUTH as just the base URL).

HOST_OAUTH_PATH: '/oauth2/api/v1/token'

FlashPipe comes with the examples repository below, where you can find examples of the different use cases. You can check out the one which covers OAuth on Neo.

Regards

https://github.com/engswee/flashpipe-demo

Sam Hepworth

August 18, 2021 at 6:37 pm

Hi Eng Swee,

Thanks for the quick response. That's what I was missing, its working perfectly now.

Thanks,

Sam

Eng Swee Yeoh

Blog Post Author

August 18, 2021 at 7:08 pm

Fantastic! Glad to hear it's working now!

Sunil Kolassery

August 18, 2022 at 9:35 am

We have 3 tenants architecture, but need to support 6 S4 tenants in the integrated landscape ( Dev-4QA plus Training - Prod ) as per project rollout plans.
We can manually copy integration packages in the QA tenant to support this currently. Is it possible to mimic & automate this in GIT.
So - Copy from same tenant(qa) the package to a renamed package (suffix addition), configure the props in the renamed package & deploy to same tenant

R4\PolVat\src - Package "PolVAT" iflow "Pol VAT" (Dev source tenant)
R4\PolVat\qa - Package "PolVAT" iflow "Pol VAT" (QA tenant)
R4\PolVat\qa\bqb - Package "PolVAT bqb" iflow "Pol VAT" (QA tenant)
R4\PolVat\qa\rt4 - Package "PolVAT rt4" iflow "Pol VAT" (QA tenant)

R4\PolVat\qa\trn - Package "PolVAT trn" iflow "Pol VAT" (QA tenant)

R4\PolVat\prod - Package "PolVAT" iflow "Pol VAT" (Prod)

Deepak Sharma

January 11, 2023 at 12:46 am

Hi Eng Swee,

Is it possible to add pretty print before comparing content from tenant against Git ? For message mapping .mmap files content is in XML but in 1 single line. Changing map of i fields shows as full file is change in version history in GitHub.

It will be good if pretty print is done on .mmap file after content is downloaded from tenant and before checkin to GitHub.

Thanks

Deepak

Guido Reboredo

September 20, 2023 at 6:59 pm

Hi Eng Swee,

One question, how can I upload my snapshots from Git Hub to SAP Cloud Integration. What I'm trying to do is to download the git repository and use the import feature on CI to upload a package or an iFlow that I snapshot from another environment, but when trying to do that it's throwing an error about the metadata of the iFlows. Do you know why?

Thanks,

Regards

Technical Articles

Want to sync your IFlows to GitHub? FlashPipe it!

Did you say cpiGit?

All you need is love… FlashPipe!

Look mom, no tools to install!

Psst.. FlashPipe is open-source

Assigned Tags