Skip to Content
Technical Articles
Author's profile photo Eng Swee Yeoh

Want to sync your IFlows to GitHub? FlashPipe it!

Did you say cpiGit?

One of the most requested feature on the Customer Influence Project for SAP Integration Suite is native Git support for Cloud Integration.

 

While waiting for SAP to provide such capability, this gap has been filled in various manners by the community, largely utilising the public API available to download the integration artifacts from the Cloud Integration tenant.

 

When FlashPipe was in its development phase, I was reluctant to include such functionality in it. It was my conviction that SAP should be delivering such a vital and much-requested functionality, especially in light of the fact that any self-respecting development platform these days would have such functionality natively. Additionally, I didn’t feel the urgency to reinvent the wheel in view of the existing ‘workarounds’ already available.

 

It has been more than a year since the above request was submitted and as time goes by, it is becoming uncertain if this will be delivered despite the huge demand for it from the partner and customer ecosystem.

 

Since the release of FlashPipe, I have revisited my position on this, and I have realised otherwise that I cannot wait any longer, and it is time to take it into my own hands. Coincidentally, whilst working on the GitHub platform, I also realised that such a functionality is a “low-hanging fruit” (since most of the functions required for it is already available in FlashPipe), but the benefits are aplenty.

 

All you need is love… FlashPipe!

I am happy to announce that since release 2.1.0, FlashPipe now supports syncing of Integration Flows from a Cloud Integration tenant to a Git repository.

In particular, if you are already using GitHub, you can get it up and running in a ⚡flash⚡(pardon the pun!)

 

Once you get the GitHub Actions workflow in place, you can run the workflow, and let FlashPipe do the heavy lifting for you – download the content from the tenant, compare it against the repository, and commit any differences.

 

 

 

Voila, and it’s done ✅!

 

Look mom, no tools to install!

Even if you do not use FlashPipe for deployment of your IFlows, it can come in handy to back up your precious IFlow development work to GitHub. And you can even schedule it on a periodic basis and have the peace of mind that your work is always safe! And all this is achieved without requiring any additional tool or programs to be installed on your local computer.

 

If you haven’t tried FlashPipe, head over to its documentation page and try it for yourself. The following page provides you the steps to get started.

Sync IFlows to GitHub with FlashPipe on GitHub Actions

 

Psst.. FlashPipe is open-source

Oh, and did I mention…. FlashPipe is open-source and licensed under Apache 2.0, so you don’t have to worry about giving it a go!

 

Assigned Tags

      30 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo prachetas singh
      prachetas singh

      Thanks a lot for this wonderful blog.

       

      I tried to replicate but at Sync step getting 401 error.

       

      Client ID,  Client Secret  and environmental variables seems to be correct.

       

      nv:
      HOST_TMN: abc.it-cpitrial02.cfapps.eu10-001.hana.ondemand.com
      HOST_OAUTH: abc.authentication.eu10.hana.ondemand.com
      OAUTH_CLIENTID: ${{ secrets.DEV_CLIENT_ID }}
      OAUTH_CLIENTSECRET: ${{ secrets.DEV_CLIENT_SECRET }}

       

      [INFO] Using Basic Authentication for https://abc.authentication.eu10.hana.ondemand.com:443
      [INFO] Using OAuth 2.0 Authentication for https://abc.it-cpitrial02.cfapps.eu10-001.hana.ondemand.com:443
      [INFO] Getting artifacts in integration package PackageForTransportDemo
      
      [ERROR]  Response body = 
      Exception in thread "main" io.github.engswee.flashpipe.http.HTTPExecuterException: Get designtime artifacts of IntegrationPackages call failed with response code = 401
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
      	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
      	at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:83)
      	at org.codehaus.groovy.reflection.CachedConstructor.doConstructorInvoke(CachedConstructor.java:77)
      	at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrap.callConstructor(ConstructorSite.java:84)
      	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:59)
      	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:238)
      	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:250)
      	at io.github.engswee.flashpipe.http.HTTPExecuter.logError(HTTPExecuter.groovy:40)
      	at io.github.engswee.flashpipe.http.HTTPExecuter$logError$0.call(Unknown Source)
      	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
      	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
      	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:128)
      	at io.github.engswee.flashpipe.cpi.api.IntegrationPackage.getIFlowsWithDraftState(IntegrationPackage.groovy:49)
      	at io.github.engswee.flashpipe.cpi.api.IntegrationPackage$getIFlowsWithDraftState.call(Unknown Source)
      	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
      	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
      	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:128)
      	at io.github.engswee.flashpipe.cpi.exec.DownloadIntegrationPackageContent.execute(DownloadIntegrationPackageContent.groovy:48)
      	at io.github.engswee.flashpipe.cpi.exec.DownloadIntegrationPackageContent$execute.call(Unknown Source)
      	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
      	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
      	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
      	at io.github.engswee.flashpipe.cpi.exec.DownloadIntegrationPackageContent.main(DownloadIntegrationPackageContent.groovy:20)
      Error:   Execution of java command failed
      Error: Process completed with exit code 1.

       

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      I'd suggest the following:

      • Go through the OAuth client setup page (below) to make sure your service instance is set up correctly

      https://engswee.github.io/flashpipe/oauth_client.html

      • Try with Basic auth to check if it is also getting the same issue

      If it still does not work, please raise an issue on the GitHub (https://github.com/engswee/flashpipe/issues) so that we can look into this further.

       

       

      Author's profile photo prachetas singh
      prachetas singh

      Completely missed the setup of Oauth client.

       

      I had "ESBMessaging.Send"   role assigned to my user, that was the problem.

      Thanks for the setup instructions.

       

      It is working now.

      🙂

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Fantastic. Glad to hear it is working now 😉

      Author's profile photo Philippe Addor
      Philippe Addor

      Hi Prachetas

      I see from the error message that you have set this up on a trial account. I Did you successfully create a service key with plan "API" on it? When I try to create a new key for API access I get the error described here: https://answers.sap.com/questions/13005478/scp-abap-trial-service-broker-error.html

      I however can create keys for plan "integration-flow" without problems.  Maybe you or somebody can let me know if that's still supposed to work.

      Best regards,

      Philippe

      Author's profile photo Rob Hofman
      Rob Hofman

      I truly believe this will be a pivotal monent in de SAP Integration timeline! Thanks for developing this, can’t wait to get my hands dirty with this!

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Thanks Rob. I see that you are already getting your hands dirty with it on GitHub. Hope you like it so far! 🙂

      Author's profile photo Rob Hofman
      Rob Hofman

      It really is a joy to use 😁! Is the sync with a git repository on azure devops possible using azure devops pipelines?

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Good to hear that, Rob. About syncing to Azure Git Repo, I haven't tried that myself but I would think it would be possible.

      Why don't you give it a try? For GitHub Actions, there is an action available that does the "push to Git", so you'll have to find if there is an available task in Azure to do the same, or if not use the command line.

      Author's profile photo Rob Hofman
      Rob Hofman

      Hi Eng, indeed had to use the command line. I had some issues (mainly because this is the first time I create a pipeline) but got it working 😉 thanks for the great tool!

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Hi Rob,

       

      I was away for a while, but thanks for the update. It's great to hear that you got it working, and you learned something along the way 😉

      Would you kindly share what you had to do for it to work with Azure repository, for the sake of those who might be interested also in it?

       

      Regards

      Eng Swee

      Author's profile photo Rob Hofman
      Rob Hofman

      I was stuck a long time on this line:

      echo "GIT_SRC_DIR=$GITHUB_WORKSPACE/${{ github.event.inputs.directory }}" >> $GITHUB_ENV

      I the end it appeared I could just add the following to the env variables.

                GIT_SRC_DIR : $(System.DefaultWorkingDirectory)/iflows
      The other thing is as you said I had to push using a command.
      Author's profile photo Jacques Otto
      Jacques Otto

      Hi Rob,

       

      Any chance you could share an example of how you created this for azure?

      Author's profile photo Pieter Janssens
      Pieter Janssens

      Hi Rob,

      Would you be willing to contribute your ADO pipeline (.YML) to sync with git over here https://github.com/engswee/flashpipe-demo?

      Best regards,

      Pieter

      Author's profile photo Fatih Pense
      Fatih Pense

      Thanks for sharing this, Eng Swee! Using GitHub actions is a great idea.

      Regards,
      Fatih

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Thanks Fatih. Indeed GitHub Actions is quite nice - there are tonnes of stuff you could do with it.

      Author's profile photo cpi learner
      cpi learner

      Wow. it works great. I am able to add the package to GitHub.

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      That's fantastic! Glad you like it 😉

      Author's profile photo Cveto Ljubic
      Cveto Ljubic

      Hello. Sorry to write you here. I attend your course of FlashPipe Simplifying CI/CD. I was just wondering where can I get a video of course. Thank you for info. Best Regards, Cveto

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Hi Cveto

       

      Our team will be doing some editing on it first. Once it's done, you will get an email with a link to it. Please do be patient while our team works on it.

       

      Thanks

      Eng Swee

      Author's profile photo Cveto Ljubic
      Cveto Ljubic

      Hello, thank you for the reply. For good things it is worth the wait 🙂

      Best Regards,

      Cveto

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Hi Cveto

       

      You should have received the email with the recording already. Hope your patience is well rewarded 🙂

      Author's profile photo Cveto Ljubic
      Cveto Ljubic

      Hello Eng, I have received everything. You are the best. Best Regards Cveto

       

      Author's profile photo Sam Hepworth
      Sam Hepworth

      Hi Eng Swee,

      Thanks for releasing Flashpipe, version control is something I have been missing for a long time in CPI, and this is an excellent addition.

      I have got it working with Cloud Foundry, however I am having an issue with getting the OAuth to work with Neo.

      To begin with I was using HOST_OAUTH: oauthasservices-xxx.eu2.hana.ondemand.com without the token path, however I was getting a 404 error. I suspect I need to add the full path so added /oauth2/api/v1/token, however I get a 400 error. I have tested the OAuth token using Postman and the only difference I can see is that the port is included, so not sure if this is the issue.

      [INFO] Using Basic Authentication for https://oauthasservices-xxx.eu2.hana.ondemand.com/oauth2/api/v1/token:443
      Exception in thread "main" io.github.engswee.flashpipe.http.HTTPExecuterException: Get OAuth token call failed with response code = 404

      Do I have the OAuth host correct?

      Thanks,

      Sam

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Hi Sam,

       

      Glad to hear that you find FlashPipe useful. For OAuth in Neo, the path differs from CF, so you need to set the additional parameter (whilst keeping HOST_OAUTH as just the base URL).

      HOST_OAUTH_PATH: '/oauth2/api/v1/token'

       

      FlashPipe comes with the examples repository below, where you can find examples of the different use cases. You can check out the one which covers OAuth on Neo.

       

      Regards

      https://github.com/engswee/flashpipe-demo

      Author's profile photo Sam Hepworth
      Sam Hepworth

      Hi Eng Swee,

      Thanks for the quick response. That's what I was missing, its working perfectly now.

      Thanks,

      Sam

      Author's profile photo Eng Swee Yeoh
      Eng Swee Yeoh
      Blog Post Author

      Fantastic! Glad to hear it's working now!

      Author's profile photo Sunil Kolassery
      Sunil Kolassery

      We have 3 tenants architecture, but need to support 6 S4 tenants in the integrated landscape ( Dev-4QA plus Training - Prod ) as per project rollout plans.
      We can manually copy integration packages in the QA tenant to support this currently. Is it possible to mimic & automate this in GIT.
      So - Copy from same tenant(qa) the package to a renamed package (suffix addition), configure the props in the renamed package & deploy to same tenant

      R4\PolVat\src - Package "PolVAT" iflow "Pol VAT" (Dev source tenant)
      R4\PolVat\qa - Package "PolVAT" iflow "Pol VAT" (QA tenant)
      R4\PolVat\qa\bqb - Package "PolVAT bqb" iflow "Pol VAT" (QA tenant)
      R4\PolVat\qa\rt4 - Package "PolVAT rt4" iflow "Pol VAT" (QA tenant)

      R4\PolVat\qa\trn - Package "PolVAT trn" iflow "Pol VAT" (QA tenant)

      R4\PolVat\prod - Package "PolVAT" iflow "Pol VAT" (Prod)

      Author's profile photo Deepak Sharma
      Deepak Sharma

      Hi Eng Swee,

      Is it possible to add pretty print before comparing content from tenant against Git ? For message mapping .mmap files content is in XML but in 1 single line. Changing map of i fields shows as full file is change in version history in GitHub.

      It will be good if pretty print is done on .mmap file after content is downloaded from tenant and before checkin to GitHub.

      Thanks

      Deepak

      Author's profile photo Guido Reboredo
      Guido Reboredo

      Hi Eng Swee,

      One question, how can I upload my snapshots from Git Hub to SAP Cloud Integration. What I'm trying to do is to download the git repository and use the import feature on CI to upload a package or an iFlow that I snapshot from another environment, but when trying to do that it's throwing an error about the metadata of the iFlows. Do you know why?

       

      Thanks,

      Regards