HCI: Using EGit for version management of Eclipse-based Integration Projects
One of the most frustrating aspects of working with HCI is the buggy development environment of the Eclipse-based Integration Designer plugin.
While in general some of bugs are just minor annoyances (error pop ups that can be “cured” by a restart), the most disastrous is when a perfectly working iFlow no longer works after some changes. Even though the processing logic of the iFlow is reverted back to the original, it sometimes will still not work. Sometimes, iFlow changes will cause local checks to fail with non-descriptive errors. Sometimes, the changes will pass local checks but will fail during deployment to the tenant, again with non-descriptive errors in the tail log. It took a while to understand what was happening as it occurs in a random fashion.
As I was new to HCI, I tended to develop the iFlows in a stage-by-stage manner – include certain functionality, save, deploy and test it out before enhancing it with further functionality. Initially, the errors cause me to think that some of the logic that I was trying to implement were not possible or not supported, even though the errors do not specifically mention that. However, I found out that if I were to develop the similar logic all at once in a new iFlow, it would work!!
This led me to the conclusion that “iFlow files can be corrupted when making changes!“
In my experience, corruption tends to happen more often when there are deletion of objects or connectors in the iFlow. Although iFlows are designed using a graphical layout editor, the content is actually saved as an XML file. Therefore my guess is that when some editing is done via the graphical iFlow editor, the underlying XML is not adjusted correctly, causing corruption of the generated file.
Furthermore, HCI does not come with native version management, therefore changes are not easily reversible.
After much frustration getting my iFlows corrupted from time to time, and having to rebuilt them from scratch, I decided to use Git for version management of my HCI Integration Projects. In this blog, I will share with you how the EGit plugin within Eclipse can be used to configure the Git version management system for local development of HCI iFlows. This is similar to the approach I’ve blogged about below on using EGit for Java development in PI/NWDS.
Using EGit for Java source code management in NWDS
Below are component versions of the Eclipse plugins that I was using. Hopefully future versions would provide a more stable iFlow editor.
Eclipse Plugin Versions: Adapter 2.11.1, Designer 2.11.1, Operations 2.10.0
In order to use EGit, the EGit plugin needs to be installed in the Eclipse environment. For those using Luna SR2 (4.4.2), it should already come together with EGit 3.4.2 as shown below.
Before using EGit in Eclipse, some initial configuration needs to be performed first. Please follow the same steps as indicated in Initial Configuration section of the following blog.
Using EGit for Java source code management in NWDS
Manage Integration Project with EGit in Eclipse
Following are the steps to put an HCI Integration Project under EGit’s version management.
Step 1 – Create Git Repository
Change to Git perspective – Window > Open Perspective > Other … > Git
Click the Create a New Git Repository button.
Specify the location for the new Git repository.
Once the repository is created, it will be listed under the Git Repositories tab.
Step 2 – Import Integration Project into Git Repository
Next we want to import/share the HCI Integration Projects in the newly created Git Repository. Switch back to the Integration Designer perspective.
Right click on the project, and select Team > Share Project.
Select Git from the sharing type.
Select the recently created repository from the dropdown list for Repository.
Step 3 – Commit contents to be tracked
After the project has been shared in the Git repository, we can proceed to commit the contents that are to be tracked.
Right click on the project, and select Team > Commit.
Provide a message for the commit and select the files that are to be tracked – in general I will select all source codes (iFlow, XSD, WSDL, Scripts) and project configuration files.
Now that the contents have been committed to the Git repository, I am free to modify the contents of the iFlow (or any of the commited objects) without worrying if any changes would render the iFlow useless.
Each time I’ve made changes to the iFlow and verified that it’s working, the commit step is repeated to save a version of that to the repository.
Step 4 – Reverting iFlow to previous version
Let say after some changes have been made to the iFlow and it becomes corrupted causing errors during local check or deployment, we can revert to a previous version by right clicking the iFlow object in the Project Explorer and selecting Replace With.
We can either select HEAD revision (which is normally the last committed version) or Commit… for some other version from the commit history.
As shown, by using EGit as the version management system for HCI Integration Projects, we can avoid the potential disastrous effects of changes rendering an iFlow corrupted and useless. There are also other benefits of using EGit since there is no native version management system for HCI objects that are developed locally on Eclipse. Besides iFlow, the other objects in an Eclipse HCI Integration Project can be tracked too like WSDLs, XSDs, Groovy scripts and even Message Mapping which cannot be easily “version-managed” by other methods.
I’d definitely recommend those working on HCI with Eclipse to use EGit if there is no other version management tool in place.
Thanks for this!
I have experienced the bugs in Eclipse too where my iflow got corrupted and everything in it was deleted. Fortunately I could always import the iFlow back from the tenant so I never lost my developments.
However, we are looking for a version management tool for our developments in HCI and this looks promising. I also noticed that it is possible to create a shared repository.
In my own experience, sometimes the local iFlow check is fine but it will not work when it is deployed, therefore downloading the iFlow back from the tenant does not resolve the issue as the tenant has the latest corrupted iFlow.
Git is used in a lot of areas (not just Java, SAP, HCI, etc) and yes, it is possible to have a shared repository. There are many ways for this, i.e. public GitHub respository, network directory, private Git installation, etc. You can read more about Git and EGit.
Hello Eng Swee,
I see that your article talks about how to 'maintain' one's HCI iflow via GIT.
Have you and your team also tried 'parallel development' on the same Iflow while using GIT
Example : developer 1 does some changes on Iflow 1, and then developer 2 also changes the same flow. Later they both merge and push their changes into GIT on the same iflow
Has this been tried? Because per my experience, this version maintenance via GIT works as long as 'Only 1 developer works on an Iflow at a time'. If more than one person does the iflow changes and merge in GIT, it corrupts the iflow again!
Git has many features and "parallel development" is indeed one of them. However, my article focuses on Git's version management feature (which works in both solo development mode and team based development).
IMHO, Git's approach to parallel development works best with source code based development (i.e. Java, PHP, etc). iFlows are a little bit different because the underlying XML is generated from the graphical iFlow editor. As such I don't think it makes sense to have parallel development of iFlow via Git.
Saying that I still think there is a benefit in using Git for team-based developments of Eclipse/HCI projects until SAP comes up with a native version management for HCI.
you are absolutely right, when you just pull others changes into yours normally your XML structure might break. But you can make use of https://git-scm.com/docs/gitattributes and define all IFlows shall be treated as binary data (new line in .gitattributes: *.iflw binary). Then a pull will not silently/fast forward merge (and probably break your IFlow), but instead result in a conflict if you and someone else changed it. Then you have to solve the conflict by either keeping your changes or accepting the others (yours will be lost then).
But please be aware that there exists a bug in EGit which does not always pay respect to the gitattributes - with the git command line tool it worked pretty well for me.
I've not implemented HCI yet but I have a question. Have we a "control version" of the objects deployed in tenant as we have in PI with a history of all previously objects?
AFAIK, if it is developed on Eclipse locally, no. Not so sure for those developed using Web UI (as I didn't use that extensively) but I can't remember seeing that.
Thus the reason for this blog! 😉