Technical Articles
Tools and best practices for SAP CPI – a roundup post
Setting up your first interface in SAP CPI can be done really fast. Nevertheless, as in almost all disciplines, mastering a technology takes some time and serious effort and with the right toolset you can boost your efficiency.
My favourite source for learning new things in the field of SAP CPI is this platform itself, the SAP blogs. Unfortunately good articles may get lost over time and just by googling around you may not find all of the hidden gems. Thus I want to share my favourite collection of SAP CPI related articles and tools in this roundup post.
This article is structured into the following sections:
- Tools for SAP CPI
- Best practices, patterns and advanced techniques
- Summary & Feedback
Tools for SAP CPI
With the right tools, work on the SAP CPI system can be considerably simplified. In recent years, a couple of tools have already been published, some of them free of charge, some of them chargeable. The following list gives a brief overview of SAP CPI tools, their pricing model and the added value of each tool.
CPI Lint
CPILint is a commandline tool written by Morten Wittrock which works as a linter for Integration Flows. You can define a ruleset which contains things like naming schemes, use of different mapping types or scripting technologies. Once executed, CPILint reports if an IFlow is compliant to the pre-defined ruleset. This tool helps to keep up the quality of your IFlows.
License/price: MIT license/Free
More information: https://blogs.sap.com/2019/02/01/meet-cpilint/
Installation/usage/download: https://blogs.sap.com/2019/08/31/cpilint-version-1.0.0-is-ready/
CPITracker
With the CPITracker we have a second tool from Morten Wittrock. The CPITracker is a tool which scans the CPI regularly for different system parameters and software component versions like CPI build number, Groovy version or supported XSLT version. If the tracker records a change it tweets the changed values on the following Twitter account: @CPITracker
License/price: Free
More information: https://blogs.sap.com/2017/03/07/introducing-hcitracker/
Installation/usage/download: https://twitter.com/cpitracker
RealCore CPI Dashboard
The RealCore CPI Dashboard is a IFlow-based tool which allows you to monitor your CPI instance (including system parameters like CPU-, RAM- and disk usage), view passwords and log files as also setup an mail-based alerting. The tool was developed by myself for my employer the RealCore Group, but released as open source software.
License/price: MIT license/Free
More information: https://blogs.sap.com/2019/08/19/advanced-monitoring-and-health-check-with-realcores-cpi-dashboard/
Installation/usage/download: https://blogs.sap.com/2019/08/22/how-to-install-and-configure-the-realcore-cpi-dashboard/
TSIM (The Scheduled IFlow Monitor)
The TSIM is another IFlow-based tool (=the whole tool is built within an IFlow) which allows you to get an overview of all IFlows which use an timer/scheduler. This tool, developed by Ariel Bravo Ayala, comes with a webview as also with an data-based REST backend, so that you could use the scheduling information in third-party/your own tools.
License/price: MIT license/Free
More information: https://blogs.sap.com/2019/10/28/tsim-the-scheduled-iflows-monitor
Installation/usage/download: https://github.com/ambravo/tsim/tree/master/bin
FormatConversionBean
The FormatConversionBean was initially developed for SAP PI/PO, but meanwhile also released for SAP CPI. The tool was developed by Eng Swee Yeoh and serves as a format conversion library which can be used when developing Integration Flows. It allows to convert data between different formats like XML, JSON, Plain or Excel with ease.
License/price: MIT license/Free
More information: https://blogs.sap.com/2018/09/04/formatconversionbean-arrives-in-cpi/
Installation/usage/download: https://github.com/engswee/equalize-cpi-converter
CPI Logs
The tool’s name might be a bit misleading, but the tool itself is worth a look. The CPI logs tool was written by Ivo Vermeer and allows to download message attachments from the SAP CPI. Thus you can easily/automated download previously saved payloads from your IFlows. The tool is written in NodeJS. So you better have a Node version installed, before testing the tool.
License/price: MIT license/Free
More information: https://blogs.sap.com/2019/02/05/cpi-attachment-dowloader/
Installation/usage/download: https://www.npmjs.com/package/cpi-logs
SuperEasy Log Viewer
The SuperEasy Log Viewer is another NodeJS-based tool. It was written by Fatih Pense and serves as a downloader for SAP CPI “ljstrace”-log files. Thus it serves the same function then the above mentioned RealCore CPI dashboard. If you simply want to get the log files and don’t need the other features of the RealCore CPI dashboard, you should go with this tool.
License/price: Free
More information: https://blogs.sap.com/2018/11/02/supereasy-log-viewer-for-cpi-open-source/
Installation/usage/download: https://github.com/medepia/supereasy-logviewer-for-cpi
SuperEasy Partner UI
The SuperEasy Partner UI is another tool written by Fatih Pense. Like SuperEasy Log Viewer this tool comes as NodeJS application, too. With SuperEasy Partner UI you can view (and in the future manage) all the entries of the SAP Cloud Platform Partner Directory. If you work with the partner directory, this tool is a real bonus, because there is no official graphical interface and the partner directory is only accessible via API.
License/price: Free
More information: https://blogs.sap.com/2019/11/26/supereasy-partner-directory-user-interface-for-cpi-open-source/#
Installation/usage/download: https://github.com/fatihpense/supereasy-partnerui-for-cpi
Figaf IRT (Cloud)
Figaf IRT is one of the most powerful / extensive tools in this list. It was developed by Daniel Graversen and can assist you in the following areas: Monitoring of CPI (system- and message-based), alerting, testing of interface (including testcase recorder), documentation, GIT-integration for versioning/collaborating on CPI content and devops. The tool comes as an Java application which either can be installed on premise or bought as SaaS from Figaf’s cloud solution. Besides SAP CPI also SAP PI systems are supported by Figaf IRT.
License/price: One time fee (on-premise), montly fee for Cloud-based solution
More information: https://blogs.sap.com/2019/07/01/how-figaf-can-help-manage-sap-cpi/
Installation/usage/download: https://figaf.com/tools/sap-pi-cpi-test-and-lifecycle-tool-figaf-irt/
SAP App Center: https://www.sapappcenter.com/apps/44233
Int4 IFTT
The IFTT from Int4 (the company for which, among others, Michal Krawczyk and Eng Swee Yeoh work) is the most mature tool in the list. Over the last 9 years it has grown to an impressive testing solution, which enables you (once set up) to automatically test all of your interfaces. And it doesn’t stop at SAP CPI, but also supports SAP PO and SAP ERP – enabling you to run end-to-end testcases. With its sender-/receiver-virtualization system it allows you to test interfaces, even if one of the communication partners isn’t practically involved in the test. If you want to learn all of its features, I can recommend you to watch the IFTT openSAP course.
License/price: One time fee (SAP App Center)
More information: https://blogs.sap.com/2019/10/06/migration-to-cpi-with-int4-iftt/
Installation/usage/download: https://www.int4.com/sap-testing/
WHINT Solutions
WHINT Solutions is a set of different tools for SAP CPI, feveloped by whitepaper.id GmbH. The WHINT Solutions provide tools for more transparency (PDF documentation and reporting tools) as well as tools to allow monitoring (Excel-based interface catalog and message volume reports) and alerting of interfaces. All solutions are delivered as Integration Content, running on the your/the customers tenant or as a service without the need of an additional server. Same as for Figaf’s IRT tool the WHINT solutions support also SAP PI/PO.
License/price: One time fee / yearly subscription
More information: https://www.integration-excellence.com/interface-management/
Installation/usage/download: https://www.integration-excellence.com/whint-solutions/ifm-sap-cpi/
After this little excursion into the world of SAP CPI tools, let’s look at specific blog posts now…
Best practices, patterns and advanced techniques
As already announced in the introduction, I consider the SAP blogs platform as one of the most important sources for learning new things. Unfortunately, many good articles often go under or are not discovered because it is not clear what you have to google/search for.
The following (incomplete) list is a collection of in-depth articles that highlight specific topics, techniques or areas of SAP CPI. For a better overview I grouped the articles into meta categories:
“Best practice” and patterns
- “Eng Swee’s tips for CPI development” / Author: Eng Swee Yeoh
- The article deals with his 8 favourite tips for SAP CPI development. (Eng Swee names it “tips”, others would say “best practices”.)
- “Cloud Integration – Configure Asynchronous Messaging with Retry Using JMS Adapter” / Author: Mandy Krimmel
- Learn how to implement a restart feature for async messages in SAP CPI by use of SAP Enterprise Messaging service and the JMS adapter.
- “SAP CPI: Asynchronous Messaging Best Practices” / Author: Adam Kiwon
- An overview post, that explains to you which sender adapter supports (or even not) which retry/re-process modes.
- “I *heart* Groovy mapping” / Author: Eng Swee Yeoh
- A profunded talk on the virtues of using Groovy scripts in SAP CPI including examples for the most common tasks including a note on unit testing.
- “HCI: Using Eclipse WSDL Editor for SOAP-based integration” / Author: Eng Swee Yeoh
- Do not let the HCI in the title confuse you. Until today there is no WSDL editor in the CPI, which is why this article has not lost any of its actuality.
- “SAP HCI – Security FAQ and Checklist!” / Author: Sunita Sudarshan
- In this article Sunita lists 14 points round about security questions in the context of SAP CPI. Even if the article is a little bit older, most of the information is still valid and read-worthy.
- “Use Self-Signed SSL Client Certificate to connect SAP backend to SCPI with Custom Domain configured” / Author: James Taylor
- In this post James shows how to connect an SAP ERP with the CPI, using client certificate-based authentication while using self-signed certificates. This comes handy if you don’t want to pay an CA for getting certificates from them.
- “Modularising CPI Groovy scripts using POGO” / Author: Eng Swee Yeoh
- This articles explains how to follow the POGO priciples to split your Groovy function/logic for sake of a better re-usability. Must read!
Productivity and tricks
- “Rapid Groovy Scripting in CPI: Dynamic Loading and Execution of Script Code at Runtime” / Author: Vadim Klimov
- This article explains how to load and run Groovy scripts at runtime. Combined with the next article in the list, you can really increase your development speed.
- “Rapid Groovy Scripting in CPI: Tips to Improve Developer Productivity when Rerunning Dynamically Loaded Script” / Author: Vadim Klimov
- Learn how to combine the dynamic Groovy script execution from the article above with an IDE like IntelliJ to speed up your Groovy script development process.
- “Pimp My Groovy – boosting CPI Groovy developments with IntelliJ IDEA” / Author: Eng Swee Yeoh
- Read this one if you want to learn how to setup IntelliJ as your new favourite Groovy scripting IDE, including some advices on local standalone- and unit testing.
- “Get to know Camel’s Simple expression language in CPI” / Author: Morten Wittrock
- A good overview on the subject of Apache Simple expressions, which shows the essential skills and Simple syntax to you right to the point.
- “Using Camel’s Simple in CPI Groovy scripts” / Author: Eng Swee Yeoh
- Apache Simple is the expression language you can use in nearly every IFlow element when the input box says that you can enter “Expressions”. But did you know, that you can parse/evaluate Apache Simple expressions in Groovy scripts, too?
- “Blog series: HTTPS tracing and debugging – A simple way” / Author: Ariel Bravo Ayala
- A blog series consisting of 5 articles dealing with analyzing / recording HTTP connections. Although the articles are not exclusively CPI-related, they should be part of the standard repertoire of any integration developer.
- “How to stop messages in CPI manually” / Author: Raffael Herrmann
- A small dive into SAP CPI’s internals, which teaches you on how to cancel/stop SAP CPI messages in a manual way.
Testing and QA
- “Run Unit test for SAP CPI easier” / Author: Daniel Graversen
- In this article Daniel explains how to run unit tests for your Groovy scripts by help of his tool “Figaf IRT”. You may have seen in the list above that this tool isn’t free for productive environments, but for the simple task like this you can use the free community edition with some restrictions.
- “CPI’s Groovy meets Spock – To boldly test where none has tested before” / Author: Eng Swee Yeoh
- Learn how to setup an TDD environment that allows you to run unit tests for Groovy scripts/-functions by use of Spock.
CPI insights – deep dive into CPI
- “Performance Analysis of Runtime Node in SAP Cloud Platform Integration: Collection of Thread Dumps” / Author: Vadim Klimov
- If the ljstrace-file doesn’t deliver enough information, you might want to sneak into Java thread dumps. Vadim shows you how to do so.
- “Performance Analysis in CPI: Memory Consumption and Garbage Collection Analysis” / Author: Vadim Klimov
- Another brilliant article of Vadim showing you how to analyze memory usage and garbage collection logs.
- “CPI Runtime Node Instrumentation: Dynamic Loading of a Java Agent Using Attach API” / Author: Vadim Klimov
- In this third article Vadim once again shifts the limits of the feasible and explains how to attach Java Agents to the SAP CPI’s JVM on the runtime node(s) to collect metrics or implement additional functions like custom tracing.
- “Exploring CPI’s filesystem’s content” / Author: Ariel Bravo Ayala
- This blog post is a deep dive into SAP CPIs file system starting with a good explanation how to access the file system due to a combination of Groovy scripting and IFlow objects.
- “CPI: Exploring the hiden and hideous /Operations url” / Author: Ariel Bravo Ayala
- In this article Ariel shows us the results of his efforts in reverse engineering the CPI /operations API which is the key driver for the Web IDE we use in our daily work.
- “Dark Side of Groovy Scripting: Behind the Scenes of Cloud Platform Integration Runtime” / Author: Vadim Klimov
- Learn how to read JVM properties, reverse engineer SAP CPI’s internal classes and look into actual SAP CPI implementation. The best – all this is just some lines of Groovy code away.
APIs – automating CPI
- “Cloud Integration – Remote OData API’s for Integration Flows” / Author: Archita Sarangi
- Learn how to use the official SAP CPI apis. With the knowledge of this article and your inspiration you’re ready to go, to implement the next tool for the list of tools above.
Other sources / roundups
- “Introduction to Developer’s Corner” / Author: Gayathri Narayana
- A *huge* list of SAP CPI blog posts. Lots of basic stuff, but also some in-depth articles. Take a coffee (or two to five – as I said, its a huge list) and try to find the hidden gems.
Summary & Feedback
Now we have reached the end again. First of all, three insights that I gained while writing the article:
- Writing an roundup article isn’t less time consuming than writing about a specific/new topic. (Even if you do not have to write down new findings in roundups.)
- Eng Swee Yeoh is disproportionately represented in the list. (Sorry, but the articles are simply to good to be not mentioned.)
- Vadim Klimov is the king of extra long blog titles.
And now? Now it’s your turn. I did try very hard, but I am pretty sure that I have probably missed a tool or another “must read”-blog post. If you have an idea or a suggestion, what is missing in the listings, just write a comment.
I hope you enjoyed reading this post and if I could point you to at least one article/tool you didn’t know, than this roundup did serve its purpose. I’m closing this one with a kudos to all authors and tool developers listed in this article. Thank you for your commitment!
Hi Raffael
Great work compiling this list, it must have taken lot of time making.
Great post, and thanks for mention the Figaf Tool, you did forget our Git integration. We are in the App Center (https://www.sapappcenter.com/apps/44233)
One of the posts that I visit most often is Morten Wittrock Post about the camel simple expression. Knowing that will help save some time.
https://blogs.sap.com/2016/11/25/get-to-know-camels-simple-expression-language-in-hci/
And by the way, most of the people (with multiple blogs) on the list are here in Barcelona.
Daniel
Hi Daniel,
thanks for your comment and your article suggestion. I placed Morton's article onto the list.
Did I got you right? You didn't mean another/a second Figaf tool for Git, but that I forgot to mention Figaf IRT's GIT capabilities? I just added it to the description of your tool. Let me know if it's ok for you now.
Regards,
Raffael
Hi Raffael
Yes that is correct what you added.
We just have "one" product, with multiply deployment options.
Hi Raffael
This is awesome work coming up with such a compilation. I definitely do not underestimate the amount of time and effort you took to come up with it - kudos! I'd love to see this post being kept up to date as time progresses as it's really worth place in every CPI developer's bookmark.
Also, I'm really enjoying the stream of high quality content coming from you - keep them coming! 🙂
A couple more that I would include from my own "favourites":-
First one is Ariel Bravo Ayala's blog series on HTTPS debugging/tracing. I love in particular deploying mitmproxy on CF and using it as a reverse proxy - it has saved me a lot of time whenever some HTTP issues happen in a CPI adapter and I need to know what is actually happening at the HTTP layer.
https://blogs.sap.com/2019/01/01/blog-series-https-tracing-and-debugging-a-simple-way/
The second one is a more general one - SAP Introduction to Developer's Corner which is in some ways a compilation like yours. I keep it handy in my bookmark whenever I need to refer to "official" content by SAP folks around specific CPI features. In this aspect, I applaud the many SAP authors that have continue to regularly publish content around CPI to keep all of us up to date about the changes happening in CPI.
Thank you again for including my work in this post - it is an honour to be listed among so many other integration developers/architects/practitioners/aficionados that I have great respect for.
Regards
Eng Swee
Hi Eng Swee,
thanks for the positive feedback and your kind words. I'm really happy that you like the idea of the post. I added your suggested article to the list. The series of articles by Ariel Bravo had completely escaped me so far. Good that we talked about it - the blogs are really great! 🙂
Regards
Raffael
Hi Raffael,
Great work! I would like to recommend the following articles and solutions to make your posting even more complete:
On Asynchronous Messaging: https://www.integration-excellence.com/sap-cpi-asynchronous-messaging-best-practices/
WHINT Solutions for SAP Interface Management (Transparency & Control): www.whint-solutions.com
Best Regards, Adam
Hi Adam,
thanks for your feedback and input. I added your suggestions to the article right now, but why so shy? It's good content - you could have just written that it's your work that you suggested. I don't care if someone promotes his own content as long as it's valuable for the community.
Regards,
Raffael
Yes that one was missing.
Great work, Raffael, and very nice summary of materials and tools! And special thanks for nominating me for the longest blog posts' titles author. 😉 Jokes aside, it is very pleasant to see that work of CPI fellow developers who commit their time and efforts to make contributions to community, attracts attention and interest.
I agree with Eng Swee: it would be really good if in the future, you could update this blog post from time to time with new links and references to other materials that you or community think add value or are worth being aware of in the CPI area, it could then be used as an entry or landing page from CPI developers and for CPI developers. (though, I realize it will actually require continuous efforts to go through new blog posts, and keep this one up to date).
Hi Vadim,
Thanks for the feedback. Keeping this blog up to date is on my todo list. I won't add any CPI related article in the future, but those which go a little bit deeper into details/those with a special topic.
Also looking forward to receive constant input from the community with new suggestions for the list above. 😉
Regards
Raffael, thank you for your efforts in putting this all together.
Hi Raffael,
Good blog.Thanks for the sharing the information.
Regards
Bhargava Krishna
Hi Raffael,
Thanks for writing the post, it really helps having all info in one place. And also thanks for the mention of my personal tool.
Kr.
Ivo
Awesome work Raffa, Thanks for putting this into one place.