SAP Cloud Integration: move away from those IDoc messages!

[PiotrRadzki]

Introduction

I’m coming back to you with a new Blog because we need to talk about IDoc messages! 

This time I will focus more on the design and architecture of integration solutions that we as Architects and Developers create on SAP Cloud Integration, SAP PO or any other integration middleware that you use in your Organization. As SAP Integration Suite with all services included is already quite a mature iPaaS, you should already now consider migrating your legacy integrations from SAP PO (PI, XI or Business Connect if you are still there!) or any 3rd party middleware that does not fit to your Integration Strategy. At the moment of writing this blog, SAP was recognized by Gartner as a Leader in Magic Quadrant for Integration Platform as a Service with its SAP Integration Suite. I think it’s another argument to use such a leading platform for connecting to modern API’s! 

If you are already planning migration to Cloud Integration in your Team or Organization, then you can make use of multiple guidelines and tools that I listed at the end of this blog. SAP Community provides already a lot of best practices that should simplify this activity and with use of SAP inhouse or 3rd party tools and/or Consulting services, you can make this process really smooth. If you are still not sure how to design the correct architecture of your systems from the integration perspective, then let’s deep dive into Integration Strategy and define your path to the cloud or hybrid integration landscape. Proper Integration Management with modern tools is very important and can save a lot of cost and hassle - I’ve put ISA-M related references at the end of this blog as well. 

This Blog does not cover scenarios like a migration of B2B add-on interfaces on SAP PO to SAP Integration Suite Trading Partner Management, or ditching and redesigning interfaces for WM systems that work with IDoc messages. It’s more of an overview and a trigger for a discussion on how to modernize a technical landscape, starting with analyzing good old IDoc interfaces and listing options to move and implement API first strategy.

Rationale

Integration of various SAP systems with use of IDoc messages is still very popular and sometimes even the backbone of the interfaces developed between SAP ABAP stack systems and 3rd party. This is very common in Logistics or Production focused organizations. Considering migration or modernizing your interface is a great opportunity to review the technical solutions implemented some time ago and to think about potential migration to more modern and robust solutions. 

Your IDoc scenario might be complex, what looks as a simple IDoc integration might be just the tip of the iceberg. Using the opportunity of migration activities you can check and refactor the solution. Maybe you can move away from customized and extended IDoc messages and logic behind it? It might be worth checking out, maybe an alternative API provides better functionality? Maybe SAP provides an improved way to handle the process? Maybe your highly customized IDoc might be replaced with a standard or almost standard SOAP API which will have just a few extensions. Additionally, you will gain transparency and the cost of maintenance might be lower due to much more user friendly structures and field-names in the modern API's compared with IDoc message structures.

In some rare cases (but it will be happening more often), your IDoc might be even deprecated and not supported anymore. In this case, the need for migration from such a solution is a must. So, what are the alternatives and how to approach such analysis? This is especially relevant for all S/4 HANA transformations, you will face a need to or must to use the new way of doing things. It’s better to prepare yourself upfront before your IDoc becomes obsolete.

Alternatives

Below, I've listed standard ways (SAP delivered) to replace IDoc based integration's. IDoc can be also replaced with a fancy custom solution but here in this blog, I’m going to describe standard options only, so you can keep the core clean!

SOAP API

SOAP API's are great alternatives to IDoc messages. You may know them well under ABAP Proxy name as they are "hiding" behind SPROXY transaction code. On S/4 HANA as well on S/4 HANA Cloud, the list of provided SOAP API's is long and after some investigation it gives a nice overview of what can be achieved and which integration processes based on IDoc messages can be refactored with this type of API. Most of those SOAP API's go way back to SAP ERP and ECC systems, but they are being continuously upgraded. New SOAP API's are being developed and provided by SAP at present as well. . Every new version of S/4 HANA provides features and improvements for SOAP API's. This also gives us certainty about a well established solution and a code in it as well as a future proof approach of migrating IDoc based integration to SOAP API's. The advantage of the SOAP communication method against SAP S/4 HANA is that it can support asynchronous or synchronous communication, so in terms of functionality appropriate API can be found to provide the expected result based on an initial IDoc requirement. Another advantage of SOAP API's is a really good enhancement framework that gives flexibility of enhancing the ABAP Proxy structure with custom extensions and implementing encapsulated custom logic.

SOAP API's on S/4 HANA: SOAP | SAP S/4HANA | SAP API Business Hub

Enterprise Services SAP Help Page

SOAP API's on S/4 HANA Cloud: SOAP API | SAP S/4HANA Cloud | SAP API Business Hub

OData API

OData API is a flagship connectivity for S/4 HANA integration domain. For some of the SAP landscapes it might be quite new option coming up using the opportunity of S/4 HANA transformation. But a little bit more experienced integration specialist might recall SAP Gateway which was already introducing OData model and API's into SAP ecosystem. This was implemented as a separate system next to your SAP ERP/ECC box. With the current approach on S/4 HANA the whole OData framework is built into the system itself and integrated closely to the processes of S/4. With an impressive list of available CRUD (Create, Read, Update, Delete) operations on various Objects, it provides an interesting alternative to legacy integration solutions implemented so far. Of course, we need to remember that essentially the OData API's are used for synchronous communication therefore your IDoc based scenario might require deeper  analysis and redesigning effort to use OData. It's important to mention that OData API provides broader  access to the whole data model of an application/system itself that you integrate with using the expand and navigation attributes compared to SOAP or REST API's. On S/4 HANA and S/4 HANA Cloud we are getting API's that support OData V2 as well V4. V4 provides a significant improvement to the protocol itself and query methods. 

Just check below the available API's and try it out yourself!

OData V2 on S/4 HANA ODATA V2 | SAP S/4HANA | SAP API Business Hub

OData V4 on S/4 HANA ODATA V4 | SAP S/4HANA | SAP API Business Hub

OData V2 on S/4 HANA Cloud ODATA V2 | SAP S/4HANA Cloud | SAP API Business Hub

OData V4 on S/4 HANA Cloud ODATA V4 | SAP S/4HANA Cloud | SAP API Business Hub

REST API

In context of an S/4 HANA transformation the REST API's are not very popular as of now, currently there are only few REST API available for that system but in case of moving from an IDoc  based integration towards modern API, it is worth considering especially because off it's asynchronous processing mode which can be used in "fire and forget" scenarios. The library of REST API's on S/4 HANA is not very rich, however I hope SAP will go into that direction, especially for some popular asynchronous scenarios where a heavy SOAP envelope is not required and it would be a good candidate for lightweight REST service. In case of S/4 HANA Cloud the list of available REST API's is a little bit longer but it focuses mainly on API related to SAP Intelligent Trade Claims Management processes. It is worth observing this category and checking if something new comes up.

REST API's on S/4 HANA: REST | SAP S/4HANA | SAP API Business Hub

REST API's on S/4 HANA Cloud:  REST | SAP S/4HANA Cloud | SAP API Business Hub

Events

Events are quite new "objects" in the SAP ecosystem although the event itself has been known for a long time in the integration space. With S/4 HANA we are getting powerful access to Events that act as notifications and trigger for iPaaS to start various integration processes. Events in fact might be quite close in their nature to outbound IDoc messages which as we know can be triggered by change pointers and also act as event itself. With a highly promoted API-led and Event-Driven Integration approach (not without reason) the Events are an interesting approach and direction to pursue. So far Events are available for S/4 HANA mainly, we have also similar events functionality in SuccessFactors, SAP Subscription Billing or SAP Marketing Cloud as well S/4 HANA Cloud Business Events but we should expect more and more Event based integrations available in the SAP integration space.

We should not forget about the tools and services that support this Event strategy and approach. SAP Event Mesh has been available for some time to help you in orchestration and management of events between systems and applications. SAP Event Mesh acts as a centralized approach to event driven architectures and use cases. It is a fully managed cloud service that allows applications to communicate through asynchronous events - you can position it as your Event broker for publishing/consuming business events and seamless connectivity. It's a service deployed on BTP that can orchestrate SAP and non-SAP ecosystems and distribute events across your landscape.

We also have the latest product in SAP family called SAP Integration Suite Advanced Event Mesh, which is closely coupled with the iPaaS from SAP and provides an Event-driven architecture framework as a service for bigger messages and high latency. Advanced Event Mesh provides an Event Broker, a place to manage a mesh of brokers and whole Event Streaming, Event Management and Monitoring capabilities with full Event Portal features and functionality. More insights: SAP Integration Suite, Advanced Event Mesh (cloud.sap)

There is a nice Blog which describes differences, features and a little bit about the roadmap of both products: SAP Integration Suite, advanced event mesh vis-à-vis SAP Event Mesh and SAP Integration Suite. | SA...

Business Events on S/4 HANA Cloud Event Object | SAP S/4HANA Cloud Business Events | SAP API Business Hub

Events on S/4 HANA: Event Objects | SAP S/4HANA | SAP API Business Hub

Wrap up 

This is it for now! I tried to keep it short this time as this topic is so wide and deep so it's not possible to describe every and each potential scenario and alternatives. I hope we can have discussion based on that and discuss any other alternatives that you can come up with. I know I’m touching the top of iceberg here! Every SAP installation, every landscape and every integration is different and has its own critical aspects and nuances. It is definitely good to review the integration solutions that are running for years already and check what kind of new opportunity is available on the market to support business process. Highly customized IDoc are hard to maintain and cost of maintenance is raising rapidly with every new functionality or field added so my recommendation is to at least check if you can replace it with standard API's available and I hope this blog will bring some interesting findings for your specific scenario. The same approach might be applicable not only to IDoc based interfaces but to all custom APIs  (FM's, BAPI's, SOAP services, ABAP Proxies) developed in your SAP box over the years in SAP ERP/ECC systems - it's worth to take a look and review how they work and if custom code really necessary nowadays.

Please remember that there is always exception to the rule and specific IDoc based integration might be not good candidate for refactoring, there will be always some exception to the rules that you define in you Integration Strategy - and that is fine!   

Hope this Blog will be just a beginning of a journey to modern API's in your Organization and trigger necessary discussions.

What's your opinion about moving away from IDoc based process integrations? Do you see other alternatives? Let me know in comments and lets have a discussion about it!

[EDIT] I think it's also worth to add interesting point of view from michal.krawczyk2 about replacement of EDI/IDocs with API's. B2B/TPM scenarios are not directly a topic for this blog but it's worth to consider inputs in this blog as well: SAP EDI with API instead of IDOCs - is a design flaw! | LinkedIn

References 

Migration to Cloud Integration: 

Elevate from SAP Process Orchestration and Modernize your integration platform with SAP Integration ...

SAP Process Orchestration to SAP Integration Suite Migration | SAP BTP Transformation Program

Enablement guide for SAP Integration Suite Migration Assessment Capability | SAP Blogs

Migration tool in Cloud Integration Capability of SAP Integration Suite | SAP Blogs

Migration Tooling | SAP Help Portal

Integration Strategy: 

Exploring SAP’s Integration Strategy: Free eBook Available Now | SAP Blogs

SAP Integration Solution Advisory Methodology: Template version 4.0 available now | SAP Blogs

Integration Management: 

SAP Interface Management How-To Guide | Book and E-Book - by SAP PRESS (sap-press.com)