Motivation for this blog
Without going right to the conclusion of this blog, the performance of cross component Business Process Management (ccBPM) of SAP NetWeaver Exchange Infrastructure is neither poor nor great, it is somewhere in-between. – With this blog I like to respond to customer feedback I have received over the past year: I spoke to many customers that successfully deployed ccBPM scenarios and are very happy with it. Then I spoke to other customers that expressed concern about performance of ccBPM and therefore its viability in the enterprise. With this blog, I like to put things into perspective, share some high level roadmap information as well as highlight expert tips and tricks for implementing ccBPMs the right way. Hopefully this will drive and enable you to either extend the use of ccBPM within your organization or finally do your first prototype using this powerful tool. Since performance is primarily affected by
- SAPs ccBPM engine and
- the actual customer implementation of a BPM scenario
I will focus this blog on these two topics. In (1) I will highlight a new feature for NetWeaver 2004s in Support Pack (SP) 10 and SP19 for NW 2004 respectively. In (2) I like to provide a list of ccBPM design recommendations that I gathered from some of my colleagues in SAPs Regional Implementation Group (RIG). But first, let me talk about our process orchestration tool and the foundation it is built on.
Brief Introduction to ccBPM and its BPEL foundation
SAPs ccBPM engine is based on OASIS Business Process Execution Language standard (BPEL4WS 1.1). Therefore, any process implemented in ccBPM can be converted into standards compliant XML code what supports openness and interoperability. BPEL is an orchestration language. That means that the scope of a BPEL model is the view of one participant only and how this participant interacts with his peers. If one of these peers interacts with yet another system, this is transparent to the BPEL process if handled through a legacy interface by this peer. – Only a choreography language gives a global view at the systems landscape with all parties and all their interactions involved. Lastly I like to point out SAPs leadership in driving and extending the scope of BPEL. Together with IBM, SAP is spearheading the creation of two BPEL standard extensions:
- BPEL4People Extension for human interaction; read the official OASIS announcement here
- BPEL-SPE Extension for sub-processes; read a joint whitepaper by SAP and IBM here
New feature in ccBPM significantly improving performance
As most of you know, major functional enhancements for Exchange Infrastructure are being released every three month. The complete list, including other NetWeaver usage types as well as SPs 4 to 10 (13 to 19 for NW 2004 respectively) can be found at this link. With SP10, a new transactional behavior of ccBPM integration processes has been introduced. Before SP10, after each step in an integration process, the resulting data was persisted in the database, what adds overhead to the process and therefore affects performance. Now, with SP10, the persistence of these steps is configurable. As you can see in picture 1 showing the properties of an integration process step, you can now define if a new transaction needs to be created for a particular step. Not setting this checkmark removes the persistence from a step and results in significant performance improvements. Picture 1: Simple check box config for send, transformation and receiver determination steps For individual process steps, this behavior is available for the send, transformation and receiver determination step. However, as you can see in picture 2, you can also configure this for an entire block and define the start of end of the block as the beginning of a new transaction. Picture 2: Configuration of a block start and end If a process instance at run time fails within a block, you will be able to restart the process from the beginning of the block if you created a transaction at the beginning of the block. While you have to balance the use of longer running transactions with the need of memory, this new capability will improve ccBPM performance significantly.
Expert Tips & Tricks for designing ccBPMs
For this section, I asked my colleagues from the RIG team to provide a list of best practices or issues they have seen in the field at actual implementations. I will approach this topic in two steps:
- Consider all feasible alternatives: Is ccBPM the best way to go?
- How and why to keep ccBPMs as simple as possible?
Step 1 Scenarios with existing and simple ccBPM alternatives Multi-mappings – While ccBPM allows you to easily collect and split messages, you can achieve the same using a multi-mapping and mapping based message split. Check the documentation for multi-mappings. Please note that this does not work in all cases, for example if the target message is an iDoc. Sequential mappings – My colleagues have seen simple ccBPM processes with the only purpose of receiving a message, performing two or three successive transformation steps, and then sending this message off. For such a process you can simply use one interface mapping with sequential mapping steps. Simple lookups – Simple synchronous lookups in a ccBPM process can also be implemented via mapping lookups. This is achieved using the lookup API and is supported by the JDBC, SOAP and RFC adapters. Message collection – For message collection scenarios from single sender, try to collect messages on the sender system and send it out as a single message to your integration server. Sync to async bridge – Previously, implementation of the sync to async bridge (or vice versa) required the use of ccBPM. With SP10, you can now do this within the JMS adapter itself if you do not have requirements for monitoring of this bridge. AUTHOR NOTE ON Jan 24, 2007: One customer confirmed the significant performance improvement for the integrated bridge in the JMS adapter. My “unnamed source” 😉 is reporting an 11x difference! Step 2 Keep the ccBPM as simple as possible Mappings before or after ccBPM – Lastly, if you are implementing a ccBPM, try to do the mappings outside of the integration process itself, as picture 3 illustrates. Of course this approach can be used for every send or receive step within the ccBPM process. Picture 3: Message transformations before and after the ccBPM process Building application logic – As my colleagues report, they have seen ccBPMs in the field where it was used for building rather complex application logic as an alternative to changing legacy backend systems. In one example, multiple synchronous lookups were made either back to the sending system or to the target system unnecessarily. Of course, the performance will suffer significantly! And either you scale your XI system accordingly or you have to move some of this logic back into the legacy systems. Synchronous user interface scenarios – A synchronous user interface expects quick response time for the benefit of the user. Such a scenario is not recommended for ccBPM, especially if you are performing a data update on the backend system. Another issue are connection time outs what could result in compromising the data integrity in your systems. An example is when a user creates a purchase order and the confirmation for creation fails to reach the user due to time-outs for example. As a result, he may create a second duplicate PO. Single message loops – Do not use loops and process single messages at a time. Instead, collect your messages first and combine into one large message with several business documents inside. Large messages – Do not use ccBPM for processing of large messages, especially in collect and split patterns. Start paying close attention to resource consumption when using messages above a size of 20 MB; remember this is highly dependent on your available hardware resources. – And while XSLT mappings should be avoided for large messages in general, this applies especially for ccBPM. Small changes can make the difference – The last example shows a real live ccBPM process improvement in picture 4. Even though this example is not interesting at first sight, the performance difference is significant. In the original implementation, two separate JCBC calls were made to retrieve a sales order document header and the line items. This data then was transformed to the iDoc format of the receiving application using a transformation. Then, using a loop step, one iDoc at a time was sent to the backend application. – This sub optimal design resulted in poor performance caused by following design flaws:
- Separate retrieval of header and line items with the JDBC adapter
- Transformation within ccBPM to combine retrieved header and line items
- iDoc back end connection for one document at the time
Picture 4: ccBPM scenario improved by 20x In the improved version below, both the JDBC call and the transformation were removed from the ccBPM. The retrieval of the source messages is done via a synchronous call with two mappings in the integration engine of XI that generate the required format for the receiving system. Therefore, it is not necessary to add an additional transformation step in the ccBPM. Additionally, the iDoc adapter connecting to the target system is used to handle multiple iDocs inside one XI message. Therefore, it is not necessary to add a loop operation within the ccBPM either. In this particular implementation, the described changes resulted in a 20x performance improvement! Note: A detailed how-to-guide describing this example in more detail is currently in the editorial review process. The title is How To Implement a High Volume Process Integration Scenario and this example is covered in section 4.2.
In summary, ccBPM is not a silver bullet. Use it when you have to and use it wisely, then it is very powerful. If you go overboard or do not design your scenario properly, you have to pay the price, either in extra hardware or in poor performance (what may be acceptable in some scenarios). – It is like me using the brand new Honda minivan for riding around by myself instead of using the stinking old Saturn. While the Honda is much more adjustable, flexible and comfortable, I am paying a hefty premium for gas if I ride in it when not lugging the kids around for pick-up and drop-off 😉 Configuration of the transactional behavior of process steps within ccBPM provides a significant performance improvement. Additional product features for performance improvements are being worked on, both for the XI integration engine as well as the process engine. I will blog about them when or just before they are generally available. One more word about performance: In a world of enterprise SOA, XML message volume will increase exponentially. With all the opportunities of enterprise SOA for composing cutting edge processes, we are taking this challenge of message traffic seriously, resulting in related investment for SAP NetWeaver Exchange Infrastructure. Credits I like to thank my colleagues Jin Shin, Daniel Bianchin, Joachim Orb and Prasad Illapani from our Regional Implementation Group for their contribution of examples and feedback for this blog. Additional resources How To Configure Inbound Processing in ccBPM
- Part I: Delivery Mode (recently published I will provide the location here soon)
- Part II: Queue Assignment (soon to be published)