NetWeaver PI – I want my ccBPM’s back!
I probably won’t make too many friends with this blog, but in any case I wanted to voice my concerns about where PI is…
Firstly, I’ve been using PI (PO, XI and whatever else it’s called this month) since version XI3.0 first became available quite a few years ago. I’ve probably built several hundred interfaces to-date and have always been a staunch supporter of the product. So long as an interface is designed appropriately, PI can do many things that others say it can’t.
The product has come a long way since those early days and the new development environment with iFlows in eclipse is fantastic. Once you are used to it, it’s far better than navigating the Directory tree and managing heaps of receiver determination and interface determinations, and even ICO’s (integrated Configuration Objects).
For stateful processing, PI used to have ccBPM as the integrated BPM tool. It was based on ABAP-stack Business Workflow, although this was largely hidden from the development environment. It definitely had its issues, but it worked and was nicely integrated into the toolset.
Now with the java-only versions of PI we have PO which includes NetWeaver BPM as a replacement for ccBPM (it can do more but here I’m only talking about interfacing). SAP have kind of taken away our ccBPM and then made us pay to get it back as NW BPM.
I’ve been playing around with the combination of PI and BPM (as a PO install) for a couple of months now and have the following points to make:
- Even though PI and BPM are on the one system, they are not integrated well (if at all). In fact in the latest sp the integration is nothing more than the message id.
- BPM uses a very clunky java build process with software components and development components that don’t match nicely with PI’s SLD based software logistics.
- What you would think are core use-cases fail in BPM. For example a web service call from a BPM process step, through PI, resulting in a soap-fault crashes the BPM process. This is a known fault and SAP have been working on a fix for about 4+ weeks now.
- The development environment is clunky. There is a known bug where sometimes your deployed BPM process is always one version behind.
- The BPM history is next to useless for a developer and you constantly need to be checking the java traces.
- For logging purposes during build you need to write your own java beans as the context variable display is not enough.
- I honestly don’t believe it’s a production grade system yet (although I’m sure their are customers using it).
As a staunch supporter of the PI product for years now, its disappointing that it seems to have taken a leap backwards here. I never thought I’d say it, but I want my ccBPM’s back. It just makes it harder to defend the product with the constant barrage of un-informed information from SI’s about how we should use WebSphere, Tibco, or whatever else…
What SAP need to do (and quickly):
- Get rid of the BPM software component, development component, NWDI (yuck!) nonsense and bring it inline to how PI manages objects.
- Integrate the logs asap. From the PI message monitor I want to be able to click a link to view a graphical log of any BPM process.
- Fix up the bugs around soap faults (SAP have this underway, but it was promised a few weeks ago and its not ready yet – how could they release the software in that state?).
- The BPM process history needs to be greatly expanded to show “actual”, “useful” error messages and not just “an error occurred” (that’s rubbish SAP!). Developers also need the capability to fully inspect data elements throughout the process and not just ‘data objects’ that have been placed in the process.
Enough of a rant from me… I’m sure the product will be more integrated and more bug-free as time goes on, but for those of us at the coalface who are evangelising the use of PI/PO, it certainly makes life more difficult right now.
I cannot say I don't agree with many of your points - thanks for putting them on SDN 🙂
my other point is that we need PI to be a simple to us, "one box" middleware so the interface consultants can also work on interfaces on the backends and not do this strange divisions where one group of integration guys are working on PI/PO only and the other integration team is working on ABAP backends because it's hard for customers to find people that do SAP interfaces from end to end within SAP tools...
Nice work and thanks for putting this out there. While I agree with most of what you are saying, I come with the slight differing opinion that we need to embrace this new position and get the product fixed where it is broken, and demand better change and log integration (for example).
I personally think iflows are so much better than cryptic PI development; albeit comm channel templates, and better NWDS support, etc; still need work; so at least one step in the right direction, but software components are ridiculous to introduce into the PI world (especially since so few people know how to truly work with NWDI - IMO).
So what you've done is fight to get PO working, which is the right thing to do. What I don't believe is not learning BPM or iflows because it's clunky. PI consultants need to evolve to PO/BPM usage at least from a Middleware perspective, and hopefully others can come together to look at pattens to include how to do logging, alerting, etc. It's a major risk but what is riskier is working with half a solution!
The real risk with his approach is whether SAP push out a non-PO middleware solution to replace BO but I doubt that will happen in the near future at least (if it does it will be less feature rich than PO for some time).
I don't disagree with the "get on board" message, though I'd just like to add that I *always* design integration to be stateless as much as possible. Sometimes it isn't, but I'd rather keep state out of the middleware and where I feel it belongs: in the end applications. This is independent of the middleware; Tibco BusinessWorks does a great job handling state, but even then it's more of a pain in the long run than wrapping it up in more coarse-grained services, or ditching distributed commit patterns for async push...
So fingers crossed, I won't need to wrestle with the BPM issues Jason pointed out for some time 🙂
"Design as stateless" - that's Rule No.1.
In my opinion ccBPM is not out!
A complete migration to SAP PO is not for everyone. Stakeholder who have largely invested in ccBPM can still use by installing SAP PI (Dual Stack).
SAP PO is not the next version of PI, it is a different solution.
If there is a business or performance case to install SAP PO (JAVA AEX + BPMN +BRM) you can still have one or more instances of it by side of you SAP PI (Dual Stack) installation.
You can choose which platform would be your central integration engine for your landscape and when you configure you scenarios you can choose in which integreation engine to run them.
If you choose SAP PO as the central integration engine, you IP(ccBPM) object will still be visible and can be configured in the Swing Integration Directory. You just need to make sure you select the Dual-Stack Integration Engine where you want them to run.
Now on term of cost, there are several ways to leverage over you TCO. PO requires less hardware, you can downscale the size of the your PI installation.
You can also stop the java instance and keep only the ABAP integration engine running to save resources.
License wise, PI has traditionally be payload bases while PO is CPU based. You can leverage moving non ccBPM payload you PO.
You might not need BPMN and BRM, so maybe an AEX solution is sufficient.
You can also try to negotiate directly with SAP over the license scheme for your landscape. We have had very good experience over it, sometime reducing licenses cost for new products in a 2 digits percentile.
I hope this bring son space to breath over your well based banter.
Juan Francisco Zurita Duque
Thanks for the comments Juan Francisco Zurita Duque (sorry - I'm not sure which is your first name). The roadmap is java-only, so even though "today" you can keep the abap stack around for ccBPM - for how much longer. And for new customers, why would you want the overhead/cost of an abap stack to look after as well.
Lets just hope SAP gets this right asap, cleans up the bugs and produces a fully integrated PI (PO) that we can be proud of working with.
I know that behind the scenes they (SAP) are working on some sort of cloud "HANA Integration" solution so maybe all resources are directed at that, which may explain the PO shortcomings I've listed??? Maybe combining PI and BPM better is just not the flavour of the month at SAP (as opposed to HANA, Cloud, Mobile).
In other blogs I've also seen references to SAP investing in MuleSoft which looks like a great ESB tool with on-premise and cloud options. I've also read a blog where SAP India were hiring developers with Boomi experience which is another interesting integration tool (by Dell) used by SuccessFactors.
Maybe SAP are hedging their bets across all different types of integration technologies... Its certainly confusing.
You can call me Juan 🙂
Regarding the Java Only road map, that is true but if you compare to the ALE(IDoc) technology, is not functionality that can just be taken out.
ccBPM is based over the same workflow engine suppporting other business process in the backend applications. So as long as there is an ABAP system running in you landscape, you will be able to use it to run your ccBPM.
We have worked on a couple of upgrade/migration projects, and we have experience that many of the ccBPM in use are technical (as in required by the interface, not by a business process), like asynch-synch bridges, message aggregators etc.
The learning curve of BPMN and the pattern included have facilitated the effort of migrating those, and many times we just have decommisioned the ccBPM.
There is a visible business value on migrating ccBPM to BPMN from the design, performance, trouble shooting and monitoring.
Regarding other tools such as FSN(which is integration SaaS), MuleSoft , BCM, BODS and other. My opninion is that SAP is buiding a ESB ecosystem of solutions options. To be able to offer the right solution for the specific landscape.
Sounds like you are in the wars... and appreciate the frustrations of working with any new technology, especially after you've spent time and effort getting familiar with an older one.
Perhaps one perspective I can give you is that while I don't disagree with some of your points - the lack of technical fault handling is a particular bugbear that we are all waiting for - the advantage of BPM over ccBPM is that we now have a true business process automation framework that handles both human and system processes equally. Something that has been missing for a long time and has hampered exception handling of integration processes, which in my experience often needs to involve business users to get to a final solution.
Although coming to terms with SC,DC, NWDI and general Java Development does take a little getting used to, the benefits of being able to work on your design offline are not to be sneezed at. My main frustration with PI at the moment is that an offline option is *not* offered - which means you always have to be online even when all you are trying to do is model an iFlow to work out how your 3 tier process design is going to fit together.
Also working with BPMN diagrams I personally find much easier than the old ccBPM blocks, and far more concise.
Just wish I could get over there to give you guys a hand - you know we are trying!
In the meantime please keep raising your ideas in Idea Place - they are really being considered and there a lot of things on the roadmap to make all our lives easier. We are hoping to see an updated roadmap very soon as part of the SAPPHIRE NOW updates.
In the meantime... hang in there, mate.
Hi Jocelyn thanks for your comments. I fully understand what you're saying about BPM - it is a much more powerful and capable system in its various use-cases... But I'm coming from an integration perspective only here. ccBPM was not *great* by any means, but once you got used to its shortcomings it did the job and allowed us to have stateful integration in a way that was fully intregrated into the PI build tools.
BPM does not give us this easy way to have stateful integration anymore. I need a toolset that has this built into the ESB and not just another application sitting on the same server.
As for the SC's and DC's and NWDI... I agree with you again - you can use it once you're used to it but it's still a clunky way to do things I think. I've been doing a bit of NWCloud dev in my spare time and it doesn't have this strange SC, DC concept, nor do other java-based platforms that I know of. We need a BPM-like tool that works with PI's method of modularisation - it should not be necessary to learn all the SC, DC, NWDI stuff.
I also agree here that BPMN is much nicer than ccBPM's diagrams but it doesn't do me much good unless its integrated intro PI properly.
Anyway - these are just my opinions as someone that's been doing interfaces for a long long time now. I 'get it' that BPM does so much more but when I put my integration hat on: its irrelevant. 😉
On the contrary - you've made one new friend in me 😉 .
I largely agree with most things you have highlighted but I wouldn't say:
I can kind of see the bigger picture with the approach SAP is adopting. I would say:
Just a thought...
Thanks for the comment Trevor. If SAP are going to be "take one "step" backwards in order to take 10 steps forward" I fear that other products will end up dominating the space.
No doubt this is a very good composition of what consultants are facing each and every day working with PI.
I am really not sure why but still SAP PI is a middle-ware product to integrate SAP systems with others. SAP has put very little effort to make PI as an "Independent Middle-ware" which can be implemented in any landscape with a close competition with IBM Websphere / Oracle SOA / Software AG Webmethods etc. If you see closely the transitions of different PI versions, its slow.
Still SAP not sure which direction PI should go. This actually creates lot of confusions to the teams who are going to invest their money in PI. I have never found any these type of options with other integration platforms. May be some people think its a flexibility, but I am not convinced.
1. Single or Dual Stack? Why not SAP just chose a single path. If Java, well enough, where actually SAP BPM standing. Still its in immature stage, performance is not up-to the mark and I have seen very slow progress.
2. SAP B2B or Seeburger?
3. ABAP Stack ccBPM or NW BPM?
4. Rest - why Advantco?
Lot of work needs to be done in PI. In an Open market, we have to compare every product. I want to see landscapes where PI will have a Middleware identity and Clients will implement where neither Sender nor Receiver is a SAP Product and consultants would be able to show advantages of this tool over other Middleware leaders.
Thanks for the comments Nabendu... I agree with you that the innovation cycle with PI is very slow...
We are currently working on a message flow monitor to track end-to-end integration scenarios.
To avoid new BPM developments and replace existing ccBPM objects, client is trying to keep ABAP stack. Anyway, we will wait for the best coming next few days in PI space. 🙂
Thank you so much for sharing your feedback with us and the community. Let us try to express our point of view in general and as well address your specific concerns.
SAP NetWeaver BPM as a product has been on the market now for five years. The product can definitely be considered to be stable and mature. Many customers are using it either standalone or in combination with SAP NetWeaver PI in high-volume, mission-critical scenarios.
The Process Orchestration bundle, which is available as of SAP NetWeaver 7.31, has been well received in the market and we see a strong adoption by our customers.
Surely, SAP will continue to invest significantly in the Process Orchestration solution, in addition to new offerings like SAP HANA Cloud Integration or SAP Operational Process Intelligence.
In regards to your comment on the future of SAP's integration technology roadmap, SAP NetWeaver Process Orchestration is our strategic on-premise integration solution. An update to the respective product roadmaps on the Service Marketplace will be published soon.
Regarding the issues you describe, we would like to mention that::
We will definitely not revert back to ABAP/ccBPM, but continue in improving and enhancing our Process Orchestration solution by listening to feedback from customers. We sincerely hope that at some point you will become as well a supporter of SAP NW Process Orchestration as we continue to evolve our products going forward. Thanks again for your passion for our product(s) and your feedback.
Product Management, Technology & Innovation Platform, Integration & Orchestration
SAP NetWeaver Process Integration
SAP NetWeaver Business Process Management
Hi Christian, thanks for your reply... Just to go through some of your points:
- I understand that BPM on its own maybe a mature solution that customers are using - I'm really only talking about PI as an ESB and requiring BPM for stateful scenarios. It here that I find things lacking... and I do hope SAP are working on it of course. 😉
- You have mentioned that we can re-use PI mappings inside the BPM... Well not fully. You can't model any errors with it because BPM calls those mappings as a web service and the wsdl has no soap faults defined. This means you cannot create any boundary event for the mapping. Obviously you can just let the BPM fail, then modify the data or whatever is necessary to kick it off again. But being able to model the error with a boundary event would be handy.
- Are we likely to see a merger of the software logicstics between PI and BPM? (Pleeeeease....)
- Just been told (by OSS) we need to wait another week for the soap fault fix - development are still working on it supposedly - they are also still investigating the version management issue. I'll post back here when we get a resolution on both of them.
These are the kind of blogs that we need to see more on SCN. I truly appreciate sharing these pain points of implementing PO. Thanks a ton!
We ourselves are looking to move to PO since we need to align our infrastructure with the SAP roadmap. Frankly we have never had issues with the ccBPM stack and NW BPM will be a completely new horizon for us to cover.
We need to understand that this is a randomly changing world and technologies are getting overhauled every day.
SAP PO does provide excellent features moving the integration suite to something of a CIS nature. Hence I will urge SAP to consider these feedbacks and help provide the developers a better experience of using the tool.
As for all of us who want to raise concerns, please document them on https://ideas.sap.com/ct/s.bix?c=97F64433-5773-403E-A7C6-D7EB3BD6428D
+1 for sure. Well done Jason on a balanced blog pointing to some issues!
As you know we're also on a single-stack PO system now but don't currently have BPM use cases. So we might scrape by while waiting for some of the improvements I'm sure Voker, Christian & team have planned.
To be fair, I know other integration products also have some of these integration issues between their stateless and stateful parts, so SAP isn't alone in this. But to continue the historical peculiarities of NWDI and other misalignments is not a great game play by any means. It's been well over 18 months since the (in)famous Web Dynpro Java Kiss of Death blog, and things have not been getting better for that technology stack and the oddities it introduced... High time to import some of the NEO approaches into the on-premise stack, I'd say!
It's really interesting to read a blog like this, coming from the PI viewpoint. Here are some of my thoughts on just a few of your points:-
Personally, I would actually like to see ALL of SAP development move to the SC/DC model, including ABAP, PI, etc... This level of consistency would be a positive in my eyes. At least you can now integrate NWDI with CTS+ which is a step forward.
This is definitely an area SAP can improve.
I recently completed a PI & BPM customer implementation where we had no problems with this area of the integration, however we were working with dual stack PI and separate CE so maybe it is a new symptom introduced with PO?
For me, the whole debugging and development stage of an SAP Java based solution can often be a pain for the reasons you mention, however it just takes a different approach to still be productive. I long ago gave up on trying to debug Web Dynpro Java or BPM's for instance and instead rely heavily on logging statements for tracking run time issues. It's a nightmare when compared to how powerful other tools are, such as ABAP, but again a necessary evil.
In short, I'd like to think the increased focus on the Java stack driven by PO will actually increase the toolset for all of us over time - it just might be a bit painful for a while as we get there!
Hi Gareth - just on point (3) - yes they are lower level issues that SAP are trying to resolve right now. You must have a soap fault defined before you can create a boundary event. Thanks for posting.
I thought as much.
It's interesting that this functionality appears broken in PO, whereas I've had it working in older versions of PI and CE working together.
Thank you for mentioning the pain areas in PO, it is very useful to me especially becuase we are planning to upgrade our PI system from 7.11 dual stack.
Your comments are literally scaring me now, because last week I recomended PO 7.4 (SR1) single stack with B2B AddOn (Replacing Seeburger), BPM and BRM.
We only have one ccBPM scenario in our landscape, in general we discouraged developers to use ccBPM. I had high hopes with the new netweaver BPM but with your comments I need to do a second thought on it.
Kindly let me know if you have any suggestions or recommendations to me.
Hi Nagesh, I don't really have anything to follow up with for you unfortunately... Just remember the golden rule - try not to put any business logic into your interfaces... Which kind of means - try to avoid BPM.
The issue I mentioned in the blog about soap faults is resolved in the latest service packs, but nothing else in the latest 7.40 version.
Let me just give you an update on the points discussed in this blog:
For the business user you can either provide a custom history based on Reporting Data Sources, or go for a more advanced solution with SAP Operational Process Intelligence:
Operational Process Intelligence | In-Memory Computing | SAP HANA | SAP
More to follow.
For the benefit of the community can I request SAP to provision the following so that many architects and integration leads can take an informed decision;
1. What are various patterns supported and not supported in comparison to ccBPM vs BPM? A fair assessment will be really helpful.
2. Additional whitepapers, articles or blogs on how to develop various patterns in BPM (ex. Collect pattern, broadcast pattern, serialize etc)
I share the same fear of proposing migrations as we have customers heavily invested on to ccBPM and we lack confidence if most of our integration patterns might fit into a smooth migration.