Lately I’ve been writing in this blog about EDI and IDoc messages and transactions. While I find this a fascinating subject, at times I’ve felt as if I’ve been getting a little bogged down in the details of one piece of an overall puzzle.
The puzzle of course is the integration of systems, applications, and communications to produce a seamless automated process that completes a key transaction between two business partners.
There’s a lot of gobbly-gook around this subject of integration. But when you get down to it, past all the jargon, it’s still only about moving data from point A to point B. This movement is critical because it’s always part of a processing cycle that completes a business transaction between two companies.
It’s a little more complicated, of course, because it involves automated routings, transformations, and communications that can depend on multiple protocols and security requirements. More importantly, this data is the lifeblood that binds the two companies together in a mutually beneficial business relationship that can also extend to other companies and organizations, supporting a complex web of commerce.
The key is that the integration process be transparent to the users at both ends, who are generally business people whose jobs and inclinations are not to tinker with technology. It needs to just work without the users having to think about it.
Message and transaction standards are a critical part of the package, one of the tools to be used to simplify and standardize development and to build a process that works reliably and is invisible to the user. But there are also other tools and standards, and communications protocols, that must be brought into play.
The most important piece of the puzzle, however, is the ability to visualize how all these pieces can be brought together to form a seamless, generic architecture, an approach, that can be used to plug in all similar interfaces in a consistent and coherent manner.
Your Mind On Brainware
This is real software, in the human brain, developed through experience and creativity. I like to call this brainware Integration Thinking.
I’ve been groping towards integration thinking for quite a few years now and this is largely reflected in the development approach I describe in my book from SAP Press Architecting EDI with SAP IDocs.
In a perfect world, there’d be only one business ecosystem that a company could plug into to build its integration architecture. The SAP ecosystem, with all its business applications and Netweaver integration stack, is about as close to that perfect world that presently exists.
But in the real world, companies make different purchasing decisions about software and systems based on a lot of different criteria. Which means that even among SAP customers there are a lot of different middleware systems out there that are not likely to be replaced by SAP PI anytime soon, if ever.
And each developer has to deal with the tools that are on hand at his client or employer’s site. So you need to push aside whatever your preferences might be and approach integration not as a systems issue but as an architectural and strategic challenge. That means knowing your tools and how to make them work together to produce a generic plug and play architecture that serves your client’s integration needs, whether with his business partners or his internal systems.
This is integration thinking.
Now I’m in a rather unique position: I work both sides of the integration street. I’m an certified ABAP programmer and a Sterling Integrator (SI) developer.
My more than 15 years of SAP experience has given me the opportunity to code across virtually all ERP modules and to write just every type of ABAP program that’s out there. I can build a custom IDoc, code the function module behind it, and configure SAP to either generate and send it or bring it in and post it.
Data has always been my passion so I’ve focused on the wide range of conversion and interface tools in SAP, from custom BDC programs and file extracts to the LSMW, the HR Interface Toolbox, IDocs, BAPI’s, RFC’s, XI/PI, and so on. I’ve had wonderful experiences with them all.
But I’ve also worked extensively with Sterling Integrator, formerly GIS, on large scale domestic and international SAP EDI projects for more than five years. And I’ve done it all with SI, from installation to reverse engineering the database, trading partner maintenance, mapping, service and adapter configuration, BP (Business Process Model) development, custom coding with Java objects, communications, and creating and maintaining links with SAP.
What’s interesting about my work with Sterling Integrator is that it’s always in conjunction with development and configuration work in SAP.
Between A Rock And A Hard Place
What’s frustrated me about working with both systems is that I’ve often found myself caught between two teams and two groups of managers: SAP and EDI. There were even times when the auditors looked askance at my cross-platform development efforts … but of course, in the real world, nobody ever tries to fudge anything with the auditors, do they?
So I occassionally found myself in a position where I was formally not allowed to do any ABAP programming but where in fact the most complex SAP integration jobs were assigned to me: ABAP, SAP configuration, GIS mapping and BP development, communications … everything.
In the end, the practical needs of the business always triumphed over the short-sighted restrictions, whatever their cause might be, that sometimes threw obstacles in my ability to work in both systems at the same time.
So from the beginning of my work with Sterling Integrator, I was always immersed in the end-to-end development of interfaces between SAP and the external trading partners or internal systems, even if at times this was unofficial but tolerated because, as a client once told me, the systems I build always work.
The result is that I’m a multi-lingual progammer who approaches every new interface with insight into what is required in each system and what tools I need to deploy to make it work. And while I would prefer to work with XI/PI, and have promoted it in the past, I’m essentially agnostic about whatever middleware systems the client happens to have, as long as I can make it work with SAP.
The integration thinking kicks in when I step back, look at the problem, and determine whether it’s a one-off interface or part of a cluster of transmissions that may be to different partners or systems but that share common characteristics that can be accomodated in a generic process flow, with the emphasis on “generic”.
Finding That Generic Sweet Spot
At that point, you can design architecture and define an approach that uses the tools and standards in both SAP and Sterling (or whatever middleware system you use) to build a generic plug and play integration system for all similar interfaces, that makes decisions about specific functionality and routing based on data that is available at runtime.
Once you have your approach, you can begin coding and building your architecture, with an eye to how you would integrate a future trading partner or system.
This is integration thinking … brainware that develops through deep knowledge and experience of all the tools available to you, whether systems, message standards, programming languages, databases, XML data, application output, documents, communication protocols, and so on.
It may take years of hard work to build up this brainware capacity. But you can cut down that time by freeing yourself of the restrictions of looking at an integration problem from the point of view of one system or process, or as a point to point transmission or call, and by trying to understand all the resources and systems available in your organization and how they can work together to form a seamless processing model.
Like I said, there’s a lot of gobbly-gook and jargon around this topic of integration. You may be saying that this is more of the same. So we need to look at a practical, concrete example.
I’ve been working lately on payment reconciliation interfaces to banks and this got me thinking about this whole issue. We can get a better understanding of what I’m trying to say by looking at how this integration thinking translates into a specific interface architecture.
We’ll get back to my series on EDI and IDoc message standards after we’ve worked through these thoughts about integration thinking. This brainware, after all, is the context for all of my integration efforts.