An approach to handle total record counts in IDOC to JMS scenarios

[Former Member]

Hello Folks,

I intend to share an approach we put to use in one of our client's PI environment, where in we are integrating their core SAP AFS solution with their legacy systems.

The technical challenge in this case is to append a total record(at the end of Business day) to a file delivered to a legacy system. This total record will have to be the last record in the file and should give an exact count of the total number of records triggered from the source SAP system for a particular interface on that business day.

I will explain furthur using one specific scenario as a reference.The high level design goes like this, Daily Shipments(/AFS/SHPMNT05 IDOCs) are configured to be triggered to PI all through the day(Autosys job scheduler triggers IDOCs collected in SAP every 3 hours(just an example time interval)). These IDOCs are transformed to a legacy format in PI mapping runtime and delivered to JMS queue(JMS content conversion is done to transform data to a CSV/fixed length format) using the PI JMS adapter.

The legacy system pulls all these messages off the queue at about midnight, every day, and then creates(runs a third party utility on MQ) a file out of it in AS/400 and then processes the file contents. This legacy program expects a trailer record as the last record in the file. This trailer record should contain the total count of materials shipped that particular day.

So, we had to make sure, we deliver the control record as the last message to the JMS queue. We did debate on how to achieve this and decided to follow an approach that we believe will achieve this with minimal maintenance. Please note that, our intention is to capture the total number of records triggered from SAP on that particular day, NOT the total number of records successfully processed by PI on that particular day. No changes were done to the original XI interface that does the IDOC to JMS conversion. This interface follows the original process flow and keeps placing messages(each message could have multiple data records) in the JMS queue all through the day.

We created a outbound message interface in PI, generated a client proxy in the source SAP system out of this Message Interface. This client proxy will be triggered just before midnight, once every day, and will send the total record count to PI. If you ask, what this client proxy does, it reads the edidc & edid4 tables, based on the IDOCTYPE & MESSAGE TYPE & START/END TIMES used by this interface, gets the count of idocs triggered daily and sends it to PI. We schedule this program using a variant in SAP. PI will then place the record count in the same JMS queue(we use the same JMS communication channel).

The advantage i could see here, is that, the existing PI interface design is not affected by this requirement. This avoids a whole lot of issues that may arise,say, if we maintain some sort of counter using ccBPM(ccBPM usage is a strict No No from the client) or in a ZTABLE(we believe it is not a good practice making synchronous data updates from within XI to a ZTABLE using a ZRFC from within XI mappings using JCO)

1. Shipment IDOCs sent from SAP AFS to SAP PI using sender IDOC adapter
2. SAP PI transformation step to convert the IDOC-XML to the legacy XML format
3. PI JMS adapter converts the target XML to legacy format using content conversion and delivers to the JMS queue
4. At about midnight, client ABAP proxy program sends the total record count to SAP PI
5. SAP PI transforms the content to target XML format and reuses the same communication channel to deliver the message to the JMS queue
6. Legacy system pulls messages off the queue, checks the total against the actual number of records recieved, if correct, process, if not, notify support desk.

Also, we made sure the job that triggers the IDOCs is suspended before the client proxy executes. There could be potential pitfalls in the approach we have taken, but i beleive, with proper choreography of the JOB scheduler in SAP system, we could achieve our goal with minimal maintenenace issues. If you feel otherwise, let know and we could improve/finetune our apprach.

As a sidenote, I believe this apporach could also be used with reciever file adapters.  Thanks for taking your time to read until the end. Hope it was worth your time.