We looked at each piece of our integration puzzle for our custom bank payment remittance interfaces in my last posting Bank Payment Interfaces: The Pieces Of Our Plug’n’Play Puzzle.
Now we’ll put them all together and describe how they work end-to-end to send a remittance file to a particular bank while at the same time providing an open integration environment that can also be plugged into by other banks.
The run-time process flow is illustrated in the Visio illustration below.
Going With The Flow
So let’s step through each item on the bank remittance process flow. We’ll extract data for an imaginary financial institution called the Bank of Universal Kredit, which we’ll shorten to BUK.
1. Kick off extract by bank
We began with a custom ABAP program that extracted remittances for one bank. We called this program ZAPREMIT.
ZAPREMIT was orginally conceived as a one-off extract and interface based on standard report RFCHKE00, which has all the logic you need to pull remittance data from SAP. We changed ZAPREMIT so that it can also be used to select and process data for any bank that we choose to send remittances to.
We also created one variant for each bank that will be used to schedule their extracts. BUK runs every evening at 6:00 pm.
2. Data pulled in bank’s format
We’ve seen in previous postings that the data for the remittance is read from tables PAYR, REGUP, REGUH. The problem is that each bank has its own unique remittance file format. The data is essentially the same but the structure of the send file is unique to each bank.
Logic was written into ZAPREMIT to accodomate each bank’s file structure. Payment data is pulled from the three tables and passed to structured string variables and an internal table that matches the header, payment detais, and summary records expected by the bank. Some banks don’t use a header or a summer but each unique structure is accomodated before any other processing is undertaken.
3. Data passed to IDoc function
We decided to use an IDoc to get the data out of SAP. ZAPREMIT originally kicked out a formatted flat file and then triggered an SFTP Send command through a script at the operating system level.
The header and summary data collected for BUK is passed to 1,000 byte unformatted strings while the details are passed to an internal table with a single 1,000 byte field. These match up with the SDATA field of the outbound IDoc.
Remember, the data passed to the 1,000 byte strings has already been formatted by ZAPREMIT to match each bank’s unique flat file format for remittances. The formatting is only done once, by program ZAPREMIT.
Each bank has its own IDoc basic type that exactly matches the structure of the flat file expected by the bank. However, the basic type is linked to a message type that is the same for all bank remittances. We called the message type ZAPREMIT. BUK’s basic type is ZAPREBUK01. Each bank’s IDoc is populated by the same function module: Z_MASTER_IDOC_CREATE_ZAPREMIT.
Extract program ZAPREMIT calls Function Z_MASTER_IDOC_CREATE_ZAPREMIT, which takes the 1,000 byte strings, identifies BUK’s basic type, passes the 1,000 byte strings to the SDATA field of each segment, and assembles the segments in the correct sequence: one header, one to many detail records, and one trailer.
In practical terms, this process slaps an IDoc control segment on top of the bank’s unique flat file structure. This opens up all the processing and monitoring capabilities of the IDoc interface to our custom remittance extracts, including the RFC connection to the EDI system in SI.
5. Outbound IDoc exported to file
Once the IDoc has been assembled and written to the database at status 30, it’s ready for export. The outbound partner profile for bank partner BUK, with message type ZAPREMIT and basic type ZAPREBUK01, is configured to send the IDoc immediately and to trigger the EDI system. It points to a file port that is set to trigger the EDI system through the RFC destination set up in SM59, just like all other EDI IDocs.
The IDoc is exported to a file in a folder on the SI application server.
6. RFC to SI OB receiving BP
An RFC call is made through the RFC destination and JCo (SAP’s Java Connector) to a listening BP in SI. The BP kicks off, picks up the IDoc file, and passes it along to the next BP. The same listening BP is called for all EDI IDocs. It knows what to what to call because the next BP name is passed to SI with the filename in RFC parameters. It’s read by an XPath statement and used to call the next BP. We’ll look at exactly how that’s done in a future posting.
7. Processing BP is called
The processing BP kicks off. First it calls an extract map that splits the IDoc into multiple files by partner number and moves key identification data from the control segment into SI process data.
Then it reads the custom table in SI that maps IDoc control segment to EDI routing data. It uses a JDBC service to build an SQL statement to read the table. Selection options are fed to the SQL by XPath statements that pull control segment values from SI Process Data.
8. Returns bank remittance routing data
The JDBC SQL read returns a range of data that we’ll need to route the remittance to the correct bank including:
- Communication type: SFTP for BUK. But this can also be configured for AS2 or other forms of secure Communications.
- Communication BP: SEND_SFTP for BUK. Identifies communications BP to call.
- SSH Profile ID: SSH profile for BUK in SI, which includes SFTP server name, port, user name, host and client certificates that allow secure access to BUK’s SFTP server.
- Name of the map that will be used to convert the IDoc to the flat file that BUK is expecting. Since the IDoc exactly mirrors the BUK flat file structure, this map only strips the control segment from the IDoc. So it’s very easy to write.
9. Run conversion map
Once this data has been read from the custom SI EDI routing table, the processing BP executes the map, which is identified through an XPath statment to the map name that has been saved to a node in Process Data.
9A – 9C. The translation fails
If the map fails to convert the IDoc to BUK’s final flat file format, the oubound IDoc — message type ZAPREMIT — is sent to the STATUS interface BP to return the error. The control segment of ZAPREMIT is mapped to the STATUS IDoc control and data segments and a status code of 05 — Translation failed — is passed, along with the outbound IDoc’s IDoc number and other key values.
This is then sent into SAP to update the outbound IDoc with the error status.
10. Translation succeeds
Two separate processing streams are kicked off.
10A – 10C. Return successful translation status.
The outbound IDoc is sent to the STATUS interface BP and mapped to a STATUS IDoc with a successful translation status of 06. The STATUS IDoc is then sent into SAP to update the original outbound IDoc.
10. SEND_SFTP BP called
The processing BP calls SFTP_SEND and passes the SSH profile ID and the translated flat file. SFTP_SEND runs a series of SFTP services that mak the connection, trigger the SEND command to pass the flat file, and closes the connection. The connection is made by the SFTP ClientBegin service that reads the SSH profile ID from Process Data with an XPath statement. The profile ID then allows it to identify all the information from the SI database that it needs to connect to BUK’s SFTP server.
11 – 11C. Status interface is called
Whether or not the SFTP send succeeds, the STATUS interface BP is called with the outbound IDoc. If it succeeds, status 12 — Transmission OK — is passed to the STATUS IDoc. If it fails, status 11 — Transmission failed — is passed. The STATUS IDoc is then sent into SAP to update the original outbound IDoc with success or failure.
Failure means the SFTP problems need to be researched. Once SFTP is working, the only causes of failure should be a change in the host certificate that hasn’t been communicated to you — we are dealing with banks after all — or the bank’s system is down.
This is a pretty generic interface. It works in the same way for every bank and once the base infrastuctre is built, we have a development process that we follow each time to onboard a new bank. We’ll look at that in my next posting.