AttachmentSplitterBean – Split attachments into child messages
Update 19 Feb 2016: New parameters sequenceId for EOIO processing (courtesy of Raffael Herrmann) and replaceMainWithAttachment for main payload replacement
Update 5 Feb 2016: New parameter filenameFromAttachmentName for additional determination of attachment’s file name (courtesy of Raffael Herrmann)
Introduction
Splitting a message into multiple messages is a common integration requirement. One of the most common approach is with multi-mapping, however this approach only works for splitting the main payload of the message and not the attachments (unless a custom Java multi-mapping is developed). The splitting feature is also available in the following adapters/modules:
- Seeburger BIC/Splitter module
- B2B Add-on EDISeparator adapter
- Sender mail adapter (available from PI/PO 7.31 SP14, 7.4 SP09)
However, all the above functionality are limited to their respective adapter/module and
.AttachmentSplitterBean aims to provide a generic adapter module solution that is reusable for multiple scenarios across different adapter types. It is configurable and can be used for various scenarios as described in the following Use Cases section.
Use Cases
Below are some example use cases for this module:
- Split attachments generated from files within a ZIP archive
- Split mail attachments
- Split SOAP/Proxy attachments
- Split attachments generated using “Additional Files” feature of NFS/SFTP adapter
- Split attachments generated after Java mapping
Source Code and Deployment Archive
AttachmentSplitterBean belongs in the same adapter module project as FormatConversionBean. Refer to following blog for the location of source code and/or EAR deployment file.
FormatConversionBean – One Bean to rule them all!
Module Parameter Reference
The JNDI name for the module is listed below for adding the module to the processing chain of a communication channel.
Module name = Custom_AF_Modules/AttachmentSplitterBean
Below is a list of the parameters for configuration of the module. Certain parameters will automatically inherit the default values if it is not configured.
Parameter Name | Allowed Values | Default Value | Remarks |
---|---|---|---|
mode |
binding, channel |
Required field. Determines which channel the child messages should be sent to
|
|
qualityOfService |
EO, EOIO, BE |
Required field. Determines quality of service for child messages | |
sequenceId |
Uppercase string of 1-16 characters |
Required field when qualityOfService = ‘EOIO’. Sets sequence ID of EOIO queue |
|
contentType | Sets the content type of the child messages. If not configured, the content type is automatically determined from the corresponding attachment’s content type | ||
storeFileName | Y, N | N | Y = Determines the filename (if available) from the content type of the attachment, and stores the filename in Dynamic Configuration attribute of the child messages |
filenameFromAttachmentName | Y, N | N |
Available when storeFileName = ‘Y’. Y = Determines the filename from the attachment’s name instead of content type. Useful when attachment is binary content. |
fileNameAttr | FileName | Dynamic Configuration attribute for file name | |
fileNameNS | http://sap.com/xi/XI/System/File | Dynamic Configuration namespace for file name | |
adapterType | Valid adapter types in the PI system |
Required field when mode = ‘binding’. Adapter type for the channel where the child messages will be sent to |
|
adapterNS |
Required field when mode = ‘binding’. Adapter namespace for the channel where the child messages will be sent to |
||
fromParty |
Available when mode = ‘binding’. Sender Party of sender agreement where the child messages will be sent to |
||
fromService |
Required field when mode = ‘binding’. Sender Service of sender agreement where the child messages will be sent to |
||
toParty |
Available when mode = ‘binding’. Receiver Party of sender agreement where the child messages will be sent to |
||
toService |
Available when mode = ‘binding’. Receiver Service of sender agreement where the child messages will be sent to |
||
interfaceName |
Required field when mode = ‘binding’. Sender interface name of sender agreement where the child messages will be sent to |
||
interfaceNamespace |
Required field when mode = ‘binding’. Sender interface namespace of sender agreement where the child messages will be sent to |
||
channelID |
Required field when mode = ‘channel’. Object ID of the channel where the child messages will be sent to |
||
replaceMainWithAttachment | Y, N | N | Replaces content of main payload with first attachment’s content. This allows use of single iFlow for design if content of original main payload is no longer required for further processing |
messageLog |
pre, post |
Saves a log version of the message that is viewable in Message Monitor
|
|
logLocation | Name of log version when messageLog is populated. Location defaulted to value in messageLog if logLocation not populated |
Example Configuration
To demonstrate the usage of this module, we will use a scenario whereby attachments are generated by the “Additional Files” feature of the SFTP sender channel.
iFlow 1 – Main scenario to pickup main payload
In the main iFlow, a sender SFTP channel is configured to pick up additional files as attachments. As shown below, all .txt files a split prefix are picked up. Additionally, it will also pick up files of the same name with .file1 & .file2 extensions.
The custom module is configured in the module chain. For this example, mode = ‘binding’ is used, therefore the details of the sender agreement where we intend to send the child messages to are required.
Note: All attachments are sent as child messages to the same sender agreement.
iFlow 2 – Secondary scenario to route generated child messages to final destination
For the secondary scenario, the sender system/interface is configured to match the values in the module parameters above. Although it is technically possible to use a sender comm channel of any type, I recommend using the SOAP channel as it works in listening mode and will execute processing only when a child message is routed to it.
Additionally, if the attachments are non-XML payloads, ensure that the SWCV of the sender interface is blank.
Testing Results
Firstly, the main .txt files and the additional .file1 & .file2 files are uploaded to the SFTP source directory.
When the files are picked up by the SFTP channle of iFlow 1, the additional files are attached in the message.
Subsequently, when the message is processed through the module, the attachments are retrieved and dispatched as new messages to iFlow 2.
In the message monitor, both the main message and child messages are displayed.
And finally, we can view the content of the two child messages generated and routed on iFlow 2. These child messages are routed to the receiver system of iFlow 2 – for simplicity purposes, the delivery to the final receiver system will not be shown here.
Conclusion
With AttachmentSplitterBean, we now have a readily available solution to handle scenarios that require processing of attachments. The modules simplifies such scenarios as the child messages can be routed to a secondary iFlow scenario and handled separately. It will no longer be necessary to develop a complex logic to handle both main payload and attachments within a single scenario. Furthermore this is available to be used in any adapter type that allows extension of the module chain.
Below are some real life scenarios where utilizing this module can simplify the design:-
- Preserve file names of input Zip file – using Java Mapping. Solution: after the Zip file is unzipped with Java Mapping to create attachments, the attachments can be split to child messages via AttachmentSplitterBean in the receiver channel of iFlow 1. In iFlow 2, DynamicAttributeChangeBean can be added in the sender/receiver channel to dynamically set the filename and target location.
- Sender file adapter – check files before transmission to PI – Solution: AttachmentSplitterBean can be used to split the main PDF file and additional TXT file. The main PDF file can be processed as normal to the target file system, while the TXT file will be processed by the second scenario to map to IDoc (any additional modules can be added to the second scenario as required).
Hi Eng Swee,
I am new to PI and just stumble across your blog post and the contents are simply AWESOME!
By the way, I am trying to create something similar - an email attachment splitter bean and currently I am stuck not able to find the ConnectionFactory interface used in the source. I am not able to locate the library which contains that interface - com.sap.engine.interfaces.messaging.api.ConnectionFactory.
I am using NWDS 7.3 and I could not find the interface in the - javadoc as well. Not sure what am I missing at the moment and hopefully you can point me to the right direction 🙂
Thanks,
Mike
Hi Mike
Thanks for your feedback. Glad that you liked this 🙂
Unfortunately, the class is not part of the XPI Adapter Libraries that comes bundled in NWDS 7.31. It is part of JAR file com.sap.aii.af.ms.ifc_api.jar. Unfortunately, it is also not covered in SAP's JavaDocs 🙁
You can refer to the following Wiki on where to get the following JAR file in the PI file system.
Where to get the libraries for XI development - Process Integration - SCN Wiki
Rgds
Eng Swee
Hi Eng Swee,
Thanks for your help. I am able to build and deploy the custom attachment splitter now and it is working nicely 🙂
By the way, I have got another question which I think you might be able to give me some pointers 🙂 . I have a requirement whereby only those attachments should be splited and sent as individual message but not the main message. So let's say if I have a main message with 2 attachments, only those 2 attachments should be sent to the next process in the process flow configuration, the main message should stop/exit from the process flow.
Do you know if there is a way to do that?
Best regards,
Mike
Hi Mike
Unfortunately, the current module logic does not cater for suppression of the main message. You can either route the main message to a dummy file receiver channel or if the main message is in XML, you can try to add an XPath condition when determining Receivers to suppress it. You can refer to my comments in the following thread on a similar request.
Alternatively, you can probably try to modify the module code to replace the main message with the first attachment, or some sort of similar logic.
Rgds
Eng Swee
Hi Eng Swee,
Thanks for your quick response. Guess I will go with your suggestion to make the kludge whereby overwriting the main message with the first attachment data while sending the rest of the attachments as new messages 🙂
Thanks for your suggestion and I really appreciate that!!
Cheers,
Mike
Hi Eng Swee,
This is indeed a great blog. 🙂
Regards,
Souvik
Hi Eng Swee
This is,Indeed, a great knowledge to share .
Helped it in one of customer requirement in 7.4
Thanks,
Hi Eng Swee,
Thank you very much for sharing this great blog 🙂
Br
Bhargava Krishna
Hi Eng Swee,
We tried the same as per blog. But, we are not able to achieve the attachment file name in the receiver side. Attachment file name is taken from the "File List" parameter from the Sender Channel not from the original attachment. We already specified the "filenameFromAttachmentName" as "Y" and "storeFileName" as "Y". Kindly, suggest us how to proceed to achieve this. Thanks a lot.
Hi Eng Swee,
A very nice blog.
Thanks
Arijit
Hi Eng Swee,
This was working perfectly. However, we have come across a scenario where the consumer isn't sending the filename in the content type. So I changed the module in the sender adapter to get the filename from the attachment name (filenameFromAttachmentName) but its not working. Each attachment is called 'Attachment x' even through the attachments have correct names. Am I missing something?
Hi Robert
To be honest, the functionality for the filenameFromAttachmentName parameter was contributed by Rafael, and I didn't get to test it myself as I didn't have a scenario to reproduce it.
Saying that, the logic for it is really simple, it just uses the getName() method of the Payload interface, and there's nothing there that will add the "attachment-x" values.
Can you please go to the details screen from the message monitor (click Open Message) and see how the fields are displayed in the Name column of the Attachments tab?
Another place to check would be clicking on the View Message Content link from the main monitor to view the raw content of the message that would provide the whole MIME envelope. This might give some clue as to where the "attachment-x" values are coming from.
Regards
Eng Swee
HI Eng Swee
I checked the attachment and that looks good (I am testing with SOAPUI which provides the name in the content type but for the purposes of this test I need it from the name).
When I check the actual payload I can see that the SAP attachment name is attachment-x
So it looks as if the code is doing what it is supposed to do, its getting the attachment name. Do you have any suggestions?
Regards
Rob Warde
Hi Rob
Can you do an actual test from the consumer that isn't sending it in the content type, and then see how the MIME envelope is being constructed? In particular, I would like to see where exactly they are populating the attachment file name (maybe Content Disposition, etc). If it's somewhere else, maybe it's possible to use some of the other methods of the standard class to access the field and extract it. As it is, seems like getName() might not work for your case.
Testing from SOAP UI might not give the accurate behaviour.
HI Eng Swee
Its very similar. Note there is a very odd content type and the payload iteself does contain the filename in the content disposition.
Sorry about the size of the images but you can see it clearly when you right click and open in new tab
Hi Rob
This looks much clearer now, and yes, the filename needs to be retrieved from the content disposition in this case.
If this is an urgent requirement on your end, I would very much encourage you to fork the GitHub repository and make the changes yourself. The details of how to set this up is available in one of my other blog posts. Once things work well, it would be great if you can contribute back the functionality so that it is available for others in the future. This is essentially my intention in setting all this up and making it available publicly in order to encourage open source collaboration between members.
Otherwise, this is something that I will put on the enhancement list, but I can’t promise right now when I can complete the changes. Furthermore, I’m not able to reproduce the scenario on my end, so it will depend on you to deploy my changes and test it out – I can imagine that this can turn out to be a lengthy process.
Either way, I have created this as an issue in my GitHub repository below in order to track it. Let’s continue the conversation there so as not to clog this blog with the nitty gritty details of the change.
https://github.com/engswee/equalize-xpi-modules/issues/7
Regards
Eng Swee
Hi Robert, hi Eng Swee,
I just did some quick coding concerning the error and added a Pull Request to Eng Swee’s Github repo. It would be nice, Robert, if you could test these changes on your system, to check if everything is handled right.
Kind regards,
Raffael
Gentlemen
Many thanks, we were trying to fix this ourselves but were having a few issues with EGit. I will ask my developer to deploy this to development and unit test.
Regards
Rob
Hi Raffael,
Many thanks and appreciate your help. Would it be possible for you to generate the latest EAR file with the code change that you did and upload that to https://github.com/engswee/equalize-xpi-modules/releases/ ?
Thanks,
Arijit
Hi Arijit,
since this is Eng Swee's repository, I think Eng Swee as maintainer should decide when to add a new release. But I'm in contact with Eng Swee and we're discussing the changes. If we agree to have a good codebase, I think he'll merge my changes and upload the new release.
Kind regards,
Raffael
Thanks very much Raffael, appreciate your help.
Hi Eng Swee,
Any chance you can update the EAR File, then we can test it now. Many thanks!
Regards
Arijit
Hi Arijit
As mentioned to Rob in my earlier response, the comment section of this blog is not a suitable place to discuss on this collaboration effort.
To maintain the integrity of the repository, we need to work through the code changes and test it properly before having any new public release. Please join the discussion in GitHub so that we can work out how we can move this forward collaboratively.
Regards
Eng Swee
Hi, I try using custom module, but i get error when use BS System:
exception caught during processing mail message [1]com.sap.aii.af.lib.mp.module.ModuleException: Couldn't retrieve inbound binding for the given P/S/A values: FP=;TP=;FS=BS_D1X028;TS=;AN=SI_DUMMY;ANS=urn:dummy;
Thanks for any help.