Introduction:
The objective of this blog is to showcase how it is possible to migrate three different use cases for Mail adapter sender and receiver handling from SAP Process Orchestration to Cloud Integration, a capability of the SAP Integration Suite. For more information about the official guidelines for attachment handling on Cloud Integration, visit the Help Documentation.
The first and the second use case involves the extraction of attachments from unread emails using the Mail adapter sender. Subsequently, each extracted file (mail attachment) is sent to a specific folder on an SFTP server while keeping the original attachment name. This blog explores some test cases, including scenarios with and without message mapping, a mail with a single or multiple attachments, and with different formats such as XML, TXT, and PDF.
The third use case involves the extraction of files from an SFTP server to pick up files and subsequently sending them as an email with attachments using the Mail adapter, while keeping the original file name from the SFTP folder. In this use case, no transformations are done.
Pre-requisites:
- Cloud Integration with access to SMTP mail server
- Cloud Integration with access to SFTP server’s directory
- Deploy the known_host keys for SFTP in Cloud Integration security material. You can find more information on: Cloud Integration – How to Setup Secure Connection to sftp Server.
- Deploy the security material on Cloud Integration for SFTP and for Mail.
Scenario 1: Mail to SFTP: Handling a Single Attachment in XML, TXT or PDF Format
Variant 1: Integration Flow without Message Mapping
In this scenario implemented on SAP Process Orchestration and later Cloud Integration, the Mail adapter sender polls emails from an inbox and extracts the attachments. The attachment can be in XML, PDF, or TXT format; no transformations occur within SAP Process Orchestration, and the mail attachment is sent to an SFTP folder via SFTP adapter receiver.
Integrated Configuration Details on SAP Process Orchestration:
In this scenario, the adapter modules described in the following were used to ensure that the correct file format of the attachment is sent to an SFTP folder. Additionally, to read the attachments, the ‘Keep Attachments’ attribute is set as true.
Figure 1 - Mail sender adapter: Connection and processing configurations.
For more information on the PayloadSwapBean module, visit the PayloadSwapBean in the Module Processor documentation.
The ‘Attachment.ContentType’ parameter is used to define the content type of the mail attachments. It can be set to 'text/xml' for XML files, 'application/pdf' for PDF files, or 'text/plain' for TXT files, depending on the attachment file format.
The general Mail adapter configurations are shown in the following image:
Figure 2 – Mail sender adapter: General configurations.
For more information on the SAP Process Orchestration Mail adapter, seeConfiguring the Mail Adapter.
The SFTP receiver adapter configurations for this scenario are shown in the following image:
Figure 3 - SFTP receiver adapter: Destination.
For more information on the SAP Process Orchestration SFTP Adapter, see Configuring the SFTP Receiver Channel.
Integration Flow Details on Cloud Integration:
The Mail adapter sender triggers the integration flow on Cloud Integration through a scheduler, which is defined to poll emails every minute for testing purposes. To access the attachment in the email, a Groovy script is used, and this script stores, counts, and retrieves the original file name of the attachment. This means that the file sent to the SFTP folder will have the same name as the mail attachment.
The "Prepare Attachments" Groovy script initializes the message with the attachment details and validates if the email has any attachment. If it doesn't have a single attachment, on the Cloud Integration message monitor, a custom header is set showing “Not Found” for mail attachments. Otherwise, if the email has an attachment, a custom header is set to “Found” for mail attachments and the Groovy script proceeds to storing the attachment in the 'Attachments' property.
To maintain the original attachment name in the SFTP folder, the Groovy script stores that information in the 'AttachmentName' property.
The following is the “Prepare Attachments” Groovy script:
def Message processPrepareAttachments(Message message) {
// Create a new Map of attachments and store it in a property.
def attachments = new HashMap<String, DataHandler>(message.getAttachments())
// Check if the mail has attachment
if (attachments.isEmpty()) {
// If the mail doesn't have attachments, set a custom header
def messageLog = messageLogFactory.getMessageLog(message)
// Custom header property to indicate that no attachments were found
messageLog.addCustomHeaderProperty('Mail Attachments', 'Not Found')
// Set an empty value for the Attachments property
message.setProperty('Attachments', '')
// All processing is complete, return the modified message.
return message
}
// Store attachments in a property called 'Attachments'
message.setProperty('Attachments', attachments)
// Store the number of attachments in a property called 'AttachmentCount'
// This part will not be used in the current test case.
// message.setProperty('AttachmentCount', attachments.size())
// Get the original attachment filenames and store them in a property.
def originalFilenames = attachments.keySet().toList()
message.setProperty('AttachmentName', originalFilenames.join(','))
// Logging custom headers
def messageLog = messageLogFactory.getMessageLog(message)
// Add a custom header property to show that attachments were found
messageLog.addCustomHeaderProperty("Mail Attachments", "Found")
// All processing is complete, return the modified message.
return message
}
After this step in which the “Prepare Attachments” Groovy script is executed, a router checks whether the ‘Attachments’ property is empty. If the ‘Attachments’ is empty, the message processing ends. Otherwise, if ‘Attachments’ property is not empty, the message processing continues, and the attachment is sent to the SFTP folder maintaining their original name.
The “EnableLog” Content Modifier was added to set the “EnableLogging” property to “true”, enabling the message logging for the Groovy script “Log” within the integration flow.
Integration Flow Design:
Figure 4 – Mail adapter to SFTP adapter handling a single attachment in XML, TXT or PDF format without message mapping.
Integration Flow connection details:
The sender Mail adapter configurations on Cloud Integration are shown in the following image:
Figure 5 – Mail sender adapter: Connection and processing configurations.
For more information on the Cloud Integration Mail adapter, see Configure the Mail Sender Adapter | SAP Help Portal.
The Inbox folder has been configured to retrieve “Only Unread” emails with attachments for which the post-processing was set to “Mark as Read”.
The receiver SFTP adapter configurations are shown in the following image:
Figure 6 – SFTP receiver adapter: Target configurations.
For more information on the Cloud Integration SFTP adapter, see Configure the SFTP Receiver Adapter | SAP Help Portal.
Variant 2: Integration Flow with Message Mapping
The difference between this scenario and the previous one is the use of message mapping and an additional “Get Attachment” Groovy script to retrieve the attachments from the message properties and preparing them for further processing within the message mapping in the integration flow.
Integrated Configuration Object Details on SAP Process Orchestration:
Besides the use of an operation mapping within the integrated configuration object, all other configurations remain the same as in the previous scenario.
Integration Flow Details on Cloud Integration:
As in the previous scenario, the Mail adapter sender triggers the integration flow. To access the attachments in the email, the “Prepare Attachments” Groovy script is used to check if the email has attachments and to store, count, and retrieve the original attachments names. A second Groovy script called “Get Attachments” is used to retrieve the attachments from the message properties and prepare them for the processing.
This Groovy script is responsible for processing the attachments within a message object and will iterate through the attachments stored in the message properties and retrieve the next attachment to process. It updates the message body with the content of the retrieved attachment. After this, the Groovy script removes the processed attachment from the message and stores it in the 'Attachments' property. This process ensures that each attachment is handled sequentially. Finally, the script updates the 'AttachmentCount' property to reflect the number of remaining attachments after processing.
The following is the “Get Attachments” Groovy script:
// This function retrieves and processes the attachments after they have been prepared.
def Message processGetAttachments(Message message) {
// Retrieve the attachments map from the message properties.
def attachments = message.getProperty('Attachments')
// Get the key of the next attachment to process.
def nextKey = attachments.keySet().iterator().next()
// Remove the attachment from the map based on the next key.
def attachment = attachments.remove(nextKey)
// Replace the message body with the contents of the attachment.
message.setBody(attachment.getContent())
// All processing is complete, return the modified message.
return message
}
After the "Prepare Attachments" and "Get Attachments" Groovy scripts are executed, the message is transformed through the message mapping. After these steps, the attachment from the email is sent to the SFTP folder while preserving the original name.
Integration Flow Design:
Figure 7 - Mail adapter to SFTP adapter handling a single attachment in XML, TXT or PDF format with message mapping.
Testing and Results:
In both cases, the message processing is very similar. After the deployment of the integration flow, the Mail adapter checks the email inbox folder in search for unread emails.
In the first case in which the message mapping is not used, after reading the attachment of the email through the “Prepare Attachments” Groovy script on Cloud Integration, the attachment is delivered to the specified SFTP folder defined on the SFTP adapter receiver.
In the second case with message mapping, the “Prepare Attachments” Groovy script reads the attachment of the email and, using the “Get Attachment” Groovy script, it retrieves the attachments from the message properties and delivers them to the specified folder on the SFTP adapter.
In both cases, if an unread email is found without attachment, the interface raises the following error on Cloud Integration message monitoring since the “Prepare Attachments” Groovy Script expects an attachment:
Error Details
org.apache.camel.CamelExecutionException: Exception occurred during execution on the exchange: Exchange[ID-e781c31c-8ee7-4182-6a03-f546-1706174998939-60-4], cause: java.lang.AssertionError: Empty Attachments. Expression: attachments.isEmpty()
After the successful execution, the attachment will be stored on SFTP folder with the original filename and format.
Scenario 2: Mail to SFTP: Handling Multiple Attachments with the Same or Different Formats (with/without Message Mapping)
In this scenario, it is considered that the emails to be read have multiple attachments that could be grouped in bulk with attachments in the same format or include attachments with different formats (such as XML, PDF, or TXT).
The next integration scenarios include examples with and without message mapping in SAP Process Orchestration and Cloud Integration and, as it was done for the previous examples, the email attachments are sent to the SFTP folder with the original file name.
Integrated Configuration Details on SAP Process Orchestration:
In this scenario, we consider an email with three attachments and with the same format and operation mapping on SAP Process Orchestration.
Note that since the 'Attachment.ContentType' parameter on adapter module defines the content type of email attachments and if the email has multiple files with the same format, each attachment is processed individually. However, if the email has multiple files with different formats, only the file format defined on the Mail adapter module is processed and the other formats are ignored.
Integration Flow Details on Cloud Integration:
Proceeding with the Cloud Integration integration flow for this scenario, the “Prepare Attachments” groovy script is used to access the attachments, store the files, and count the number of attachments.
Compared with the previous scenario, we added the following code snippet to the "Prepare Attachments" Groovy script to count the total number of attachments and save that information in the 'AttachmentCount' property in the loop process.
// Store the number of attachments in a property called 'AttachmentCount'
message.setProperty('AttachmentCount', attachments.size())
After the “Prepare Attachments” Groovy script, a loop process is added to the integration flow to execute the integration process, which means that this loop iterates until certain conditions are met. The condition expression for each iteration is defined as ${property.AttachmentCount} > ‘0’, which means that the loop continues as long as there is at least one attachment to be processed.
The integration process involves retrieving attachments using the “Get Attachments” Groovy script with an additional code before the processing is complete:
// Store the name of the attachment in a property named ‘AttachmentName’.
//message.setProperty(‘AttachmentName’, nextKey)
// Update the AttachmentCount property to reflect the remaining attachments.
//message.setProperty(‘AttachmentCount’, attachments.size())
This code segment is responsible for storing the name of the attachment being processed, and the overall progress of the attachment processing (remaining count) is updated to simplify the management of the attachments within the integration process.
After these steps, the attachments is sent to the SFTP folder maintaining their original names.
Integration Flow Design:
Figure 9 – Mail adapter to SFTP adapter handling different attachments in XML, TXT or PDF format with message mapping.
Testing and Results:
To test this interface, two use cases were considered. For the first test, the message mapping was removed from the integration flow, and it was tested with 3 different attachments in the email. The second test includes message mapping and an email with three attachments in the same format (Figure 6).
The first test case considers an email with a set of three attachments in different formats, without message mapping in Cloud Integration. After the deployment of the integration flow, the Mail adapter checks the unread email, then sends it to Cloud Integration, and finally delivers it to the specified SFTP folder defined previously on the integration flow, keeping the names of the mail attachment.
The second test case is based on a similar integration flow but with message mapping. To test it , an email with three attachments in the same format was considered. After the successful execution, it’s possible to check the attachments of the email according to its format located on the SFTP folder.
However, if an unread mail is found without attachments, the interface will be in error on Cloud Integration message monitoring since the “Prepare Attachments” Groovy script on the integration flow expects an attachment:
Error Details
org.apache.camel.CamelExecutionException: Exception occurred during execution on the exchange: Exchange[ID-216c204e-c0e4-4e77-4407-eaa5-1705861105494-40-2], cause: java.lang.AssertionError: Empty attachments Map in property Attachments
Scenario 3: SFTP to Mail: Multiple Files in XML, TXT or PDF Format without Message Mapping as Attachment
In this scenario, the process is the opposite as the previous scenarios, and it is initiated through an SFTP Sender adapter, which polls files in PDF, TXT, and XML formats from an SFTP folder. For the next use cases, the same tests is conducted on SAP Process Orchestration and Cloud Integration, where the files are sent without any transformations. Subsequently, they are sent as attachments via email using the Mail adapter receiver.
Integrated Configuration Details on SAP Process Orchestration:
After the deployment of the integrated configuration, the SFTP Adapter sender polls from a configured folder every minute, and if files are found, they are sent to the SAP Process Orchestration system.
To ensure that the mail can receive files as attachments, the attribute ‘Keep Attachments’ is set as ‘true’ on the adapter configurations.
The SFTP sender adapter configurations are shown in the following image:
Figure 10 – SFTP sender adapter: Connection and processing configurations.
For more information on the SAP Process Orchestration Mail Adapter, see Configuring the SFTP Sender Channel.
The receiver Mail adapter configurations are shown in the following image:
Figure 11 - Mail sender adapter: General Settings.
For more information on the SAP Process Orchestration Mail Adapter, see Configuring the Receiver Mail Adapter.
Integration Flow Details on Cloud Integration:
After deploying the integration flow, the SFTP sender checks every minute if files that should be processed exist on the SFTP and then sends them to Cloud Integration.
For managing email attachments in this integration flow, the "Process Files" Groovy script is used. This script is responsible for processing the files from the SFTP folder by converting them into attachment objects and attaching them to a message. With the help of this Groovy script, the file names are retrieved from the message headers, and the file content is retrieved from the message body. Then, it creates a ByteArrayDataSource with the file content, generates an attachment, and adds this attachment to the message.
The following is the “Process Files” Groovy script:
// This function processes the files by creating an attachment object and adding it to the message.
def Message processFiles(Message message) {
// Retrieve file name from message headers
def fileName = message.getHeader("CamelFileName", String)
// Retrieve file content from the message body
def body = message.getBody(InputStream)
// Create a ByteArrayDataSource with the file content
def dataSource = new ByteArrayDataSource(body, "application/octet-stream")
// Create an attachment using the data source
def attachment = new DefaultAttachment(dataSource)
// Add the attachment to the message using the file name as key
message.addAttachmentObject(fileName, attachment)
// Return the modified message
return message
}
After these steps, the files are sent as attachment by email while maintaining their original names.
Integration Flow Design:
Figure 12 - SFTP adapter to Mail adapter handling different attachments in XML, TXT or PDF.
Integration Flow Connection Details:
The Sender SFTP Adapter configurations are shown in the following image:
Figure 13 - SFTP sender adapter: Source configurations.
For more information on the Cloud Integration SFTP Adapter, see Configure the SFTP Sender Adapter | SAP Help Portal.
The receiver Mail adapter configurations are shown in the following image:
Figure 14 – Mail receiver adapter: Connection configurations.
For more information on the Cloud Integration Mail Adapter, see Configure the Mail Receiver Adapter | SAP Help Portal.
An important aspect to consider in the Mail receiver adapter is the configuration of the "Mail Attachment" on section for “Processing”. This option should be set as “true”, which enables the inclusion of all attachments for the email.
Testing and Results:
After deploying the integration flow, the SFTP sender adapter polls all the files in the folder defined previously. In this case, three different formats were considered (as shown in the following image). No transformations were considered in Cloud Integration, and the “Process Files” Groovy script processes the files by extracting their content and creating attachment objects, which are then added to the message.
For each file in the SFTP folder, an individual email is generated , as depicted in the following image:
Conclusions:
In this blog, we demonstrated how to migrate different use cases using Mail adapter sender and receiver from SAP Process Orchestration to Cloud Integration while keeping the same functionality.
In the first use case, the Mail adapter sender retrieves emails from an inbox and extracts the attachments (XML, TXT, and PDF formats) without any transformations on Cloud Integration. Subsequently, the email attachments are sent to the SFTP folder as files via the SFTP adapter receiver. For this purpose, a solution was with a simple integration flow was implemented to process a single attachment. It uses the "Prepare Attachment" Groovy script to store, count, and retrieve the original name of the file.
In the second use case, both scenarios with and without message mapping were considered using the same adapters, handling a bulk of mail attachments in various formats. The solution involved a loop that is executed until the attachments are fetched, along with a Groovy script to complete the process. The "Get Attachments" Groovy script is placed within the loop and is responsible for retrieving the attachments from the message properties and preparing them for processing.
For the third and last use case, the process is reversed compared to the previous use cases, and the interface is initiated through the SFTP sender adapter. This adapter polls files from an SFTP folder in PDF, TXT, and XML formats and sends them as attachments via mail. To implement this scenario, the "Process Files" Groovy script was used to process the files from the SFTP folder by converting them into attachment objects and attaching them to an email.
In conclusion, the integration flow implemented for the second use case works when the email has a single or multiple attachments in any format, with or without transformations on Cloud Integration. However, if a simple and quick solution without involving loops is required, the first option is easier to implement, considering it involves only a single attachment in the email and no transformations on Cloud Integration.
When comparing the integration flow with message mapping on Cloud Integration for use case 1 and use case 2, the primary difference is the necessity of adding an additional code snippet to store the name of the attachment and update the number of attachments in the loop process. Otherwise, the integration flow doesn’t work with the message mapping.
- SAP Managed Tags:
