SFTP Adapter – Handling Large File
In SFTP adapter the following interactive options are included for handling large sized files:
- Bypass PI Runtime For File Transfer
- Chunk Mode
Bypass PI Runtime for File Transfer
This option allows user to choose a directory on the PI system where the incoming message or file is saved. The intended receiver receives a file in the output directory containing the notification of the file where it is saved on PI system.
Chunk Mode
This option allows user to divide an incoming message into chunks of message ranging from 1 MB to 50 MB size as configured.
Prerequisites
You need to install ESR content patch (Minimum patch 02 of SP 04 for XI CONTENT SFTP ADAPTER 1.0) provided as part of the SAP Note 2144272. Upon successful deployment of ESR content patch, apply minimum of the patch number 8 or more of PIB2BSFTP SP04 for availing the fixes.
Procedure
Enabling Bypass PI Runtime for File Transfer
1. Choose Edit in the Edit Communication Channel screen.
2. Open Processing tab under the Parameters tab.
3. In Large File Handling area, perform the following:
3.1 In the Special Processing Mode field, set the value as Bypass PI Runtime For File Transfer.
3.2 In the Absolute File Path (including Filename) field, specify a valid path along with the file name for saving the incoming file on the PI system.
Example for the file path: //host/../../../../<file_name>.<file_type>
Note: To include time stamp to the file name, add “%TS” sequence to the file name, to avoid overwrite of the file.
4. Perform the following steps to verify the configuration and the file transfer:
4.1 Open Communication Channel Monitor.
4.2 Start the Sender Channel.
4.3 Go to the Storage location specified in the Large File Handling section at the time of configuration and verify that the incoming message has been saved.
4.4 Also verify that a file containing the details of the storage location has been put in the out directory of receiver channel.
Example: An XML file will be created in the storage location with the details as given below:
Parameters such as source file location, target file location, file size, time of creation are available in the XML file.
4.5 You can also verify the audit log as shown below.
Enabling Chunk Mode
1. Choose Edit in the Edit Communication Channel screen.
2. Open Processing tab under the Parameters tab.
3. In Large File Handling area, perform the following:
3.1 In the Special Processing Mode field, set the value as Chunk Mode.
This value appears for selection only when the Quality of Service value is Exactly Once in order (asynchronous)
3.2 In the Maximum Size (MB) field, select a desired value of chunk size you want the incoming message to be divided by.
4. Perform the following steps to verify the configuration and chunking of message:
4.1 Open Communication Channel Monitor.
4.2 Start the Channels.
4.3 Verify that the audit log for checking the chunking happens.
The adapter creates a new XI message for each chunk.
4.4 Match the received file with the source file to verify the integrity of the message.
Hi,
Good to know the new features of the SFTP Adapter.
1. Does the large File handling supports the mapping ( i.e. structural transformation) or it can be used only for pass through scenarios ?
2.when we use the option By Pass PI Runtime ...Can we specify only PI Directory (NFS) or we can use any other File directory.. Also what will be advantage of Bypassing the PI Run time.
Hi Rajesh,
1) Large File Handling options will not support mapping.
a) In Chunk Mode: Mapping is strictly not recommended as these are not logical chunks. The chunks will be made on the basis of size.
b) In Bypass option: The large file will not be part of XI message payload. XI message payload will contain a metadata file. You can have mapping for this metadata file if needed.
2) You can specify any file directory(Local or NFS) which is reachable from PI box.
Bypass option will help you to download large files from SFTP server into your local file system without risking OOM errors.
Hope that clarifies your doubts.
regards,
Piyush
Thanks for these addtional options .. got 2 questions regarding the 'chunk mode':
(1) Will the receiver SFTP adapter automatically combine the chunks again ? ( Like with the FTPS adapter )
(2) Is the 'chunk' functionality cross adapter meaning that we can read the chunks in via the SFTP¨adapter and have a receiver FTP(s) adapter combine them again ( or vice versa ) ?
Thanks,
Steven
Hi Steven,
Yes, chunking can be used across adapters. For e.g. you can do chunking using File Sender and combine them using SFTP receiver(and vice-versa).
Thanks & regards,
Piyush
Thanks Piyush for your fast response 🙂
Can SFTP Adapter with big file handling config working with PGP module in SFTP sender adapter?
Hi Vasantha,
Very good blog - thank you for sharing.
Are you sure the feature " Bypass PI Runtime for File Transfer" available in SFTP Receiver Adapter? in our current installation I can only see this option in the Sender Channel and I don't see in the Receiver channel. Please clarify
Thanks,
Laxman