Skip to Content
Technical Articles
Author's profile photo Bhalchandra Wadekar

Dynamic Setting of Archive Directory for Post Processing in SFTP Sender Adapter

In this blog, how to move file to different locations after processing is finished is demonstrated. This blog also serves as answer to question: Move original file to another folder on SFTP server in case of exception.

The structure of this blog is as follows. First, requirements from the question are specified. Then, dummy scenario is created to fulfil requirements. Finally, solution is demonstrated.

Requirements

  • Files in ‘input’ directory must be processed
  • When processing is successful, the processed file must be moved to ‘output’ directory
  • When processing is failed, the processed file must be moved to ‘error’ directory

Scenario

XML files will be read from ‘input’ directory. XML files will contain single node ‘Payload’ having single sub-node called ‘Success’. If ‘Success’ is true, processing will be successful. If ‘Success’ is false, process will end with Error End.

Possible Payloads

Success

<Payload>
    <Success>true</Success>
</Payload>

Failure

<Payload>
    <Success>false</Success>
</Payload>

Solution

The solution uses Content Modifiers to set the property ‘archiveDirectory’. In successful scenario, ‘archiveDirectory’ is set to ‘/output’. In failure scenario, Exception Subprocess is used to catch the exception and Content Modifier is used therein to set ‘archiveDirectory’ to ‘/error’.

Flow Steps

Source for SFTP

Here, Source > Directory is set to ‘/input’ as per requirements. Please fill in Address and Credential Name as per your configurations.

Success? Router

The Success? Router routes based on Success property’s value. If ‘/Payload/Success’ is ‘true’ then ‘archiveDirectory’ property is set to ‘output’ otherwise flow ends with Error End.

Set Archive Directory to ‘output’

If successful, ‘archiveDirectory’ property is set to ‘output’.

Set Archive Directory to ‘error’

If failed, ‘archiveDirectory’ property is set to ‘error’.

Processing for SFTP

After processing is complete (whether succesfully or not), the file is moved to a location specified dynamically using ‘archiveDirectory’ property.

Parameter Value
Post-Processing Move File
Archive Directory /${property.archiveDirectory}/${file:name}

Execution

For testing this flow, Success.xml and Failure.xml files, containing payloads specified in Possible Payloads section above are put in ‘input’ directory.

After flow polls files, Success.xml file is put in ‘output’ directory and Failure.xml file is put in ‘error’ directory.

Alternate Scenario(s)

Error occurs when sending the file to receiver SFTP Server

In this scenario, file is sent to SFTP server. However, ‘Create Directories’ option on SFTP Receiver is disabled and the directory in which file must be put is not created on server to deliberately cause exception.

Integration Process

SFTP Receiver Target Configuration

Please fill in Address and Credential Name as per your configuration.

SFTP Receiver Processing Configuration

Note that ‘Create Directories’ option is disabled.

Example Run

1. Input

File ‘Non Existing Directory.xml’ is uploaded to ‘input’ directory. The file contents are same as Success under Possible Payloads above.

2. Success? Router

As ‘/Payload/Success’ is ‘true’, ‘Yes’ branch is followed.

3. Set Archive Directory to ‘output’

Exchange Property ‘archiveDirectory’ is set to ‘output’.

4. Send file to SFTPReceiver

File is sent to SFTPReceiver.

5. Error occurs because the directory ‘nonexisting’ does not exist

As ‘nonexisting’ directory does not exist, exception is thrown by SFTP and is caught in CPI as:

org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: nonexisting, cause: 2: No such file or directory

6. Exception Subprocess is triggered

As exception occured, Exception Subprocess is triggered.

7. Set Archive Directory to ‘error’

Exchange Property ‘archiveDirectory’ is set to ‘/error’.

8. Exception Subprocess ends with Message End and triggers the Processing tab of SFTP Sender

In the processing tab, the configuration for Post Processing is to Move File to /${property.archiveDirectory}/${file:name}. As per these configurations, SFTP Sender Adapter moves the read file from ‘/input’ directory to ‘/error’ directory.

9. Output

Conclusion

As demonstrated, file can be moved to different locations when processing is finished using dynamic path. This is achieved when integration process ends with Message End i.e, message is completed either through success path or error path via Exception Subprocess, The exchange property ‘archiveDirectory’ was set at runtime and used to specify path where file should be moved dynamically.

Assigned Tags

      17 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Avinash Varma Nadimpalli
      Avinash Varma Nadimpalli

      Hi Bala,

       

      Hope you are doing good.

       

      I have a requirement to read the directory details  dynamically and pass into mapping as a input for one of the filed in CPI.

      could you please guide me the camel expression to fetch the folder details dynamically from the source SFTP.

       

      Thanks in advance!

       

      -Avinash

       

      Author's profile photo Bhalchandra Wadekar
      Bhalchandra Wadekar
      Blog Post Author

      Hi RVS Avinash Varma Nadimpalli,

      Check the File Language in Apache Camel documentation. If it doesn't help, you can ask a question to SAP Community.

      Hope this helps,

      Bala

       

      Author's profile photo Nitheesh Cherrikkal
      Nitheesh Cherrikkal

      Hi Bala ,

      Thank you for the wonderful explanation . Would just like to check , in normal real time XML payload , what should I check for a condition to see if my payload is successful or in error ? To move the files accordingly to the folders .

       

      Appreciate your help .

       

      Thanks ,

      Nitheesh

      Author's profile photo Bhalchandra Wadekar
      Bhalchandra Wadekar
      Blog Post Author

      If you receive payload in CPI, then you can route to Error End if the value of certain element indicates the error using XPath.

      Hope this helps,

      Bala

      Author's profile photo Avinash Varma Nadimpalli
      Avinash Varma Nadimpalli

      Hi Bala,

       

      Hope you are doing good.

       

      Could you please provide your inputs on below issue.

      https://answers.sap.com/questions/13090745/move-original-files-to-another-folder-on-sftp-serv.html

       

      Tried with above approach but unable to do as we are having multiple files to move and only one final RFC connection.

       

      Thanks,

      Avinash

      Author's profile photo Vinod Patil
      Vinod Patil

      Hello Bhalchandra,

      Nice blog!

      If I am not wrong, in this case, iFlow status in CPI monitoring will always be successful.

      In case of error, is there any way to fail iFlow status and still move file to error folder?

       

      Regards,

      Vinod Patil

      Author's profile photo Bhalchandra Wadekar
      Bhalchandra Wadekar
      Blog Post Author

      Thanks, Vinod Patil.

      As per Help Document, only successful messages trigger Post Processing. So, Post Processing is not an option for your requirement.

      You can use the combination of Bram's comment and Sriprasad's answer like so:

      • In the first flow
        1. Read the file
        2. Store contents in Data Store
        3. Delete the file using Post Processing
      • In the second flow
        1. In success, write file to 'archive' directory and end with End Event
        2. In failure, write file to 'error' directory and end with Error End Event

      Hope this helps,

      Bala

      Author's profile photo Avinash Varma Nadimpalli
      Avinash Varma Nadimpalli

      Hi Vinod Patil,

       

      Hope you are doing good. are you able to achieve this. ,

       

      Hi Bhalchandra Wadekar,

       

      Thanks for the wonderful blog and help.

      I have the same requirement and i stuck in deleting the file.Able to move the files to error folder but unable to delete.

      what would be the input adapter for second flow in the above provided solution ?

       

      Thanks,

      Avinash

      Author's profile photo Bhalchandra Wadekar
      Bhalchandra Wadekar
      Blog Post Author

      The second flow will start on schedule using the Timer Start Event.

      Hope this helps,

      Bala

      Author's profile photo Avinash Varma Nadimpalli
      Avinash Varma Nadimpalli

      Hi Bala,

       

      Thanks for the help.

      It worked with process direct.

       

      Thanks,

      Avinash

      Author's profile photo Appasab Patil
      Appasab Patil

      Hi Avilash,

      I did try with Process Redirect but it still fails my first flow in case my second flow (subscription flow) errors out which again will not delete my source file.

      Could you let me know if there is any specific option you used to delink the processing of second flow with first flow. Basically i want the first flow to be successful once the process redirect successfully makes call to next flow and if there is any error in second flow i have the exception sub process defined there it self.

       

      Thanks and Regards

      Appasab

      Author's profile photo Sven Kelemen
      Sven Kelemen

      Hi Bala, works great, thanks, but what if there is an exception in the exception subprocess, e.g. in error subprocess ‘archiveDirectory’ property is set to ‘error’ and email is sent with the exception information. Sending email fails for some reason (noticed with quota issue on the test O365 account). Because successful messages trigger Post Processing, postprocessing fails. This need something like nested exception subprocess or storing contents in Data Store?

      Author's profile photo Bhalchandra Wadekar
      Bhalchandra Wadekar
      Blog Post Author

      Hi Sven Kelemen,

      Nested exception handling is not supported.

      You could achieve the scenario you mentioned in another way by having a Local Process Call in the main Exception Subprocess. In the Local Integration Process, you can send an email. If sending the email fails, the Exception Subprocess inside the Local Integration Process will handle the error and end with End Message Event. Please ensure that, in the main Exception Subprocess, you have set the directory to desired error directory before calling the Local Integration Process.

      Exception%20Handling%20with%20Email%20Notification

      Exception Handling with Email Notification

      You could achieve the same using a separate Integration flow to send an email and calling it using ProcessDirect Adapter. Remember to handle exceptions in this process and end with an End Message Event or an Escalation End to continue processing in the main subprocess.

      Hope this helps,
      Bala

      P.S.: I don't have access to SFTP now. So, can't test this solution. Please confirm if the solution helps and works for you for the benefit of future readers :).

      Author's profile photo Sven Kelemen
      Sven Kelemen

      Hi Bala, it works as required, file is moved to property.archiveDirectory e.g. 'error' even when sending mail fails. Thanks. But it would be good to get error in the Message Monitor. Now status is 'Completed' meaning error information is lost. Kind regards, Sven

      Author's profile photo Bhalchandra Wadekar
      Bhalchandra Wadekar
      Blog Post Author

      Hi Sven Kelemen,,

      Thank you for confirming the solution works. I am glad :).

      Could you please try using Escalation End in the Exception Subprocess of the Local Integration Process?

      Exception%20Handling%20with%20Escalation%20End

      Exception Handling with Escalation End

      Your requirement and the behaviour of Escalation End is explained with an example in the blog Ending the message the right way!

      Kind regards,
      Bala

      Author's profile photo Sven Kelemen
      Sven Kelemen

      Hi Bala, thanks again, this is better, now getting "Escalated" instead of "Failed" or "Canceled" which would be preferred.

      Author's profile photo Mel Khol
      Mel Khol

      Hello,

      Just wanted to add that I was having a lot of trouble getting this to work. If I hardcoded 'error' into the field Archive Directory, it worked fine, but as soon as I set the property archiveDirectory to 'error', it wasn't working (the file just stayed where it was).





      In the end, this did work: ${property.archiveDirectory}/}/${file:onlyname.noext}.${file:ext}


      The file coming from the SFTP was a CSV, not an XML (no idea if this is the reason the solution described in the blog post wasn't working for me).