How to delete file post processing if the integration fails with exception?

[Former Member]

Hello,

I have noticed this behavior recently that when due to any exception throughout the integration flow, the post-processing mechanism(especially "delete file") for the sender SFTP doesn't work. Before, it was deleting the file irrespective of what happens with the consecutive steps of the iFlow. This has eventually kept the integration flow running for eternity(until I undeployed of course:]) because the scheduler was set to every 10 seconds recurreance.

One option was to have an Exception sub-process, but that will keep End Message Event as default and would indicate that the message was Successful/Completed in the monitor message processing. From this, I figured there was an update recently wherein the mechanism changed from deleting as soon as a poll to deleting if only the message was successful/completed.

But, the reason I can't keep the End Message Event and had to use Error End Event is that there was a demand from the client I work for that the monitor message processing should display failed message although the exception was handled by the Exception sub-process.

Please let me know if any inputs are required from my end, and it would be great if you have anything for me.

Thanks,

Mahek.

[sunny_kapoor]

Hi Former Member,

As you have written

"From this, I figured there was an update recently wherein the mechanism changed from deleting as soon as a poll to deleting if only the message was successful/completed."

Are you sure the behaviour has been changed recently?

Can you please share from where you have got this information that behaviour has been changed.

I think the behaviour of post-processing is correct because usually you do not like to delete the file if the message processing fails.

Coming back to the alternate approach for your use case. you can do the following:

1. In exception subprocess of main flow, call another integration flow using pocess direct reciever adapter and pass the directory and filename using the headers.

2. In another flow which has process direct sender adapter, use the Poll Enrich step to fetch the same file from the same directory using dynamic headers (this you have passed from your main flow) and configure post processing as delete file and end it with end message event.

3. End the exception subprocess in the main flow with error end event. This will mark the message as failed.

I hope this will help

Regards,

Sunny

[Former Member]

Hello Sunny,

At the point where I wrote that "I figured", I meant there might have been an update recently or in the past 6 months. Because in a scenario where I worked back in late 2020 the behavior was definatley different, although I have no evidence for the same.

And thank you for the inputs, I will try that out and post the status here.

Thanks,

Mahek.

[Former Member]

Hello Sunny,

It did not work, I did as you mentioned but the problem is that somehow the poll enrich step in the processDirect flow is not deleting the file.

Just for testing purposes I even tried replacing the sender process direct with a start timer, then it was working as expected, meaning there was no flaw in that poll enrich step.

Thanks,

Mahek.

[sunny_kapoor]

Hi Former Member,

Yes, I think the design I have proposed would not work because we are trying to delete the same file in process direct flow which the main flow is also reading.

You can change the design something like this:

Main flow, start it with timer and call sub flow with process direct which should just only read the sftp file and give the file response to this main flow and keep all other business logic in this main flow.

Sub Flow would just read the file and send the response to main flow and

1. In case of succesful read, the file would get deleted automatically irrespective of your main flow has some errors.

2. In case of exception here, the file would still gets deleted because we are ending the exception subprocess with End Message event.

Regards,

Sunny

[Former Member]

Hello Sunny,

I can't actually do that because the start timer will cause a lot of unnecessary message triggers and in turn a lot of entries in the monitor message processing. Where as the Sender SFTP would have an entry only if the file was polled.

Thanks,

Mahek.