"Stop on Exception” for Iterating/General Splitter

former_member497124 · ‎01-16-2018

Introduction and Motivation

The General Splitter, along with the Iterating Splitter, is one of the most used Flow steps in an IFlow (Integration Flow). It helps to reduce the memory footprint of an IFlow by splitting very large incoming XML payloads into smaller chunks for further processing. In the streaming mode, the splitter doesn't create an internal DOM, and it starts producing chunks even before it reaches the end of the payload. This means that the splitter itself has a very small memory footprint.

Please remember: The General Splitter preserves all nodes on the way to the first split point, whereas the Iterating Splitter only extracts the nodes at the split point.

In this blog, I am going to explain the option “Stop on Exception” for Iterating/General Splitter.

The Option "Stop on Exception"

In the scenario below, the integration process contains a Content Modifier, which has an XML payload in the message body. The payload is sent to the subprocess “GS_Router”.

Let’s use the following payload as an example:

<root>

  <x>

     <y>

         <z id="1">1</z>

         <z id="2">2</z>

         <z id="3">3</z>  

     </y>

     <y>

         <z id="4">4</z>

         <z id="5">5</z>

         <z id="6">6</z>  

     </y>

 </x>

</root>

The General Splitter in the subprocess “GS_Router” splits this payload using the XPath expression “//z” into six chunks, which are sent by the “Send” step to Receiver2. Once the whole payload has been split, without any exceptions, the IFlow ends with the status COMPLETED.

To simulate an exception, we can define a condition expression //z[@id=”3″] in the router in the subprocess. In this case, the third split leads to an “Error End Event”. What happens next depends on how we set the splitter option “Stop on Exception”.

If this option is set, the splitter stops after the third split and the IFlow ends with the status FAILED. 3 files are created by Receiver2.

If this option is not set, the splitter continues splitting and Receiver2 gets all 6 chunks. Here as well the IFlow ends with the status FAILED.

In the next scenario, I have moved all steps into a main integration process.

Receiver1 gets the same number of chunks as Receiver2 if the router stays on the default path. With our example payload, both Receiver1 and Receiver2 create 6 files.

If the router has a condition expression that leads to an “Error End Event” after the third split, then here as well the behavior depends on whether the splitter option “Stop on Exception” is set.

If this option is set, the splitter stops after the third split and the IFlow ends with the status FAILED. 3 files are created by Receiver2. Receiver1 only creates 2 files.

If the “Stop on Exception” options is not set, the splitter continues splitting and Receiver2 gets all 6 chunks. The IFlow ends with the status FAILED and Receiver1 only creates 5 files.

We notice that Receiver1 always creates 1 file less than Receiver2. What file is missing and why?

To answer this question, we extend the integration process by an exception subprocess.

If we run this scenario, Receiver3 always gets the following chunk

<?xml version="1.0" encoding="UTF-8"?>

<root xmlns:n0="sap">

  <x>

    <y>

       <z id="3">3</z>

    </y>

  </x>

</root>

That is the missing file of Receiver1! It is the third chunk, where the exception was raised.

At the end of the blog, I’ll tell you how to create unique file names when using splitters with an SFTP receiver.

In the screenshot below, we see the target of an SFTP receiver:

If you select “Append Timestamp”, you can add a timestamp to each file name. However, the chunks are produced so fast by the splitter that the time resolution of this feature is not sufficient.

To get truly unique file names, you can use the variable ${exchangeId} as part of the file name.

Also, ${property.CamelSplitIndex} can be used to number the chunks sequentially.