This blog describes how to configure exception handling in a splitter scenario. It describes the splitter in some sample scenarios using different exception handling configurations.

Usage of General and Iterating Splitter with Exception Handling

In many Cloud Integration scenarios big messages are split into smaller parts using a splitter pattern. The smaller chunks are then processed in cloud integration. In such scenarios multiple errors can occur, which can be handled in different ways. This blog gives some general recommendations and describes the options available in some sample scenarios.

General Recommendations 

For using the splitter with exception handling there are some important recommendations:

1. Combination of Stop on Exception with End Events in Exception Subprocess

The exception handling defined in the Splitter takes precedence over the end event defined in the Exception Subprocess with respect to continuation with the next split. This means the next split is only executed if Stop on Exception is not set. If Stop on Exception is set, the next split is not executed anymore independently of the end event used in an Exception Subprocess. Because of this runtime behavior, we recommend using only the following combinations:

Combine Stop on Exception with Error End Event

If you set the Stop on Exception flag in the Splitter either use no Exception Subprocess, or if you need a specific error handling, configure the Exception Subprocess to end with an Error End event.

Using the Message End event would, in this combination, lead to a probably unexpected behavior because the end event is only relevant for the single split, not for the overall split processing. This means, the overall split processing would stop.

• In case no Gather is used, the processing would end with a Completed message, even if not all splits were executed or tried to be executed because the split processing stops.
• If a Gather is used, only the single split would be completed, but the next split processing would not be executed anymore. The processing would continue with the splits already executed in the Gather. The final status of the message will then depend on the further message processing.

Because of this runtime behavior we recommend to always use the Error End event when Stop on Exception is set.

Do not set Stop on Exception when using Message End Event

If you do not set the Stop on Exception flag in the Splitter either use no Exception Subprocess, or,  if you need a specific error handling, configure the Exception Subprocess to end with a Message End event.

Using an Error End event would, in this combination, lead to a probably unexpected behavior because the end event is only relevant for the single split, not for the overall split processing: The overall split processing would continue.

• In case no Gather is used the processing would end with a Failed message, but the next split would still be executed because Stop on Exception is not set in the Splitter.
• If a Gather step is used, only the single split would end, but the overall processing would continue. The final message status will then depend on the further message processing.

Because of this runtime behavior we recommend to always use the Message End event when Stop on Exception is not set.

2. Use Separate Local Process for Split Processing

Do the processing of the single splits (between Splitter and Gather or after the Splitter if no Gather is used) in a separate Local Process. Doing so, you can configure a dedicated exception handling for errors occurring in the split processing using an Exception Subprocess in this local process.

Additional Considerations when Using the Gather Step

When using the Splitter together with a Gather step the following recommendations should be followed additionally.

1. Create Valid XML in Exception Subprocess

If you handle split errors in an Exception Subprocess and use a Gather step after the Splitter, create a valid XML to be passed to the Gather step. Otherwise the Gather step could stop with an error because of the wrong format and the whole message processing would end with a Failed status.

2. Avoid Splitter – Gather Scenarios without Exception Subprocess

Avoid configuring Splitter – Gather scenarios without error handling in an Exception Subprocess when using Combine XMLs of the same or different format in the Gather step. Otherwise the Gather step may stop with an error because the payload at the time the exception occurs has the wrong format (wrong XML or no XML at all) and the whole message processing would end with a Failed status.

 

Let’s discuss how to follow the recommendations in some sample scenarios. First, I show you some sample scenarios with Splitter, but without Gather. There are different scenarios with different exception handling depending on the overall use case for the scenario. Further down I will describe the specific recommendations in sample scenarios using Splitter and Gather.

Splitter Scenarios without Gather

The following scenarios are simple sample scenarios using a Splitter without a Gather step after the Splitter.

Scenario 1: Splitter without Gather with Stop on Exception

Scenario:

If you want to stop the complete message processing in case of an error in the processing of one of the splits, the easiest way would be to configure this using the Stop on Exception option in the Splitter:

In this sample scenario a bulk message is received by the SOAP adapter, this message is then split into single chunks. The processing of the single chunks is done in a separate Local Process. In the sample scenario there simply is a Request-Reply call fetching additional data. Afterwards the single splits are sent to the Receiver.

Note the following important point:

• In this scenario, no Exception Subprocess is configured. If you need to configure a specific exception handling in an Exception Subprocess and want to stop the complete processing using Stop on Exception follow the recommendations in scenario 2.

Runtime Execution

If an error occurs during message processing, the processing stops, and the message gets a Failed status. If some splits were executed successfully before the error occurred, their data is already sent to the receiver, no roll-back of the executed calls is possible. This means, parts of the original message have been sent to the receiver, some not.

The sender system usually gets the failed status back and resends the message (if configured for retry). When the sender system resends the message, the whole message is executed again.

In this scenario, re-executing the whole message means that also those split parts that had been successfully sent out at the first time are sent to the receiver for the second time. This leads to duplicate messages in the receiver system. So, either the receiver system can handle duplicates, or you have to configure your scenario differently to avoid those duplicates. One option would be to do a dedicated exception handling in an Exception Subprocess as described in scenario 3.

Scenario 2: Splitter without Gather with Stop on Exception and Exception Subprocess

Scenario:

As already mentioned, you need to use an Exception Subprocess if you want to do a specific error handling. If we want to end the complete processing, but would like to do a specific error handling before that, like writing a notification to someone, we can combine the Stop on Exception option in the Splitter and an Exception Subprocess:

Like in scenario 1, a bulk message is received by the SOAP adapter, which is then split into single chunks. The processing of the single chunks is done in a separate Local Process, where we fetch additional data via a Request-Reply call and then send the single splits to the Receiver.

The difference to scenario 1 is that we add an Exception Subprocess in the Local Process, where we configure the specific error handling. As shown above ,in the Exception Subprocess added to the Local Process, as a simple error handling we write a notification mail to someone. We end the Exception Subprocess with an Error End event because we want to end the complete processing anyway and do not continue with the next split.

Note the following important point:

• In this scenario, an Exception Subprocess is configured with an Error End event together with Stop on Exception. If you need to configure a specific exception handling in an Exception Subprocess and do not want to stop the complete processing, but handle the single erroneous splits, follow the recommendations in scenario 3.

Runtime Execution

The runtime processing in case of an error in this scenario is a little different from scenario 1. When an error occurs during split processing, the exception is caught by the Exception Subprocess. The error handling is executed, which means that the notification mail is sent to the Receiver. Afterwards the complete processing ends because of the Stop on Exception option in the Splitter. The next split is not executed. The message gets a Failed status in the monitoring.

Similar to scenario 1, if some splits were executed successfully before the error occurred, their data is already sent to the receiver, no roll-back of the executed calls is possible. This means, parts of the original message have been sent to the receiver, some not.

The sender system usually gets the failed status back and resends the message (if configured for retry). When the sender system resends the message, the whole message is executed again.

Also, in this scenario, re-executing the whole message means that the already sent split parts are sent to the receiver again. Which leads to duplicate messages in the receiver system. So, either the receiver system can handle duplicates, or you have to configure your scenario differently to avoid those duplicates. One option would be to do a different exception handling in an Exception Subprocess as described in scenario 3.

Scenario 3: Splitter without Gather with Exception Subprocess

Scenario:

As already mentioned, the Exception Subprocess can be used to configure a specific error handling and then continue the overall processing. Let’s take the sample scenario 2 but configure the Exception Subprocess with an Message End event. Make sure that Stop on Exception is not selected in the Splitter:

Like in scenarios 1 and 2, a bulk message is received by the SOAP adapter, which is then split into single chunks. The processing of the single chunks is done in a separate Local Process. In the sample scenario there simply is a Request-Reply call fetching additional data. Afterwards the single splits are sent to the Receiver.

The difference to scenario 1 and 2 is that we add an Exception Subprocess in the Local Process, where we configure the specific error handling for the single split parts. To be able to do a specific error handling for the single chunks we retrieve a unique id for the chunk using a Content Modifier and write it into a property. In this sample scenario we use a property with the name ‘CharacterName’:

In the Exception Subprocess added to the Local Process we write the chunk to a data store using a Data Store Write step with the unique ID (property ‘CharacterName’)  as Entry ID. Note, that we select the option Overwrite Existing Message. With this setting we can make sure that another execution of the same split does not create a new entry for the same split but overwrites the existing entry.

The important thing is that we end the Exception Subprocess with a Message End event. From runtime perspective, this means the error raised by the split processing is caught by the Exception Subprocess and handled there. The overall processing continues with the next split because Stop On Exception is not selected.

Note the following important point:

• In this scenario, an Exception Subprocess is configured with a Message End event. For this configuration Stop on Exception should not be selected.

Runtime Execution

The runtime processing in case of an error in this scenario is different from scenarios 1 and 2: When an error occurs during split processing, the exception is caught by the Exception Subprocess. The error handling is executed, which means that this split message is written to the data store and is not sent out to the Receiver. The overall processing continues until all splits are executed. The message gets a Completed status in the monitoring.

The splits which caused an error, are stored in a data store and can, for example, be polled by a different integration flow and handled differently.

Note, that this is only a simple sample error handling configuration, in the Exception Subprocess you can configure the specific handling required for your scenario. But keep one important aspects in mind:

• Do not set Stop on Exception in the splitter and end the processing in the Exception Subprocess with a Message End event if you want to continue processing of the next splits. The continuation of the split execution depends on the Stop on Exception setting.

Scenario 4: Splitter without Gather without Stop on Exception and without Exception Subprocess

Scenario:

If you don’t need to take care of errors for single splits but just want to continue processing of all possible splits, you could use the option to configure the scenario without Stop on Exception in Splitter and without Exception Subprocess. The scenario would be the same as scenario 1 but Stop on Exception would not be selected.

Runtime Execution

If an error occurs during split processing, the single split is not completed but the whole processing continues. This means that all successful splits are sent out to the Receiver. The splits that could not be sent successfully are just ignored.

The message gets a Failed status, because not all splits could be executed successfully.

Splitter Scenarios with Gather

The following scenarios are simple sample scenarios using a Splitter step combined with a Gather step which combines the single splits using an aggregation strategy.

Scenario 5: Splitter with Gather with Stop on Exception

Scenario:

Also, with the Gather step the splitter scenario can be configured to stop the complete message processing if an error occurs. Again, we configure this using the Stop on Exception option in the Splitter:

In the small sample scenario, a bulk message is received by the SOAP adapter, this message is then split into single chunks. The processing of the single chunks is done in a separate Local Process. In the small sample scenario there simply is a Request-Reply call fetching additional data. All the splits are combined in a Gather step configured with Combine option:

Afterwards an XSLT Mapping is executed, and the message is sent to the Receiver. The whole scenario looks like shown above.

Note the following important point:

• In this scenario, no Exception Subprocess is configured. If you need to configure a specific exception handling in an Exception Subprocess and want to stop the complete processing using Stop on Exception follow the recommendations in scenario 6.

Runtime Execution

If an error occurs during message processing, the whole processing stops, and the message gets a Failed status. The sender system usually gets the failed status back and would resend the message (if configured for retry). If the sender system resends the message, the whole message is executed again, including all the split parts that were already executed in the previous execution.

Scenario 6: Splitter with Gather with Stop on Exception and Exception Subprocess

Scenario:

Also, with the Gather step the splitter scenario can be configured with Stop on Exception and an Exception Subprocess to end the complete processing, but still do a specific error handling before, like for example writing a notification to someone.

Like in scenario 5, a bulk message is received by the SOAP adapter, which is then split into single chunks. The processing of the single chunks is done in a separate Local Process. In the sample scenario there simply is a Request-Reply call fetching additional data. All the splits are combined in a Gather step configured with Combine option.

After the Gather an XSLT Mapping is executed and the message is sent to the Receiver.

The difference to scenario 5 is that we add an Exception Subprocess in the Local Process, where we configure the specific error handling. As shown above in the Exception Subprocess added to the Local Process, as a simple error handling we write a notification mail to someone. We end the Exception Subprocess with an Error End event because we want to end the complete processing.

Note the following important point:

• In this scenario, an Exception Subprocess is configured with an Error End event together with Stop on Exception. If you need to configure a specific exception handling in an Exception Subprocess and do not want to stop the complete processing, but handle the single erroneous splits, follow the recommendations in scenario 7.

Runtime Execution

The runtime processing in case of an error in this scenario is a little different from scenario 5. When an error occurs during split processing, the exception is caught by the Exception Subprocess. The error handling is executed, which means that the notification mail is sent to the Receiver. Afterwards the complete processing ends. The message gets a Failed status in the monitoring because the Stop on Exception option sets the message to Failed in case of an error independently of the kind of end event used in the Exception Subprocess.

Scenario 7: Splitter with Gather with Exception Subprocess

Scenario:

As already mentioned, you can use an Exception Subprocess if you want to do a specific error handling. This time we really want to handle the exception and do not end the whole processing. Make sure Stop on Exception is not selected in the Splitter:

Like in scenarios 5 and 6, a bulk message is received by the SOAP adapter, which is then split into single chunks. The processing of the single chunks is done in a separate Local Process. In the sample scenario there simply is a Request-Reply call fetching additional data. All the splits are combined in a Gather step configured with Combine option:

After the Gather an XSLT Mapping is executed and the message is sent to the Receiver.

The difference to scenarios 5 and 6 is that we add an Exception Subprocess in the Local Process, where we configure the specific error handling for the single splits. To be able to do a specific error handling for the single chunks we retrieve a unique id for the chunk using a Content Modifier and write it into a property. In this sample scenario we use a property with the name ‘CharacterName’:

In the Exception Subprocess added to the Local Process as a simple error handling we write the split message to a data store using a Data Store Write step with the unique ID (property ‘CharacterName’) as Entry ID. Note, that we select the option Overwrite Existing Message. With this setting we can make sure that another execution of the same split does not create a new entry for the same split but overwrites the existing entry.

In the Content Modifier after the Data Store Write step we need to create a valid XML, that can be understood by the Gather step. It needs to have the same format as the other split responses that are combined in the Gather. In my scenario I simply define an empty book tag, because this is the format expected by the Gather step.

We end the Exception Subprocess with a Message End event. From runtime perspective, this means that the error raised by the split processing is caught by the Exception Subprocess and handled there. The overall processing continues with the next split because Stop On Exception is not selected.

Note the following important point:

• In this scenario, an Exception Subprocess is configured with a Message End event. For this Stop on Exception should not be selected.

Runtime Execution

The runtime processing in case of an error in this scenario is different from scenario 5 and 6. When an error occurs during split processing, the exception is caught by the Exception Subprocess. The error handling is executed, which means that this split message is written to the data store and an empty, but valid XML payload is created which is then passed to the Gather. The overall processing continues until all splits are executed. The single splits, including the empty ones, are combined in the Gather step. An XSLT Mapping is executed, and the message is sent to the Receiver. The message gets a Completed status in the monitoring.

The splits which caused an error, are stored in a data store and can, for example, be polled by a different integration flow and handled differently.

Note, that this is only a simple sample error handling configuration, in the Exception Subprocess you can configure the specific handling required for your scenario. But keep two important aspects in mind:

• Do not set Stop on Exception in the splitter and end the processing in the Exception Subprocess with a Message End event to continue processing of the next splits.
• Create a valid XML to be passed to the Gather step, otherwise the Gather step would stop with an error because of the wrong format and the whole message processing would end with a Failed status.

Scenario 8: Splitter with Gather without Stop on Exception and without Exception Subprocess – Not Recommended

Scenario:

Also, in a scenario with Gather you could configure the scenario without Stop on Exception in Splitter and without Exception Subprocess. The scenario would be the same as scenario 5, but Stop on Exception is not selected.

Note, that this scenario setup cannot be recommended because of the following reason:

This scenario would only work in error case if the single splits going to the Gather step have the same format, because we configured the Gather step with Combine XMLs with same format. If an exception occurs in one split, the payload at the time the exception is raised would be routed to the Gather step. This payload may not have the correct format and so the Gather would end in an error. To avoid this, the recommended configuration option would be to configure the scenario with an Exception Subprocess as described in scenario 7.

 

Further Readings

For more configuration recommendations in respect to Splitter also check out the following blogs:

• Usage of Splitter Flow Steps in Local Process
• “Stop on Exception” for Iterating/General Splitter
• Behaviour of the new Splitter Version 1.2
• “Grouping” Option of Iterating/General Splitter
• Using Parallel Processing in General and Iterating Splitter