Avoidance of Infinite Iterations of Until activity within Azure Data Factory / Synapse Pipelines

Problem Statement :

In case if there is a failure of any activity inside an Until activity and the termination condition is not being met, the Until activity would go under infinite executions.

So how to avoid infinite executions in case of some error or exception scenarios.

Pre set up :

Infinite Iterations in case of Failures:

And one has to manually cancel the executions to avoid the infinite iterations.

Prerequisites :

1. Azure Data Factory / Synapse Pipeline

Solution :

1. In my previous blog of Error Logging and the art of avoiding redundant activities, we saw how to assign the failure messages of all activities within a single variable ; we would need to follow the same steps within the until activity.

Create a variable : ErrorHandling and follow the proper process to set up that variable value

New Flow :

2. Within the Until activity expression , in addition to the existing expression Add an OR aspect with the below expression

not(equals(variables('ErrorHandling'),''))

The above expression would map to TRUE in case if there is any value present within the ErrorHandling variable which in turn would state that there was a failure in one of the activity because of which a failure message got inscribed within the ErrorHandling variable.

3. Once the ErrorHandling variable maps to TRUE and irrespective of any other conditions being met or not, the OR statement would ensure that the Until loop is exited thereby avoiding infinite executions.

The JSON for the above Error handling framework is available at this GitHub location.

Note: The variable DummyFailure is to induce error in the Pipeline for testing purpose. For a Success scenario, update the type of the DummyFailure variable from Array to String.