SAP Data Intelligence: Get notifications about your pipelines

I’m so happy to find time for writing my next blog post 😊 Today’s topic is alert notification, and I’m going to keep it short and simple.

I had to provide notification pipelines by each DI project that I delivered. There are some cases when customers want to be notified about relevant events from SAP Data Intelligence, for example:

• a pipeline failed, and a customer wants to receive the notification about it
• as soon a pipeline is completed, a customer want to receive the notification with the result.

For the first case SAP Data Intelligence offers a template for email notification. To send an email a SMTP connection must be established in DI. After that you can use standard template “Example Notification”:

The workflow is following: The “Pipeline” operator triggers your pipeline. If there is no error, then an event will be sent to the “Workflow Terminator”. In this case no email will be sent. If there is an error, then an event will be sent to the “Notification” operator (“Send Email” operator can be used too). I’m going to use the pipeline (PL_Alert) with the Custom Python Operator (ZPython) that consists division by zero. This will produce an error. The email message contains in this case the raw output:

Obviously, customers will be annoyed about such email. To make the email more user friendly, you can customize the email using a custom operator. I’m going to use Python. Sometimes customers don’t want to receive unnecessary details, just the information that a pipeline failed:

#processing the error message
def on_input(error):
    email_body = "Dear User,\n\nyour pipeline failed."
    api.send("output", email_body)
api.set_port_callback("input", on_input)

The email received by a customer:

Of course, you can add more information into the email, e.g. error message, timestamp, pipeline name, etc. To add the error message, you should parse JSON of the “Pipeline” operator’s raw output. The email could be like:

This is only an example. Sure, you can make it more informative and official.

Using Custom Operators, you can also send notifications to Microsoft Teams, Slack, etc. It is up to you to decide what messenger should be used.

As demonstrated, you can create your notification pipeline smoothly. Depending on your logic, you can choose a standard template pipeline or customized pipeline. You can also build the “Notification” or “Send Email” in your pipeline directly.

Feel free to share your thoughts/ ideas in the comments 😊