How to get Email notification when Azure Data Factory Pipeline fail

However, it seems there’s no “e-mail activity” in Azure Data Factory. I would like to send an e-mail notification if one of the activities fail. In this blog I am going to explain you how to send an e-mail notification using ADF Web Activity and Azure Logic App.

Sending an Email with Logic Apps

Logic Apps allows you to easily create a workflow in the cloud without having to write much code.

First login to Azure.portal.com

Choose to create a new resource, search Logic App

Click on Create button, it will be asked to specify some details for the new Logic App:

Click on Review + create again to finalize the creation of your new Logic App. After the app is deployed, you can find it in the resource menu.

Click on the app to go the app itself. Click on Logic app designer link.

In this tip, we need the HTTP request (“When a HTTP request is received”) as the trigger, since we’re going to use the Web Activity in ADF to start the Logic App. From ADF, we’re going to pass along some parameters in the HTTP request, which we’ll use in the e-mail later on. This can be done by sending JSON along in the body of the request.

The following JSON schema is used:

{

“properties”: {

“DataFactoryName”: {             “type”: “string”         },

“EmailTo”: {             “type”: “string”         },

“ErrorMessage”: {             “type”: “string”         },

“PipelineName”: {             “type”: “string”         },

“Subject”: {             “type”: “string”         }

},

“type”: “object”

}

We’re sending the following information:

• The name of the data factory. Suppose we have a large environment with multiple instances of ADF. We would like to send which ADF has a pipeline with an error.
• The e-mail address of the receiver.
• An error messages.
• The name of the pipeline where there was an issue.
• The subject of the e-mail.

In the editor, click on New step to add a new action to the Logic App:

Click on New step it will send the e-mail. When you search for “mail” you will see there are may different actions:

Click on Office 365 Outlook and select Send an email(V2)

Once you’re logged in, you can configure the action. We’re are going to use dynamic content to populate some of the fields

The result look like this:

Once you click on Save, it will generate HTTP POST URL. Copy this URL for ADF.

Triggering the Logic App from ADF

Suppose you already created pipeline with some activities in ADF:

Add a web activity to the canvas and connect another activity to this new activity using the arrow. When the connection has been made, right-click on the connection to change it to a Failure precedence constraint. This will change the color of the connector to red.

Now we are using HTTP POST URL that will copied from the Azure Logic App

We also need to add a header, where we will set the Content-Type to application/json. In the body, we enter the following JSON (following the structure mentioned before):

{

“DataFactoryName”: “@{pipeline().DataFactory}”,

“PipelineName”: “@{pipeline().Pipeline}”,

“Subject”: “An error has occurred!”,

“ErrorMessage”: “The ADF pipeline has crashed! Please check the logs.”,

“EmailTo”: “[email protected]”

}

We’re using system parameters to retrieve the name of the data factory and the name of the pipeline. All the other fields in the settings pane can be left as-is.

Now we can run the pipeline and wait to see if any emails come in:

I hope this will help you.