How to copy data from rest API to Azure data lake via Azure Data Factory | CloudFronts

How to copy data from rest API to Azure data lake via Azure Data Factory

Posted On August 11, 2020 by Sandip Patel Posted in 

In this blog I am going to explain you how to copy data from rest API to Azure data lake using Azure Data Factory.

Step 1: Create an azure storage account via azure portal using your credentials.

A screenshot of a cell phone  Description automatically generated

Step 2: Once your storage creation done, create a container called “restapidata” that will going to use to store rest API data.

A screenshot of a cell phone  Description automatically generated

Step 3: If you have rest API endpoint then its fine otherwise search dummy rest api example in google and copy one of the URL. Here I have taken employee endpoint (http://dummy.restapiexample.com/api/v1/employees)

Step 4: Create an Azure Data Factory service in azure portal and create a pipeline.

A screenshot of a cell phone  Description automatically generated

Step 5: Create a link service for rest API.

A screenshot of a cell phone  Description automatically generated

Step 6: Create a link service for Azure data storage.

A screenshot of a social media post  Description automatically generated

Step 7: Create a dataset for rest API.

A screenshot of a cell phone  Description automatically generated

Step 8: Create a dataset for Azure data lake storage.

Step 9: Drag a copy activity in the pipeline and set a general property.

A screenshot of a cell phone  Description automatically generated

Step 10: Set a source property of copy activity. You can also click on preview data to check whether data is populated or not.

A screenshot of a social media post  Description automatically generated

Step 11: Set a sink property of copy activity.

A screenshot of a social media post  Description automatically generated

Step 11: Set a mapping property of copy activity.

A screenshot of a cell phone  Description automatically generated

Step 12: Debug the pipeline and check the output.

A screenshot of a cell phone  Description automatically generated

Also click on details to check record proceed.

A screenshot of a cell phone  Description automatically generated

I hope this will help you.


Share Story :

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close