Schedule Serverless CRON Job to Pull data from REST APIs (Part – 1)

Posted On February 22, 2020 by Yogesh Gore Posted in

REST API is an Application Program Interface that uses HTTP request to GET, PUT, POST and DELETE data, it’s an architecture style approach to communicate with third party application. In order to integrate our module with third party applications or in order to pull data from third party application to our database for analysis purpose the REST APIs are useful. For the analysis purpose we can consume REST API with SSIS using third party connector, but the problem with this approach is that, it will be required On Premise server for package deployment and job schedule. The alternate approach is to use serverless CRON expression. Since, mid 2018 the serverless development methodology is ditching the traditional development.

In this blog we are going to see how to consume REST API with Serverless CRON expression. Here, we are using Zoho People API for integrating HR modules data to Azure DB. With Zoho People API, you can extract employee’s data and form data in XML or JSON format to develop new applications or integrate with your existing business applications. Zoho People API is independent of programming languages which helps you to develop applications in any programming languages (reference).

Implementation

Authentication token generation

In order to access Zoho People API, Zoho People authentication token is required. The token can be generated using Browser mode and API mode.

For the API mode each request has Username or Email and Password needs to include in the POST body
Another approach is to register your app with Zoho by going to zoho.com/developerconsole and Add Client ID, Once added it can be used to generate new access_token.

We can check the response by use using POST request, Open any REST Client and send request by using any REST API method, here we are send request to get leave data and we are using VS Code REST Client.

If the request is valid then the we can get response in following format containing the data requested.

3. As we are getting proper response, the next step is to create CRON expression (Serverless approach) to pull the data from REST API, here we are going to use App services on Azure to create Timer Trigger Function App. The CRON expression is a time-based job schedular. which has six field to define time precision in following format.

{second} {minute} {hour} {day} {month} {day-of-week}

Each field can have one of the following types of values:

Type	Example	When triggered
A specific value	“0 5 * * * *”	at hh:05:00 where hh is every hour (once an hour)
All values (*)	“0 * 5 * * *”	at 5:mm:00 every day, where mm is every minute of the hour (60 times a day)
A range (- operator)	“5-7 * * * * *”	at hh:mm:05, hh:mm:06, and hh:mm:07 where hh:mm is every minute of every hour (3 times a minute)
A set of values (, operator)	“5,8,10 * * * * *”	at hh:mm:05, hh:mm:08, and hh:mm:10 where hh:mm is every minute of every hour (3 times a minute)
An interval value (/ operator)	“0 /5 * * *”	at hh:05:00, hh:10:00, hh:15:00, and so on through hh:55:00 where hh is every hour (12 times an hour)

4. The next step is to create function app, for development purpose we are going to use Visual Studio Community 2019/2017.

- In Visual Studio create a project by selecting File > New > Project

Select Visual C# (here we are going to use C# for development you can choose Php, Python or F#) any of your choice.

Under the Visual C# select the Azure Functions and click on Next

The next step is select a new Azure Function Application, Select the Timer trigger function choose your Azure Function version (v1, v2 or v3). Select Authorization level as Anonymous as we don’t want to include API Key for the function and kept the other settings as it is.