Schedule Serverless CRON Job to Pull data from REST APIs (Part – 1)
REST API is an Application Program Interface that uses HTTP request to GET, PUT, POST and DELETE data, it’s an architecture style approach to communicate with third party application. In order to integrate our module with third party applications or in order to pull data from third party application to our database for analysis purpose the REST APIs are useful. For the analysis purpose we can consume REST API with SSIS using third party connector, but the problem with this approach is that, it will be required On Premise server for package deployment and job schedule. The alternate approach is to use serverless CRON expression. Since, mid 2018 the serverless development methodology is ditching the traditional development.
In this blog we are going to see how to consume REST API with Serverless CRON expression. Here, we are using Zoho People API for integrating HR modules data to Azure DB. With Zoho People API, you can extract employee’s data and form data in XML or JSON format to develop new applications or integrate with your existing business applications. Zoho People API is independent of programming languages which helps you to develop applications in any programming languages (reference).
Implementation
- Authentication token generation
In order to access Zoho People API, Zoho People authentication token is required. The token can be generated using Browser mode and API mode.
- For the API mode each request has Username or Email and Password needs to include in the POST body
- Another approach is to register your app with Zoho by going to zoho.com/developerconsole and Add Client ID, Once added it can be used to generate new access_token.
- We can check the response by use using POST request, Open any REST Client and send request by using any REST API method, here we are send request to get leave data and we are using VS Code REST Client.
If the request is valid then the we can get response in following format containing the data requested.
3. As we are getting proper response, the next step is to create CRON expression (Serverless approach) to pull the data from REST API, here we are going to use App services on Azure to create Timer Trigger Function App. The CRON expression is a time-based job schedular. which has six field to define time precision in following format.
{second} {minute} {hour} {day} {month} {day-of-week}
Each field can have one of the following types of values:
Type | Example | When triggered |
A specific value | “0 5 * * * *” | at hh:05:00 where hh is every hour (once an hour) |
All values (*) | “0 * 5 * * *” | at 5:mm:00 every day, where mm is every minute of the hour (60 times a day) |
A range (- operator) | “5-7 * * * * *” | at hh:mm:05, hh:mm:06, and hh:mm:07 where hh:mm is every minute of every hour (3 times a minute) |
A set of values (, operator) | “5,8,10 * * * * *” | at hh:mm:05, hh:mm:08, and hh:mm:10 where hh:mm is every minute of every hour (3 times a minute) |
An interval value (/ operator) | “0 */5 * * * *” | at hh:05:00, hh:10:00, hh:15:00, and so on through hh:55:00 where hh is every hour (12 times an hour) |
4. The next step is to create function app, for development purpose we are going to use Visual Studio Community 2019/2017.
-
- In Visual Studio create a project by selecting File > New > Project
- Select Visual C# (here we are going to use C# for development you can choose Php, Python or F#) any of your choice.
- Under the Visual C# select the Azure Functions and click on Next
- The next step is select a new Azure Function Application, Select the Timer trigger function choose your Azure Function version (v1, v2 or v3). Select Authorization level as Anonymous as we don’t want to include API Key for the function and kept the other settings as it is.
- After clicking on Create button, it will create the Azure Function App Solution for us, the directory structure is as follow.
- The next step is to publish the function on Azure, it will deploy your function on IIS and Azure. That we will see in next part of this blog.