Handling Pagination in Logic App for JSON payloads having Linked
Introduction
Most of the times the data coming from any API is in Paginated format. API pagination is essential if you’re dealing with a lot of data. When we want to access or process the paginated data from an endpoint/api the logic app can be useful. But the question is how to handle pagination in Logic App?
Our Scenario
The payloads that we are considering have the following structure.
{
“_links”: {
“self”: {
},
“first”: {
},
“next”: {
}
},
“current_page”: 1,
“_embedded”: {
“items”: [
{
“identifier”: “11300M120PL”,
“enabled”: true,
“family”: “unclassified”
},
{
“identifier”: “11300M134TL”,
“enabled”: false,
“family”: “unclassified”
}
]
}
}
Here we API has a maximum limit of 100 records per page. Therefore, we will loop over the pages and process 100 records in each run and go to next page with help of next link in the Payload that we get from API. Note the last page of the API will have next link value as null and this is the condition that we will use in a Until Loop. Follow the steps as a reference to create your own Logic App for the API that you want to query.
Create logic App with HTTP Trigger to pull all data from Akeneo paginated and PUSH to Azure API Management PUT Endpoint.
Step 1: Add a When a HTTP request is received block.
Step 2: Choose an HHTP – choose a REST API Block next and configure it as follows.
Step 3 : Add a Initialize variable Block and configure it as follows.
Step 4: Choose a Parse JSON Data operation block next and configure it as follows
Step 5: Add an Initialize variable block and configure it as follows.
Step 6: Add an Initialize variable block and configure it as follows with value set as: outputs(‘Compose’)?[‘href’]
Step 7: Add an Initialize Variable Block and configure it as follows.
Step 8: Add an nextLink variable and choose is equal to operation
Note: Change Limits of Until Loop to a greater number than default i.e. 60. As per the data load expected.
Step 8.1: Add a HHTP – choose a REST API Block next and configure it as follows
Step 8.2: Add a Parse Json next and configure it as follows
Use a sample payload to generate its schema.
Step 8.3: Add a For Each Control Block and in ‘select output from previous steps’ add items of ‘Parse Body in Json Block’ in expression it has a formula of body(‘Parse_Body_in_JSON’)?[‘_embedded’]?[‘items’].
Step 8.3.1: Add an Append to array variable block and choose jsonarray which you declared earlier and choose the value as Current item of the For Each Loop.
After Entire Configuration the For each – Appending Items to ItemsArray Block looks as follows:
Step 8.4: Add Compose Variable block and configure it as follows with value set as: body(‘Parse_Body_in_JSON’)?[‘_links’]?[‘next’]
Step 8.5: Add Set Variable block and configure it as follows with value set as: outputs(‘Compose’)?[‘href’]
Note: The Until block after configuration must look as below.
Step 10: Add a HTTP Block and configure it as follows to retrieve the Access Token.
Step 11: Add Parse Json block to parse the Token that we received. Generate the schema for the Token from the JSON file of Token.
Step 12: After the Until block add another For each block and pass the ItemsArray.
Step 12.1: Inside the For Each Block add a HTTP block and configure it as follows. In the body section pass the current ‘For each – PUSH to AIS’ item.
After Complete Configuration the Logic App will look like following:
