10 Aug’17

Dynamics 365, EnterpriseIssue faces in retrieving 5000+ records using Web API

Introduction:

While using Fetchxml queries to retrieve more than 5000 records, first page will give you 5000 records and paging cookie. Paging cookie will help to retrieve additional records. For additional records, we need to set the paging cookie in the fetch tag.

Setting the paging cookie:

When the fetch query is executed it brings the paging- cookie with it in the resultant response. The paging cookie is attached to the next Fetch XML

The paging cookie consists of the first and last record details. Name and the GUID of the records.

When trying to execute fetch XML request. it started throwing “Page Cookie Malformed” exception. This is because there is a special character present in the paging Cookie.

You can see in the above cookie that Name has “&” which is the replacement for character “&”. This breaks the fetchXml request.There can be any character present in the record name such as (‘s). Even (‘) breaks the fetch XML.

Thus, we need to encode the paging-cookie before it is inserted in fetchXml.

pageCokies.replace(/&/g, '&');

This will help to solve the issue and we can retrieve more than 5000 records.

Written by

Krishna Bhanushali

Software Developer

2 thoughts on “Issue faces in retrieving 5000+ records using Web API”

  1. hi,
    I am facing a similar issue but I get “required white space missing” every time.

    i have tried a lot but not able to handle the exception.

    I have even changed the retrieved paging cookie format before passing it to string fetchxml.

    if (pagingCookie != null && pagingCookie != “”)
    pagingCookie = pagingCookie.Replace(“\””, “‘”).Replace(“>”, “>”).Replace(“<", "<");

    let me know what i should do.

    1. Hi Garry,

      Try to get your 2nd paging cookie in below format when you are retrieving 5000+ records.

      < fetch count='5000' page='2' paging-cookie='<cookie page="1"><cf_name last="Pre Sales Demo proposal Document" first="ADX Portal" /><cf_projecttaskid last="{1F8421C1-97C2-E611-80FE-3863BB347A58}" first="{EA9A3B6F-C873-E511-811D-C4346BADE5B0}" /></cookie>' version='1.0' >

      Please remove space in parent “< " and ">” of above code, which we have to keep to post code here.

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload the CAPTCHA.

Want to streamline your business processes?

  • This field is for validation purposes and should be left unchanged.

Recent Articles

  • Reading more then 10K records in D3FOE OData API

    11 July’ 2018

    Introduction: We all know Dynamics 365 Finance and Operations has limitation of 10K records to be fetched at a time usi...

    Read more
  • Paging in D365 Customer Engagement v9.0

    10 July’ 2018

    Introduction: The Xrm.retrieveMultipleRecords method is used to retrieve a collection of records in Dynamics 365 Custom...

    Read more
  • Set up Dynamics 365 connection in Microsoft Social Engagement

    10 July’ 2018

    Introduction: This blog explains how to Set up Dynamics 365 connection in Microsoft Social Engagement. Steps to be follo...

    Read more
  • Voice of the Customer failed to install

    10 July’ 2018

    Introduction: Many people face issues in installing Voice of Customer solution on v9 environment and trying repeatedly ...

    Read more
  • Scribe Insight AX as a Web Service Find Block issue

    10 July’ 2018

    Introduction: If we need to look up for any value from AX then we do it by using a Find Block in Scribe Insight Eg: Basi...

    Read more