Performance Optimization Techniques in Power BI
Introduction Building efficient Power BI reports can be challenging, especially when working with large datasets. One common issue Power BI users encounter is the “stack overflow” error, which can disrupt the report-building process. I In this blog I will share some performance optimization techniques that you can use in building power BI report. When using power query or importing data you might have got this error – “Expression.Error: Evaluation resulted in a stack overflow and cannot continue.” This error occurs when there’s a large amount of data is being imported or not enough memory available memory available for Power BI to complete the operation. This issue can be resolved by increasing the Memory and CPU cores that can be used by Power BI while querying or evaluations. There are two settings that we need to keep in mind – By default, the maximum number of simultaneous evaluations is equal to the number of logical CPU cores on the machine and Maximum memory used per simultaneous evaluation is 432 MB. Personally, I have kept these values in between or close to maximum value depending on my requirement and system. Also, here is link to recommendations by Microsoft for managing Power BI workload and evaluation configurations – https://learn.microsoft.com/en-us/power-bi/create-reports/desktop-evaluation-configuration Conclusion Optimizing performance in Power BI is crucial for handling large datasets and preventing issues like the “stack overflow” error. By adjusting settings for simultaneous evaluations and memory allocation, you can significantly improve report processing and responsiveness. We hope you found this article useful, and if you would like to discuss anything, you can reach out to us at transform@cloudfronts.com
Share Story :
JSON to JSON Transformation using Azure Logic Apps and Liquid
Introduction In this blog post, I’ll walk you through the process of transforming JSON to JSON using Azure Logic Apps and the Liquid Template Language. This step-by-step guide will demonstrate how you can use Azure Integration Services to achieve your transformation goals. What is Liquid Template Language? The Liquid Template Language (commonly referred to as “Liquid”) is a flexible, open-source template language developed by Shopify. It is widely used to render dynamic content in platforms such as Shopify themes, Jekyll websites, and web applications. Liquid uses placeholders, loops, and conditional statements to pull dynamic data into a web template, making it an effective tool for JSON transformation. Prerequisites To complete this tutorial, you’ll need: Sample Input JSON We will use the following sample JSON file for this tutorial: { “FirstName”: “Deepak”, “LastName”: “Ch”, “Add1”: “T square, Saki Vihar Road, Andheri East”, “Add2”: “Mumbai”, “Landmark”: “Near Car Showroom”, “PhoneNo1”: 9812727261, “PhoneNo2”: 2121233322 } Desired Output JSON The client’s requirement is to transform the input JSON into the following format: { “Full Name”: “Deepak Ch”, “Address”: “T square, Saki Vihar Road, Andheri East, Mumbai, Near Car Showroom”, “Phone”: “9812727261, 2121233322” } Step-by-Step Guide – Step 1: Create a Free Azure Integration Account Step 2: Add the Liquid Template Map Step 3: Create a Logic App Step 4: Transform JSON to JSON using Liquid Here’s the Liquid template used for this transformation: { “Full Name”: “{{content.FirstName}} {{content.LastName}}”, “Address”: “{{content.Add1}}, {{content.Add2}}, {{content.Landmark}}”, “Phone”: “{{content.PhoneNo1}}, {{content.PhoneNo2}}” } Step 5: Test with Postman Final Output The output JSON will be: { “Full Name”: “Deepak Ch”, “Address”: “T square, Saki Vihar Road, Andheri East, Mumbai, Near Car Showroom”, “Phone”: “9812727261, 2121233322” } We hope you found this article useful, and if you would like to discuss anything, you can reach out to us at transform@cloudfronts.com
Share Story :
Automating Access Token and Refresh Token Generation Using ADF and Azure Key Vault – Part 2
In continuation to our Part 1, welcome to part 2 of the blog on Automating Access Token and Refresh Token Generation Using ADF and Azure Key Vault. We have already completed the necessary setup in part 1, so if you haven’t read part 1 yet, please do so before proceeding with this part. Assumptions- Before going further, let’s first discuss the assumptions we made: Now, let’s discuss the step to create a pipeline to refresh the access token: – – Create a web activity to pull the client ID, client secret, and refresh token you created in part 1. – As for settings, you use this setup, and URI is your Azure key vault’s Secret Identifier. – Similarly, set up web activities for the client ID, client secret, and refresh token. – For the refresh token, I have done setup as shown but you may want to change it according to your API requirements. Body- grant_type=refresh_token&refresh_token=@{activity(‘Get Refresh Token’).output.value} Authorization- Basic @{base64(concat(activity(‘Get Client Id’).output.value, ‘:’, activity(‘Get Client Secret’).output.value))} – After this, use another web activity to refresh the access token using the refresh token and save it to the Azure Key Vault. Body- { “value”: “@{activity(‘Refresh Access Token’).output.access_token}” } Conlusion: This blog provides a comprehensive guide to automating the access token and refresh token generation process using Azure Data Factory and Azure Key Vault. By following the steps outlined, you can ensure seamless token management, reduce manual interventions, and maintain secure access to your resources. We hope you found this article useful, and if you would like to discuss anything, you can reach out to us at transform@cloudfronts.com
Share Story :
Automating Access Token and Refresh Token Generation Using ADF and Azure Key Vault – Part 1
Introduction In this blog, I will explain how we can automate generating access tokens or refresh tokens. When working with APIs, a common problem is the expiration of the access token or refresh token after some time. We solved this issue by using Azure Data Factory and Azure Key Vault. Azure Key Vault is used for storing API credentials as it is one of the most secure ways to store keys/secrets in Azure. Azure Data Factory is used to automate the process of generating access tokens for APIs. We are dividing this blog into two parts: Before we proceed with the blog, please test your API in Postman to know the API requirements for generating access tokens. For me, it is client ID, client secret, and Refresh Token. Steps to Set Up Azure Key Vault and Azure Data Factory: – Go to the Azure portal and create a Key Vault resource. Please make sure that your Key Vault and Azure Data Factory are in the same region. – Create a secret by generate/import and entering the required details. I have already created the secrets I need. – For the access token, you can keep the initial value as anything you want; we will update it later using an ADF Pipeline. – Set up an access policy for the Azure Data Factory to access our Key Vault. To do this, go to “Access Policy” and select the appropriate options. – Click “Next” and select your Azure Data Factory, where you will be creating a pipeline for refreshing the access token. – Now, go to Azure Data Factory Studio and set up the linked Service for your API in the Azure data factory. – The dataset is also pretty straightforward, and I prefer to use a parameter for the relative URL so that I can reuse the same dataset and just set the URL of the API I want to call during runtime: Conclusion That’s all for the setup in part 1. We’ve covered the essential steps to set up Azure Key Vault and Azure Data Factory for securely managing API credentials and setting the groundwork for automating access token generation. These tools provide a reliable and secure way to handle token expiration, ensuring smooth API operations without manual intervention. In part 2, we will discuss in detail how we can automate access token generation using Azure Data Factory. We hope you found this article useful, and if you would like to discuss anything, you can reach out to us at transform@cloudfronts.com
