Category Archives: Azure Integration
Dynamics 365 Project Operations and Business Central Integration: Automate Data Sync Without Manual Work
Please find this CloudFronts Product at – CloudFronts PO-BC Integration Module | Microsoft Marketplace Watch the webinar – CloudFronts PO-BC Integration Webinar | YouTube Summary Dynamics 365 Project Operations (PO) holds customers, projects and time. Business Central (BC) holds invoices and payments. When the two are not connected, people copy data by hand, which is slow and causes mistakes. The CloudFronts PO-BC Integration Module is a Dynamics 365 Project Operations and Business Central integration. You set up the link once, and every change in PO shows up in BC automatically. The result is no double typing, fewer mistakes and faster billing. This blog shows how it works, step by step. Table of Contents 1Introduction 2Business Challenge 3Why an Integration Hub? 4Implementation Procedure 5End-to-End Working 6Why This Approach Works 7Real-World Use Cases 8Frequently Asked Questions 9Conclusion Introduction: Why Connect Dynamics 365 Project Operations with Business Central Many project-based businesses use two systems. Project Operations (PO) is where the sales and project teams keep customers, projects and time. Business Central (BC) is where the finance team keeps invoices, payments and money records. Both systems need the same details. A customer made in PO must also exist in BC. A project made in PO must also be there when it is time to send an invoice. Without a link, someone has to copy all of this by hand. Here is the full picture of how work moves from a lead to a payment. The top row is PO and the bottom row is BC. The arrows show where the two systems share data. The full flow from lead to payment, and where PO and BC share data. The Solution: We built an Integration Hub. It is a simple web app where you connect PO and BC, set the rules once, and let the data move on its own. Business Challenge: Manual Data Entry Between PO and BC Think about a normal project business. The sales team makes a customer in PO. The project team makes a project and adds tasks. The finance team then needs the same customer and project in BC to send invoices. These details need to be shared between the two systems: Customers and contacts Projects and tasks Planned work and costs Approved time and expenses Vendor invoices and purchase orders Paid status of invoices When people copy these by hand, many problems come up: Extra effort: The same details are typed in two systems. Mistakes: A wrong phone number or address is easy to type. Details that do not match: The two systems slowly stop agreeing with each other. Late invoices: If data reaches finance late, money comes in late. Hard to grow: More customers and projects mean more copying and checking. The Objective: When something changes in PO, BC should get the change by itself. No one should have to copy anything by hand. Why Use an Integration Hub for PO and BC Integration? The Integration Hub is one place where you create and manage every link between PO and BC. You fill a form, choose what to send, and set simple rules. You do not need to write a lot of code. With the Integration Hub, you can: Set up each link once Choose what data moves, from where, and to where Write simple rules for which detail goes where Test the result before it goes live See all links in one list, with their status and last runs Add new links as your business grows In simple words: Think of the Integration Hub as a control room. From here you decide what data moves, from where, and to where. Implementation Procedure: How to Integrate Project Operations with Business Central Step 1 – Choose the Project On the first screen you pick a project. A project here is a group of links. We use the project named PO-BC, which is for Project Operations and Business Central. Choose the project. We are using PO-BC. Step 2 – See the List of Integrations Inside PO-BC you can see all the links. Each row shows its order number, its status, and its last few runs. You can run, edit or deploy each one from here. To add a new one, click New Integration. The list of integrations. Click New Integration to add one. Step 3 – Fill the New Integration Form A form opens on the right side. Here you tell the platform: Integration Name: A name for this link. For example, Accounts. Integration Type: Master means basic details that rarely change, like customers. Transactional means things that happen often, like invoices. Source System and Source Entity: Where the data starts, and what kind of record it is. Here the system is PO, and the record type is accounts. Target System and Target Entity: Where the data goes, and what kind of record it becomes. Here the system is BC, and the record type is customer. Sequence No.: The order in which links run. Customers must exist first, so this is 1. The new integration form. Use Sync CRM Fields to add any missing fields that BC needs on the PO side. Tip: There is a handy button called Sync CRM Fields. Sometimes BC needs a detail that PO does not have yet. This button creates the missing field in PO for you. Scroll down and add a short description. This helps everyone understand later what the link does. Then click Create Integration. Add a clear description, then click Create Integration. Step 4 – Open the Flow After you create the link, you can open its flow. It is a simple picture with boxes from left to right: Source: the PO data (accounts) Data Mapper: the rules that change the data into the shape BC needs Target: the BC data (customer) Return to Source: an optional step to send data back to PO The flow. The Data Mapper box is where we set the rules. Step 5 – Write the Mapping Rules PO and BC use different names for … Continue reading Dynamics 365 Project Operations and Business Central Integration: Automate Data Sync Without Manual Work
Share Story :
From Project Operations to Business Central: Following the Data Behind Project Financials
Summary Connecting Dynamics 365 Project Operations with Business Central creates a continuous data journey from project planning and execution to financial visibility. Project and task information, planning lines, time, materials, expenses, billable values, and invoices must retain their identity and meaning as they move between systems. This implementation focuses on preserving those relationships so that Budget, Actual, Billable, and Invoiced values remain traceable and meaningful throughout the project lifecycle. The result is an integration that does more than synchronize records: it creates a reliable data story that supports project operations, ERP processes, financial analysis, and decision-making. Table of Contents Why Project Data Needs a Journey The Data Journey Entity-to-Entity Mapping Planning Lines Budget vs. Actual vs. Billable vs. Invoiced Execution and Project Journals From Activity to Invoice The Analytics View Implementation Lessons Conclusion The Project Does Not Live in One System Project Operations is where teams plan and manage project work. Business Central is where the project becomes part of an ERP financial workflow. The interesting part is the space between them. When the same customer, project, task, resource, or transaction crosses that boundary, the integration has to preserve more than a name. It has to preserve relationships and translate the source record into the target system’s business model. The Engineering Question “Did the record sync?” is only the first question. “Does the record still mean the same thing?” is the engineering question. CRM Project Operations Where the Project Is Planned Customer, project, tasks, resources, estimates, time, and project activity are planned and managed here. ERP Business Central Where the Project Becomes Financial Project tasks, planning lines, journals, invoices, and financial visibility form the ERP financial structure. CRM side Figure 1: A Project Operations project with stages, customer, dates, estimates, and project context. ERP side Figure 2: The corresponding project structure in Business Central. A Different Way to Look at the Integration Think of It as a Data Pipeline Source entities become mapped entities, mapped entities become ERP records, and ERP records eventually become transactions and financial metrics. How a record moves between the two systems Project Operations projects · tasks · resources → Mapping Layer identity · transformation → Business Central projects · planning lines 01 Plan Estimates, tasks, resources, and planned quantities describe what the project is expected to consume. 02 Execute Time, materials, and expenses create operational evidence of what happened. 03 Bill Billable value moves toward invoicing, turning project activity into commercial output. Entity-to-Entity Mapping Is Where Meaning Gets Preserved One of the most important parts of a Project Operations to Business Central integration is deciding what a source entity becomes in Business Central. The mapping is effectively a semantic contract. Project Operations Business Central Why It Matters Account (Customer) Customer Customer identity must remain consistent across systems Project msdyn_project Project / Job The project becomes the parent financial structure Project Task msdyn_projecttask Project Task / Job Task Task identity is required for planning and transaction attribution Estimate msdyn_estimateline Job Planning Line Planning becomes a structured Business Central financial planning record Time Entry msdyn_timeentry Project Journal Recorded effort can become project usage Figure 3: Account data flowing into Business Central as a customer, carrying its identity across the boundary. Planning Lines Are the Semantic Bridge Planning lines sit at the point where a project plan starts becoming an ERP financial model. In Business Central, planning lines can represent planned cost, billable value, or both, depending on how they are configured. That makes them especially important when explaining the transition from “what we intend to do” to “what the ERP expects this project to cost or generate.” Quantity How much work, material, or resource usage is expected? Unit Cost / Price What cost or commercial rate is associated with the planned quantity? Total Cost / Price The planning line turns quantity and rates into project-level financial expectations. CRM side Figure 4: The work breakdown structure decomposes project work into manageable tasks. ERP side Figure 5: Resource assignments become structured planning lines in Business Central. The Four Numbers People Confuse Most These values answer different questions. Treating them as interchangeable is one of the fastest ways to misread a project dashboard. 01 Budget: What Did We Expect to Spend? Budget represents planned cost. It is a planning value, not proof that the cost has already occurred. 02 Actual: What Cost Was Recorded? Actual (usage) cost is tied to execution and recorded project usage. A blank actual does not mean the budget disappeared; it means no actual usage has been recorded in that context. 03 Billable: What Value Can Be Charged? Billable represents chargeable project value according to the project’s planning and billing configuration. 04 Invoiced: What Has Actually Been Billed? Invoiced is downstream of billable value. A project can have billable value before the corresponding amount has been invoiced. The Simplest Mental Model Budget → Actual answers the cost question. Billable → Invoiced answers the revenue and billing question. Figure 6: Project budgeting concepts in Project Operations (source: Microsoft Learn). Execution Creates the Evidence Once the project moves from planning into execution, operational records begin to tell the story of what actually happened. Time entries, material usage, and expenses are examples of records that contribute to project actuals and downstream ERP transactions. What feeds the project journal Time hours worked + Materials items consumed + Expenses project spend → Project Journal CRM side Figure 7: Time entries recorded against projects and tasks. ERP side Figure 8: Approved time entries represented in the Business Central project journal. Figure 9: Material usage follows the same project-to-ERP transaction path. Engineering Insight An integration should be traceable backwards. If a number appears in an ERP report, you should be able to explain which transaction produced it and which project and task it belongs to. From Project Activity to Invoice The final part of the journey is commercial. Billable project data eventually feeds invoicing, creating a visible chain from planned work to customer-facing financial output. 01 Project Activity Time, materials, expenses, and … Continue reading From Project Operations to Business Central: Following the Data Behind Project Financials
Share Story :
How to Integrate Dynamics 365 Project Operations with Business Central Using Dataverse
Summary Project-based organizations often manage project delivery in Microsoft Dynamics 365 Project Operations while relying on Microsoft Dynamics 365 Business Central for finance, purchasing, inventory, and accounting. Without a connected process, project teams and finance teams can end up working with different versions of the same information. CloudFronts built an integration between Project Operations and Business Central using Dataverse plugins. Instead of relying on users to manually recreate records, the integration responds to important Project Operations events and sends the relevant data to Business Central through web services. It covers customers, projects, timesheets, expenses, invoices, and inventory availability, writes the Business Central identifier back to the originating Project Operations record, and captures failed transactions in an Error Log for troubleshooting. Table of Contents The Project Integration Challenge Our Integration Architecture Keeping Project Customers in Sync Connecting Projects to Business Central Synchronizing Timesheets Moving Project Expenses into Finance Connecting Project Invoices End-to-End Project Integration Flow Business Benefits Key Implementation Considerations Frequently Asked Questions Conclusion About the Example in This Post To show how the integration behaves end to end, the screenshots use a sample customer, Westview Ltd, and its project, Westview-Project-003. These are sandbox records used purely to demonstrate the data flow between Project Operations and Business Central. The Project Integration Challenge Project-based businesses have a unique challenge: the people delivering the project and the people managing its financial impact often work in different systems. Project Operations can be the place where project teams manage customers, projects, resources, tasks, time, expenses, and project billing. Business Central, meanwhile, can be responsible for financial management, purchasing, inventory, and other ERP processes. When these systems are not connected, the same project information may have to be entered more than once. A project manager may record time in Project Operations while finance needs the same information in Business Central. A project invoice may be created in one system while the corresponding financial information needs to be reflected in another. The Core Problem Project teams need to work in the system designed for project delivery, while finance needs accurate and timely information in the ERP. Manual re-entry creates extra effort, delays, and opportunities for data inconsistency. The objective of the integration was therefore not simply to move records between two applications. It was to connect the project lifecycle with the corresponding Business Central processes. Our Integration Architecture The integration uses Dataverse plugins as the event-driven layer connecting Project Operations with Business Central. A Dataverse plugin is a piece of C# code that runs when a Dataverse record is created, updated, or otherwise processed. This provides an opportunity to respond to project-related events and send the relevant information to Business Central. High-Level Flow A user creates or updates a project-related record in Project Operations. The corresponding Dataverse event triggers the plugin. The plugin reads the required information from Dataverse. The data is transformed into the structure required by Business Central. The plugin calls the relevant Business Central web service. Business Central creates or updates the corresponding record. The Business Central identifier is written back to the Project Operations record where required. If the request fails, the integration captures the error and the data involved in an Error Log. Why an Event-Driven Approach? The integration reacts when project information changes, instead of depending on users to remember when and where the same information must be entered. Figure 1: High-level architecture of the Project Operations to Business Central integration. Keeping Project Customers in Sync Customers are a fundamental dependency for project processes. Before project transactions can be represented correctly in Business Central, the corresponding customer needs to exist there. When a customer is created or processed in Project Operations, the integration sends the relevant customer information to Business Central. What the Integration Achieves 01 Create or Sync Creates or synchronizes the customer in Business Central. 02 Map the Fields Uses the appropriate source-to-target field mapping. 03 Capture the Number Captures the Business Central customer number. 04 Write It Back Writes the Business Central identifier back to the originating record where required. This creates an important link between the two systems. The Project Operations record retains the Business Central number, so users do not have to search manually for the corresponding ERP record. In our example, Westview Ltd is created as an account in Project Operations. The integration synchronizes it to Business Central and writes the resulting Business Central customer number (C00280) back onto the Project Operations record. Figure 2: Westview Ltd in Project Operations, with the Business Central customer number (C00280) written back to the int_sourceid field. Figure 3: The corresponding Westview Ltd customer record (C00280) synchronized into Business Central. Connecting Projects to Business Central Customer synchronization is only the beginning. The central object in a project-based integration is the project itself. When a project is created or prepared for synchronization, the integration retrieves the project information required by Business Central and creates the corresponding project record. It then maintains the relationship between the Project Operations project and the Business Central project using the relevant identifiers. Why Project Identifiers Matter A Project Operations record may contain Dataverse identifiers and lookup references, while Business Central works with its own project numbers. These identifiers cannot simply be assumed to be interchangeable. The integration needs to resolve the correct project identifier and use it when subsequent project-related transactions are sent to Business Central. Project Integration Principle Synchronizing a project is not only about copying project fields. It establishes the reference that later transactions, such as time, expenses, materials, and billing, use to reach the correct Business Central project. Continuing the example, Westview-Project-003 is created under Westview Ltd in Project Operations. The integration creates the matching project in Business Central and writes the Business Central project number (PR00340) back to the int_sourceid field on the Project Operations record. Figure 4: Westview-Project-003 in Project Operations, with the Business Central project number (PR00340) written back to int_sourceid. Figure 5: The corresponding Project Card (PR00340) for Westview-Project-003 in Business Central. Synchronizing Timesheets Time is one … Continue reading How to Integrate Dynamics 365 Project Operations with Business Central Using Dataverse
Share Story :
From Trading Partner cXML to Infor LN Sales Order: How a European Life-Sciences Manufacturer Automated EDI Order Intake with Azure Integration Services
Summary A global life-sciences and laboratory equipment supplier sends purchase orders to our customer (BÜCHI’s customer-centric vision accelerates innovation using Azure Integration Services | Microsoft) as machine-generated cXML OrderRequest.xml documents. The customer runs Infor LN, which expects a SalesOrder BOD delivered through the Infor ION I/O Box. Between the two sat a translation problem that was being solved by hand. Every order arriving from the trading partner had to be read, interpreted and re-keyed into Infor LN by the order desk. That worked at low volume. It did not scale, it delayed order confirmation by hours, and it introduced transcription errors on the fields that matter most — quantities, delivery dates and ship-to addresses. We closed that gap with Azure Integration Services: API Management as the secure front door, Blob Storage as the immutable archive, Service Bus for asynchronous decoupling and retries, Azure Functions in C# for the cXML-to-BOD translation, and a write into the Infor ION I/O Box SQL database from which ION creates the sales order in LN. Key Vault and App Configuration hold secrets and environment-specific values; Application Insights makes every transaction traceable end to end. The result is a straight-through process: Partner posts cXML → Archive → Queue → Transform to BOD → I/O Box → Infor ION → Sales Order in LN. This article covers how the interface was designed, how the mapping and its business rules were kept maintainable, how failures are handled and replayed, and the guardrails that keep an asynchronous EDI interface trustworthy in production. Table of Contents From Manual Order Entry to Automated Order Intake The Business Challenge Solution Overview Receiving the Purchase Order: Azure API Management Decoupling the Interface: Azure Service Bus Translating cXML into the Infor Process.SalesOrder BOD The Business Rules Hidden Inside the Mapping Writing to the Infor ION I/O Box Handling Exceptions, Retries and Replay Technical Architecture Monitoring with Application Insights Designing the Integration Around Business Events Business Impact Design Constraints and Guardrails Final Thoughts This Blog Explains Why manual order entry from EDI purchase orders stops scaling, and what it costs. How a partner cXML OrderRequest is received, archived and queued on Azure. How the cXML document is translated into an Infor Process.SalesOrder BOD. How conditional business rules — partner cross-references, note construction, ship-date fallbacks — are kept out of hard-coded logic. How multi-line orders are looped into SalesOrderPosition blocks. How the BOD is handed to Infor LN through the ION I/O Box. How failures are retried, dead-lettered and replayed without asking the partner to resend. How a single transaction ID makes the whole interface searchable in Application Insights. From Manual Order Entry to Automated Order Intake For manufacturers that sell through large, process-driven customers, the order does not arrive as a phone call or an email attachment. It arrives as a machine-generated EDI document, posted to an endpoint at any hour of the day, in a format defined entirely by the buyer. Our customer — a European division of a global life-sciences and laboratory equipment supplier — runs Infor LN as its ERP. Their key trading partner sends purchase orders as cXML OrderRequest.xml documents. Infor LN, on the other side, expects a Process.SalesOrder BOD delivered through the Infor ION I/O Box. Both systems were working exactly as designed. The problem lived in the space between them, and that space was being crossed by a person with a keyboard. A purchase order that arrives electronically and is then typed in by hand is not an integrated process. It is a manual process with an electronic first step. The objective of the project was therefore narrow and concrete: when the trading partner posts an order, a sales order should appear in Infor LN — correctly mapped, without human intervention, and with enough visibility that the support team can prove it happened. The Business Challenge On the surface this looks like a file conversion. In practice, five things made it anything but. 1. Two Schemas That Share Almost No Vocabulary cXML is a flat, attribute-heavy commerce format. The Infor BOD is a deeply nested OAGIS structure with an ApplicationArea envelope, a DataArea payload and a UserArea carrying LN-specific properties. Almost every field required transformation rather than a straight copy. 2. Business Logic Hidden Inside the Mapping The partner’s identity codes had to be cross-referenced to Infor LN business partner IDs — with different values in Test and Production. Header comments needed a hardcoded Remark: prefix. Goods marks, customer references and order numbers had to be concatenated into a single footer note, skipping the values that did not arrive so no blank lines were left behind. 3. A Ship Date With a Rule of Its Own Requested ship date could not simply be copied. If the line carries a date, use it. If it is missing, or in the past, use the current date. If the order has multiple lines with different dates, take the earliest. A rule that reads as one sentence in a specification becomes a decision tree in code. 4. Line-Level Repetition A single order can contain many ItemOut elements, each of which becomes a SalesOrderPosition block in the BOD, carrying its own item ID, quantity, required delivery date and note. 5. No Second Chances, and No Visibility The exchange is asynchronous and real-time. If a message failed silently, the first sign of trouble would be the customer asking why their order had not shipped. Operations needed to answer “what happened to this order?” without raising a support ticket. Manual re-keying was the fallback, and it carried exactly the costs you would expect: order entry delays measured in hours, transcription errors on quantities and delivery dates, and a team doing work that added no value. Solution Overview We built the interface on Azure Integration Services as a set of small, independently deployable components rather than one monolithic job. The solution uses: Azure API Management Azure Blob Storage Azure Service Bus Azure Functions (C#) Infor ION I/O Box (SQL) Azure Key Vault and App Configuration Application … Continue reading From Trading Partner cXML to Infor LN Sales Order: How a European Life-Sciences Manufacturer Automated EDI Order Intake with Azure Integration Services
Share Story :
How a Self-Service Data Portal Solved Multi-Language and Domain Value Chaos in ERP Data
Summary Enterprises running large, multi-country ERP systems often extract data that is technically complete but practically unusable, split across duplicate language columns and encoded with undocumented numeric values. We built a self-service data platform on Azure so that business users, not just data engineers, could define, validate, and process ERP extracts without writing a single line of code. The solution resolves two of the most common ERP data problems: a single field like “Item Description” spread across nine language-specific columns, and reference fields like “Order Status” stored only as numeric codes. A custom web portal puts business users in control of table specifications, validation rules, and processing status, while Azure Databricks and Delta Lake quietly do the heavy lifting behind the scenes. Business impact: dozens of ERP tables moved from raw, multi-language, code-heavy extracts to a single, trusted, human-readable data layer, without adding headcount to the data engineering team. Table of Contents 01 About the Customer 05 Self-Service Data Onboarding 02 The Challenge 06 Medallion Architecture 03 The Solution 07 Business Impact 08 FAQs 09 Conclusion About the Customer Customer Spotlight A Leading Digital Transformation Partner — Europe Our customer is a leading enterprise headquartered in Europe, operating across diverse manufacturing and supply chain divisions. Having already standardized their ERP data through a medallion architecture on Databricks, leadership wanted to go a step further: not only manage ERP data at scale, but also connect it seamlessly into process mining tools to uncover how core processes truly perform in practice. The focus was on gaining operational clarity into workflows such as purchase order management, invoice handling, and procurement cycles. The Challenge Most organizations extracting data from a large ERP system successfully get the data out. The problem isn’t extraction, it’s making that data mean something the moment it lands. Business and IT teams found themselves asking the same questions on repeat: 1Why does the same field appear nine times, with a different value in each column? 2What does “Order Status= 3” actually mean, and who is the source of truth for that mapping? 3How much manual translation and lookup work happens before a single report can be trusted? 4Can business users resolve these issues themselves, without waiting weeks on an IT backlog? 5How do we scale this across dozens of tables without writing dozens of one-off scripts? Two problems came up again and again, and both are far more common across ERP implementations than most leadership teams realize. Multi-Language Columns Because the ERP system was configured for every Order Status the business operates in, a single logical field such as “Item Description” existed as up to nine separate columns, one per language: English, French, German, Spanish, and more. Reports built directly on top of the raw extract had no reliable way of knowing which column to use for which record. In practice, this meant a plant manager in France could open a report and see item names in German, while a sales report for the Spanish market silently pulled blank fields because the Spanish-language column hadn’t been populated for that record. The data was all there; it just wasn’t usable without someone manually deciding, table by table, which language column to trust. Undocumented Domain Values Reference fields like Country, Currency, and Order Status were stored as raw numeric codes rather than readable labels, for example Order Status: 1 = Completed , 2 = In Progress, 3 = Shipped. These mappings lived inside ERP configuration screens, not in the extracted data itself. That meant every downstream report, dashboard, or spreadsheet needed its own copy of the same lookup table, manually kept in sync. When a code changed or a new Order Status was added in the ERP, there was no guarantee every report using it would be updated at the same time, which meant leadership could be looking at the performance chart that was quietly wrong. The Solution Rather than writing custom transformation logic for every table (a solution that ages badly the moment a new table or region gets added), we designed a configuration-driven pipeline built on Azure Databricks, fronted by a self-service web application that puts control directly in the hands of business and functional users. Self-Service Web Portal Business users upload table specifications, review validation results, and queue tables for processing, entirely through a browser. Medallion Architecture Azure Databricks and Delta Lake refine raw extracts through Bronze, Silver, and Gold layers, without table-specific code. Automated Language Resolution Multi-language columns are detected and normalized automatically based on the specification, not hardcoded per table. Centralized Domain Mapping Numeric and coded reference values are resolved against a single, maintained lookup layer instead of scattered spreadsheets. Self-Service Data Onboarding: No Databricks Knowledge Required The centerpiece of the solution is a custom web application that lets a business or functional analyst, not a Databricks engineer, onboard a new ERP table from start to finish. Here’s what that looks like in practice: A business user uploads an Excel-based table specification defining the expected columns, data types, which fields are multi-language, and which fields are domain-coded and how to decode them. The portal validates the specification instantly, flagging missing mandatory columns or mismatches before any data is processed, so problems are caught at the source rather than three reports downstream. Once validation passes, the same user queues the table for processing with a single click. No notebook to open, no cluster to configure, no code to write or review. Behind the scenes, that specification feeds a generic, reusable Databricks framework that already knows how to apply the correct language resolution and domain-value decoding rules, so engineering effort doesn’t scale linearly with the number of tables. In effect, the portal turns “add a new ERP table to the analytics environment” from a data engineering request into a form a finance or operations analyst can complete in minutes, while still enforcing the same rigor and consistency a hand-built pipeline would require. Medallion Architecture on Databricks Once a table is queued through the portal, Azure Databricks takes over: Bronze: Raw ERP extracts are landed as-is, preserving … Continue reading How a Self-Service Data Portal Solved Multi-Language and Domain Value Chaos in ERP Data
Share Story :
Payroll Transformation for a Global Hardware Manufacturer Using Zoho People and Finance & Operations
As businesses scale, payroll complexity grows bringing challenges around employee data, attendance, compensation structures, and compliance. Manual processes not only consume valuable time but also increase the risk of costly errors. The Zoho People-FNO integration transforms payroll into a streamlined, automated process, ensuring accurate salary calculations, seamless data synchronization, and complete transparency across HR and finance operations. We recently implemented this solution for a global manufacturing hardware enterprise, enabling them to automate payroll workflows, eliminate manual data reconciliation, improve payroll accuracy, and reduce administrative overhead. The integration provided a scalable foundation for managing a growing workforce while maintaining compliance and enhancing the employee experience through faster, more transparent payroll processing. For organizations focused on operational efficiency and sustainable growth, this integration delivers measurable business value from day one. Understanding the Architecture of Zoho and FNO Integration The integration between Zoho People and FNO involves a clear, structured workflow. Below is an overview of the steps involved: This architecture ensures a smooth flow of data between Zoho and FNO, simplifying payroll management for businesses of all sizes. Key Advantages of Zoho and FNO Integration The integration between Zoho People and FNO streamlines payroll management, providing several key benefits: Effortless Payroll Management for Growing Businesses To conclude, efficient payroll management is essential for any growing business. By integrating Zoho People with FNO, businesses can automate payroll processes, ensure accurate calculations, and provide employees with easy access to their payslips. The seamless data flow, real-time updates, and reduced manual intervention significantly improve operational efficiency and transparency. If you’re ready to optimize your payroll system, now is the time to take action. Embrace the Zoho and FNO integration to simplify your processes, reduce errors, and create a transparent payroll system that benefits both your employees and your organization. Contact us today to learn how this integration can simplify your payroll management process. Reach out at transform@cloudfronts.com.
Share Story :
From Raw Data to Insights: ETL Best Practices with Azure Databricks
Organizations today generate massive volumes of raw data from multiple sources such as ERP systems, CRMs, APIs, logs, and IoT devices. However, raw data by itself holds little value unless it is properly processed, transformed, and optimized for analytics. In our data engineering journey, we faced challenges in building scalable and maintainable ETL pipelines that could handle growing data volumes while still delivering reliable insights. Azure Databricks helped us bridge the gap between raw data and business-ready insights. In this blog, we’ll walk through ETL best practices using Azure Databricks and how they helped us build efficient, production-grade data pipelines. Why ETL Best Practices Matter When working with large-scale data pipelines: – Raw data arrives in different formats and structures– Poorly designed ETL jobs lead to performance bottlenecks– Debugging and maintaining pipelines becomes difficult– Data quality issues propagate to downstream reports Key challenges we faced: – Tight coupling between ingestion and transformation– Reprocessing large datasets due to small logic changes– Lack of standardization across pipelines– Slow query performance on analytical layers Solution Architecture Overview Key Components: – Azure Data Lake Storage Gen2– Azure Databricks– Delta Lake– Power BI / Analytics Tools ETL Flow: – Ingest raw data from source systems into the Raw (Bronze) layer– Clean, validate, and standardize data in the Processed (Silver) layer– Apply business logic and aggregations in the Curated (Gold) layer– Expose curated datasets to reporting and analytics tools Step-by-Step ETL Best Practices with Azure Databricks Step 1: Separate Data into Layers (Bronze, Silver, Gold) – Bronze Layer: Store raw data exactly as received– Silver Layer: Apply cleansing, deduplication, and schema enforcement– Gold Layer: Create business-ready datasets and aggregations This separation ensures reusability and prevents unnecessary reprocessing. Step 2: Use Delta Lake for Reliability – Store tables in Delta format– Enable schema enforcement and schema evolution– Leverage time travel for data recovery and debugging Step 3: Build Incremental Pipelines – Process only new or changed data using watermarking– Avoid full reloads unless absolutely required– Design pipelines to safely re-run without duplications Step 4: Parameterize and Modularize Code – Use notebook parameters for environment-specific values– Create reusable functions for common transformations– Avoid hardcoding paths, table names, or business rules Step 5: Optimize Performance Early – Use partitioning based on query patterns– Apply Z-ORDER on frequently filtered columns– Cache datasets selectively for heavy transformations Step 6: Implement Data Quality Checks – Validate nulls, ranges, and duplicate records– Log rejected or invalid records separately– Fail pipelines early when critical checks fail Benefits of Following These ETL Best Practices – Scalability: Easily handle growing data volumes– Reliability: ACID-compliant pipelines with Delta Lake– Maintainability: Modular and reusable code structure– Performance: Faster queries and optimized storage– Cost Efficiency: Reduced compute usage through incremental processing Conclusion Transforming raw data into meaningful insights requires more than just moving data from one place to another. By following ETL best practices with Azure Databricks, we were able to build robust, scalable, and high-performing data pipelines that deliver reliable insights to the business. If your Databricks pipelines are becoming complex, slow, or difficult to maintain, it might be time to revisit your ETL design. Start applying these best practices today and turn your raw data into insights that truly drive decision-making. I hope you found this blog useful, and if you would like to discuss anything, you can reach out to us at transform@cloudfronts.com.
Share Story :
How to Build an Incremental Data Pipeline with Azure Logic Apps
Why Incremental Loads Matter When integrating data from external systems, whether it’s a CRM, an ERP like Business Central, or an HR platform like Zoho People, pulling all data every time is expensive, slow, and unnecessary. The smarter approach is to track what has changed since the last successful run and fetch only that delta. This is the core idea behind an incremental data pipeline: identify a timestamp or sequence field in your source system, persist the last-known watermark, and use it as a filter on your next API call. Azure Logic Apps, paired with Azure Table Storage as a lightweight checkpoint store, gives you everything you need to implement this pattern without managing any infrastructure. Architecture Overview Instead of one large workflow doing everything, we separate responsibilities. One Logic App handles scheduling and orchestration. Another handles actual data extraction. Core components: 3. Metadata Design (Azure Table) Instead of hardcoding entity names and fields inside Logic Apps, we define them in Azure Table Storage. Example structure: PartitionKey RowKey IncrementalField displayName entity businesscentral 1 systemCreatedAt Vendor Ledger Entry vendorLedgerEntries zohopeople 1 modifiedtime Leave leave Briefly, this table answers three questions: – What entity should be extracted?– Which column defines incremental logic?– What was the last successful checkpoint? When you want to onboard a new entity, you add a row. No redesign needed. 4. Logic App 1 – Scheduler Trigger: Recurrence (for example, every 15 minutes) Steps: This Logic App should not call APIs directly. Its only job is orchestration. Keep it light. 5. Logic App 2 – Incremental Processor Trigger: HTTP (called from Logic App 1) Functional steps: Example: This is where the real work happens. 6. Checkpoint Strategy Each entity must maintain: – LastSuccessfulRunTime– Status– LastRecordTimestamp After successful extraction: Checkpoint = max(modifiedOn) from extracted data. This ensures: Checkpoint management is the backbone of incremental loading. If this fails, everything fails. This pattern gives you a production-grade incremental data pipeline entirely within Azure’s managed services. By centralizing entity configuration and watermarks in Azure Table Storage, you create a data-driven pipeline where adding a new integration is as simple as inserting a row — no code deployment required. The two-Logic-App architecture cleanly separates orchestration from execution, enables parallel processing, and ensures your pipeline is resilient to failures through checkpoint-based watermark management. Whether you’re pulling from Business Central, Zoho People, or any REST API that exposes a timestamp field, this architecture scales gracefully with your data needs. Explore the case study below to learn how Logic Apps were implemented to solve key business challenges: Ready to deploy AIS to seamlessly connect systems and improve operational cost and efficiency? Get in touch with CloudFronts at transform@cloudfronts.com.
Share Story :
From Legacy Middleware Debt to AI Innovation: Rebuilding the Digital Backbone of a 150-Year-Old Manufacturer
Summary A global manufacturing client was facing rising middleware costs, poor visibility, and growing pressure to support analytics and AI initiatives. A forced three-year middleware commitment became the trigger to rethink their integration strategy. This article shares how the client moved away from legacy middleware, reduced integration costs by nearly 95%, improved operational visibility, and built a strong data foundation for the future. Table of Contents 1. The Middleware Cost Problem 2. Building a New Integration Setup 3. Making Integrations Visible 4. Preparing Data for AI 5. How We Did It Savings Metrics The Middleware Cost Problem The client was running critical integrations on a legacy middleware platform that had gradually become a financial and operational burden. Licensing costs increased sharply, with annual fees rising from $20,000 to $50,000 and a mandatory three-year commitment pushing the total to $160,000. Despite the cost, visibility remained limited. Integrations behaved like black boxes, failures were difficult to trace, and teams relied on manual intervention to diagnose and fix issues. At the same time, the business was pushing toward better reporting, analytics, and AI-driven insights. These initiatives required clean and reliable data flows that the existing middleware could not provide efficiently. Building a New Integration Setup Legacy middleware and Scribe-based integrations were replaced with Azure Logic Apps and Azure Functions. The new setup was designed to support global operations across multiple legal entities. Separate DataAreaIDs were maintained for regions including TOUS, TOUK, TOIN, and TOCN. Branching logic handled country-specific account number mappings such as cf_accountnumberus and cf_accountnumberuk. An agentless architecture was adopted using Azure Blob Storage with Logic Apps. This removed firewall and SQL connectivity challenges and eliminated reliance on unsupported personal-mode gateways. Making Integrations Visible The previous setup offered no centralized monitoring, making it difficult to detect failures early. A Power BI dashboard built on Azure Log Analytics provided a clear view of integration health and execution status. Automated alerts were configured to notify teams within one hour of failures, allowing issues to be addressed before impacting critical business processes. Preparing Data for AI With stable integrations in place, the focus shifted from cost savings to long-term readiness. Clean data flows became the foundation for platforms such as Databricks and governance layers like Unity Catalog. The architecture supports conversational AI use cases, enabling questions like “Is raw material available for this production order?” to be answered from a unified data foundation. As a first step, 32 reports were consolidated into a single catalog to validate data quality and integration reliability. How We Did It Retrieve config.json and checkpoint.txt from Azure Blob Storage for configuration and state control. Run incremental HTTP GET queries using ModifiedDateTime1 gt [CheckpointTimestamp]. Check for existing records using OData queries in target systems with keys such as ScribeCRMKey. Transform data using Azure Functions with region-specific Liquid templates. Write data securely using PATCH or POST operations with OAuth 2.0 authentication. Update checkpoint timestamps in Azure Blob Storage after successful execution. Log step-level success or failure using a centralized Logging Logic App (TO-UAT-Logs). Savings Metrics 95% reduction in annual integration costs, from $50,000 to approximately $2,555. Approximately $140,000 in annual savings. Integrations across D365 Field Service, D365 Sales, D365 Finance & Operations, Shopify, and SQL Server. Designed to support modernization of more than 600 fragmented reports. FAQs Q: How does this impact Shopify integrations? A: Azure Integration Services acts as the middle layer, enabling Shopify orders to synchronize into Finance & Operations and CRM systems in real time. Q: Is the system secure for global entities? A: Yes. The solution uses Azure AD OAuth 2.0 and centralized key management for all API calls. Q: Can it handle attachments? A: Dedicated Logic Apps were designed to synchronize CRM annotations and attachments to SQL servers located behind firewalls using an agentless architecture. We hope you found this blog useful, and if you would like to discuss anything, you can reach out to us at transform@cloudfronts.com.
Share Story :
Bridging Project Execution and Finance: How PO F&O Connector Unlocks Full Value in Dynamics 365
In a world where timing, accuracy, and coordination make or break profitability, modern project-based enterprises demand more than isolated systems. You may be leveraging Dynamics 365 Project Operations (ProjOps) to manage projects, timesheets, and resource planning and Dynamics 365 Finance & Operations (F&O) for financials, billing, and accounting. But without seamless integration, you’re stuck with manual transfers, data silos, and delayed insights. That’s where PO F&O Connector app comes in built to synchronize Project Operations and F&O end-to-end, bringing together delivery and finance in perfect alignment. In this article, we’ll explore how it works, why it matters to CEOs, CFOs, and CTOs, and how adopting it gives you a competitive edge. The Pain Point: Disconnected Project & Finance Workflows When your project execution and financial systems aren’t talking: The result? Missed revenue, resource inefficiencies, and poor visibility into project financial health. The Solution: Cloudfronts Project-to-Finance Integration App Cloudfronts new app is purpose-built to connect Project Operations → Finance & Operations seamlessly, automating the flow of project data into financial systems and enabling real-time, consistent delivery-to-finance synchronization. Key capabilities include: Role Core Benefits Outcomes CEO Visibility into project margins and outcomes; faster time to value Better strategic decisions, competitive agility CFO Automates billing, enforces accounting rules, ensures audit compliance Revenue gets recognized faster, finance becomes a strategic enabler CTO Reduces custom integration burdens, ensures system integrity Lower maintenance costs, scalable architecture Beyond roles, your entire organization benefits through: We hope you found this blog useful, and if you would like to discuss anything, you can reach out to us at transform@cloudfronts.com
