Automating File Transfers from Azure File Share to Blob Storage with a Function App

Efficient file management is essential for businesses leveraging Azure cloud storage. Automating file transfers between Azure File Share and Azure Blob Storage enhances scalability, reduces manual intervention, and ensures data availability. This blog provides a step-by-step guide to setting up an Azure Timer Trigger Function App to automate the transfer process.

Why Automate File Transfers?

• Eliminates Manual Work – Automating file movement saves time and effort.
• Enhances Storage Optimization – Blob Storage offers better performance and cost efficiency.
• Ensures Security & Compliance – Scheduled transfers ensure regulatory adherence.

Steps to Implement the Solution

1. Prerequisites

To follow this guide, ensure you have:

• An Azure Subscription
• An Azure Storage Account with File Share and Blob Storage
• An Azure Function App with a Timer Trigger
• Azure Storage SDK for C# or Python
• Azure Function Core Tools for development

2. Create a Timer Trigger Function App

1. Access the Azure Portal
   • Go to Function App and click Create.
   • Choose the Consumption Plan for cost-effectiveness.
   • Select your preferred runtime (.NET, Python, or Node.js).
2. Add a Timer Trigger Function
   • In the Function App, create a function and select Timer Trigger.
   • Define the schedule using a CRON expression (e.g., 0 0 * * * * to run hourly).

3. Install Required Packages

For C#:

using Azure.Storage.Files.Shares;
using Azure.Storage.Blobs;

For Python:

pip install azure-storage-file-share azure-storage-blob

4. Implement the File Transfer Logic

C# Implementation

using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using Azure.Storage.Files.Shares;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Extensions.Logging;

namespace MovingFileByTimerTrigger
{
    public class FileShareToBlobFunction(ILogger<FileShareToBlobFunction> logger)
    {
        private readonly ILogger<FileShareToBlobFunction> _logger;
        private readonly string fileShareConnectionString = Environment.GetEnvironmentVariable("AzureWebJobsStorage");
        private readonly string blobStorageConnectionString = Environment.GetEnvironmentVariable("BlobStorageConnection");
        private readonly string shareName = "myfileshare";
        private readonly string directoryName = "";
        private readonly string blobContainerName = "processedfiles";


        [Function("FileShareToBlobFunction")]
        public async Task Run([TimerTrigger("*/5 * * * * *")] FunctionContext context)
        {
            logger.LogInformation("Timer triggered execution started.");

            try
            {
                var share = new ShareClient(fileShareConnectionString, shareName);
                var directory = share.GetDirectoryClient(directoryName);
                var files = directory.GetFilesAndDirectories();

                var blobServiceClient = new BlobServiceClient(blobStorageConnectionString);
                var blobContainerClient = blobServiceClient.GetBlobContainerClient(blobContainerName);

                foreach (var page in files.AsPages())
                {
                    foreach (var fileItem in page.Values)
                    {
                        if (!fileItem.IsDirectory)
                        {
                            var fileClient = directory.GetFileClient(fileItem.Name);
                            var blobClient = blobContainerClient.GetBlobClient(fileItem.Name);

                            bool blobExistsBeforeUpload = await blobClient.ExistsAsync();

                            using (Stream downloadStream = await fileClient.OpenReadAsync())
                            {
                                BlobUploadOptions options = new BlobUploadOptions
                                {
                                    HttpHeaders = new BlobHttpHeaders { ContentType = "application/octet-stream" }
                                };

                                BlobContentInfo blobResponse = await blobClient.UploadAsync(downloadStream, overwrite: true);

                                // Logging blob response details
                                logger.LogInformation($"Uploaded {fileItem.Name} to Blob Storage.");
                                logger.LogInformation($"Blob ETag: {blobResponse.ETag}");
                                logger.LogInformation($"Blob Last Modified: {blobResponse.LastModified}");

                                // Determine if the file was newly created or updated
                                if (blobExistsBeforeUpload)
                                {
                                    logger.LogInformation($"File '{fileItem.Name}' was UPDATED in Blob Storage.");
                                }
                                else
                                {
                                    logger.LogInformation($"File '{fileItem.Name}' was CREATED in Blob Storage.");
                                }
                            }

                            // Ensure the file exists in Blob before deleting from File Share
                            if (await blobClient.ExistsAsync())
                            {
                                await fileClient.DeleteIfExistsAsync();
                                logger.LogInformation($"Deleted {fileItem.Name} from File Share.");
                            }
                        }
                    }
                }
            }
            catch (Exception ex)
            {
                logger.LogError($"Error occurred: {ex.Message}");
            }
        }
    }
}

5. Deploy and Monitor the Function

• Deploy the function using Visual Studio, Azure CLI, or the Azure Portal.
• Monitor execution via Application Insights and Azure Logs.

To conclude, automating file transfers from Azure File Share to Blob Storage using a Timer Trigger Function streamlines operations and enhances reliability. Implementing this solution optimizes file management, improves cost efficiency, and ensures compliance with best practices.

Begin automating your file transfers today! Need expert assistance? Reach out for tailored Azure solutions to enhance your workflow.

We hope you found this blog useful, and if you would like to discuss anything, you can reach out to us at transform@cloudfonts.com.