How to Upload CSV Files to Azure Blob Containers with Azure Blob Storage API

Welcome to today’s post.

I will give an overview of how to upload CSV files into an Azure storage container.

In a previous post, I showed how to upload a CSV file to an Azure storage container using an SSIS package.

I will be using a slightly different approach: using the Azure Blob Storage API and C# to demonstrate this.

What do we know about blob storage in Azure?

• Every storage account in Azure has containers.
• Each container can contain blobs.

We will be uploading the CSV file into the blob.

Using the Azure storage will require obtaining the connection string to the Azure storage account.

This is done as follows.

Login to your Azure subscription.

Go to your Azure storage account.

Open Access Keys.

Copy the Connection string key as shown:

Open a CMD prompt or Powershell.

Apply the command:

setx AZURE_STORAGE_CONNECTION_STRING "<storage account connection string>"

Close the CMD prompt / Powershell session.

Now open Visual Studio.

Create a new console project.

Add the NuGet package Azure.Storage.Blobs as shown:

The code to create a new container is shown:

string connectionString =
Environment.GetEnvironmentVariable( "AZURE_STORAGE_CONNECTION_STRING");
	
// Create a BlobServiceClient object which is used to create a 
// container client
BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);

// Container name
string containerName = "bookloantriggerblobs";
bool bContainerExists = false;
Azure.Storage.Blobs.BlobContainerClient containerClient = null;
Azure.Storage.Blobs.Models.BlobContainerItem containerItem = null;

// Create the container and return a container client object
foreach (Azure.Storage.Blobs.Models.BlobContainerItem 
blobContainerItem in blobServiceClient.GetBlobContainers())
{
       if (blobContainerItem.Name == containerName)
       {
              bContainerExists = true;
              containerItem = blobContainerItem;
              break;
       } 
}

// Create or use existing Azure container as client.
if (!bContainerExists)
{
   containerClient = blobServiceClient.CreateBlobContainer( 
containerName);
}
else
   containerClient = blobServiceClient.GetBlobContainerClient( 
containerName);

Note that existence of existing containers should be checked or else an error will be returned from Azure.

Code to upload the CSV into the Azure container is shown:

// Create a local file in the ./data/ directory for uploading and 
// downloading
string localPath = "./data/";
string fileName = "App Data - New Books.csv";
string localFilePath = Path.Combine(localPath, fileName);

// Get a reference to a blob
BlobClient blobClient = containerClient.GetBlobClient(fileName);

Console.WriteLine("Uploading to Blob storage as blob:\n\t {0}\n", blobClient.Uri);

// Open the file and upload its data
FileStream uploadFileStream = File.OpenRead(localFilePath);
await blobClient.UploadAsync(uploadFileStream);
uploadFileStream.Close();

Note that any existing blob with the same name is overwritten.

Also, ensure you have a subfolder \data in the application bin folder. 

Create or copy a new CSV into the folder.

If you wish to upload the source file into a subfolder within the Azure blob container, then prefix the container with the folder name as shown:

string remotePath = "data";

// Get a reference to a blob
BlobClient blobClient = containerClient.GetBlobClient(remotePath + "/" + fileName);

Code to list blobs within the container is shown:

Console.WriteLine("Listing blobs...");

// List all blobs in the container
foreach (Azure.Storage.Blobs.Models.BlobItem blobItem in containerClient.GetBlobs())
{
   Console.WriteLine("\t" + blobItem.Name);
}

After running the above code, go back to the Azure portal and refresh the list of containers under the storage account. You should see an additional container displaying:

Drill into the container and you should see the CSV file created within the blob container:

(If you have uploaded into a subfolder within the blob container then the subfolder will be created:

and drilling into the folder will show the uploaded file:

Click to inspect the properties with options to download, inspect the file or obtain the URL:

The last check we do is to compare the uploaded file is identical to the source CSV file on our local machine.

Selecting the Edit option shows the contents of the uploaded CSV file:

On the source file, the contents are as shown:

To retrieve the blob directly, the URL is of the format:

https://[storage account name].blob.core.windows.net/[blob name]/[file name] 

The upload has been successful!

More details can be found on the Microsoft documentation as follows:

https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-dotnet

In a future post I will discuss how we can use Azure Functions to check our Azure container blob for CSV files and upload them into an Azure SQL database.

That’s all for today’s post.

I hope this has been useful and informative.

Andrew Halil

Andrew Halil is a blogger, author and software developer with expertise of many areas in the information technology industry including full-stack web and native cloud based development, test driven development and Devops.