How to Decide between using an Azure Service Bus or an Azure Storage Queue

Welcome to today’s post.

In today’s post I will be comparing the differences between two cloud-based messaging platforms, Azure Service Bus and Azure Storage Queue.

In previous posts I have given an example of how each messaging platform is implemented from a developer viewpoint.

I will first provide an overview and key features of each messaging platform.

Azure Storage Queues

Azure storage queues are part of an Azure storage account, allowing storage of large numbers of messages. The messages are accessible via HTTP requests. The total number of messages in the queues is limited by the available space within the storage account. Messages are guaranteed but the ordering is not always guaranteed. The predominant cost is determined by the total size of messages within the queues. Messages have redundant storage that is either backed up locally or replicated across zones or regions.

Azure Service Bus

Azure service bus is a messaging platform that provides transactional based message queues, including publishing and subscribing and fault-tolerant delivery of messages between clients to backend services. For the standard and higher plans, the messaging is transactionally scoped. Messages are guaranteed delivery and load balanced. Message recovery is supported with Geo-disaster recovery with the premium plan. In the standard plan the use of active and passive Geo-replication of queues or topics across two namespaces. 

Key Features of Azure Storage Queues

• Message retention can be indefinite.
• Message storage requires a storage account.
• Multiple queues per storage account.
• Accessible by HTTP or HTTPS.
• Maximum message size of 64kb.
• Messages can be in any format.
• Logging of queue transactions.
• Maximum queue size of 200TB limited to size of storage account.
• Only pay for Azure storage, not the number of operations.
• Message throughput at maximum 2,000 messages per queue.
• Delivery guarantee of At-Least-Once (received messages are redelivered in the event of a failure).

Key Features of Azure Service Bus

• Message sending and receival in guaranteed FIFO ordering.
• Publishing and subscribing messaging topics.
• Asynchronous messaging.
• Load-balanced messaging.
• Fault-tolerant messaging with transaction protocols.
• Decoupling between consumer and receiver.
• Structured and unstructured messaging formats.
• Message size maximum size of 256kb.
• Increased throughput with client-side batching.
• Maximum queue or topic size of 80GB.
• Only pay for number of operations.
• Message throughput at maximum 100 messages per transaction.
• Delivery guarantee of At-Most-Once (received messages are deleted even after a failure) or At-Least-Once (received messages are redelivered in the event of a failure).

I will first provide an overview of the respective differences between the two Azure messaging platforms, then provide situations where we can select one or the other.

Differences between Azure Service Bus and Azure Storage Queues

	Service Bus Queue	Storage Queue
Ordered delivery	Yes	No
Atomic transactions	Yes	No
Batched sending	Yes	No
Scheduled delivery	Yes	Yes
Duplicate message detection	Yes	No
Decoupling of components	Yes	Yes
Load balancing	Yes	No
Load levelling	Yes	Yes
Unlimited operations	No	Yes
Unlimited storage	No	Yes
Retention time	Default 14 days	Default 7 days

Deciding which messaging platform to use

Based on the above differences between the two messaging systems, there is no static method or formula to determine which service for your system.

A major determinant can be the cost and to decide this we ask the following questions:

1. How many operations per month will your systems use to access queue operations?
2. What is the size of each message structure (payload) you will be using?
3. What features do you wish to use or can do without?

• Transactions?
• Redundancy?
• Load-balancing?
• Guaranteed ordering?
• Batching messages from sender?
• Multiple topics and subscribers?
• Subscription filters?

Typical scenarios where service bus is recommended:

1. If your systems require the use of asynchronous transactional messaging, where for example a scoped transaction must be successful for the entire transaction to succeed.
2. In a trading platform where the ordering of messages must be respected and guaranteed to deliver.
3. Where a system such as a payment batch processing system submits multiple requests to the queue as a batch and expects the requests to be successfully delivered. 
4. In a trading application, where a request to sell an item is posted. Multiple buyers subscribe to receive item categories they wish to buy.
5. Your messages will be delivered at most once* to recipients.

*Note: With service bus queues, two modes of delivery are supported: At-Most-Once and At-Least-Once. For At-Most-Once delivery, after a message is received and consumed, it is deleted from the queue even if the transaction fails. For the At-Least-Once delivery, after a message is received, it is locked, processed, then unlocked. If the message cannot be processed during locking, it times out and is released back into the queue and other recipients will view and be able to receive the message. The processing of a previously failed message can be failed when duplicate detection is enabled. In this case the message is removed from the queue is an existing duplicate message exists. Duplication detection is available in Standard or Premium tiers.

Typical scenarios where storage queues are recommended:

1. A web site where users upload images that queued for routing to a moderator.
2. Your queue size will exceed more than 80GB.
3. Your messaging throughput will exceed 100 messages per second per queue.
4. Your messages will be delivered at least once* to recipients.

*Note: with storage queues, after a message is received and during the visibility timeout, if the message is not deleted before the timeout then other recipients will view and be able to receive the message. For this reason, the delivery guarantee is At-Least-Once.

From the above overview you should have a clearer idea on which service you wish to utilise for your Azure messaging. There is no one size fits all platform and each application you evaluate has different non-functional requirements that match up to the features each messaging service can offer you.

That’s all for today’s post.

I hope you found this post useful and informative.