Service Bus Queue

Experimental

This feature is experimental and still evolving — module inputs, outputs, and behavior may change without a deprecation period, and you may encounter rough edges. Review the module README and pin your module version carefully before relying on it in production.

This guide explains how to enable an app-scoped Azure Service Bus queue for your own internal producer/consumer messaging, without going through the Event Service.

🎯 Overview

Some workloads need a simple point-to-point queue purely internal to one app (e.g. a document-intake pipeline handing work between its own API and a background processor) rather than a pub/sub event that other apps subscribe to. The Forge platform provides:

• RBAC Authentication: Sender/Receiver access scoped to each queue itself, granted to your app's Entra identity (no connection strings, no SAS tokens)
• DLQ / poison-message configuration: configurable delivery-count threshold, message TTL, and dead-lettering, per queue
• Multiple queues: provision as many independent, named queues as your app needs (e.g. an intake queue and a separate retry/aux-workflow queue) — not limited to one queue per app
• Feature Flag: opt-in; a planned, equivalent Storage Account queue option is tracked separately (see Service Bus queue vs. Storage queue below)

When to choose this over the Event Service

• Use the Event Service (topics + subscriptions, see Event Service and Event Subscriptions) when other apps need to react to something your app did.
• Use this Service Bus queue feature when only your own app produces and consumes the messages — internal work handoff, not a public event contract.

Service Bus queue vs. Storage queue

Both are app-internal, single-producer/single-consumer options; the Service Bus queue is available today behind its own feature flag, while the equivalent Storage Account queue option is planned (tracked in #1165, not yet implemented). Pick per team based on throughput and ops needs once both are available:

	Service Bus queue	Storage Account queue (planned)
Message size	Up to 256 KB (Standard)	Up to 64 KB
Ordering / sessions	FIFO via sessions (not exposed by this module)	No ordering guarantee
DLQ	Built-in dead-letter sub-queue	Requires your own poison-queue pattern
Throughput	Higher, brokered messaging	Simpler, higher scale for basic work queues
Cost	Namespace-based (shared namespace, no extra cost per queue)	Per-storage-account transaction cost

If you're unsure, prefer the Service Bus queue — the built-in DLQ and poison-message handling need less custom code, and it's available now. See #1138 for the full spike write-up.

🔧 Infrastructure Setup (Terraform)

Prerequisites

enable_servicebus_queue provisions queues through the saif-resources modules/servicebus-queue submodule. Enabling the flag requires a saif-api-service release that references a published saif-resources version containing this module; older versions cannot resolve the submodule. See the module's generated Terraform reference for the complete input/output contract.

Enable the Service Bus Queue Feature Flag

Service Bus queue support is a feature flag on the saif-apiservice module, configured in the module call the templates generate for you — infra/api/app.generated.tf.

Add a feature_flags block to the existing module "saif-appservices" call:

feature_flags = {
  enable_servicebus_queue = true
  # Keyed by a stable queue identity name (e.g. "intake", "retries") — provision as many
  # independent queues as your app needs. Omit `servicebus_queues` entirely to get a single
  # "default" queue with all sub-settings at their defaults.
  servicebus_queues = {
    intake = {
      max_delivery_count                   = 10
      lock_duration                        = "PT30S"
      default_message_ttl                  = "P14D"
      dead_lettering_on_message_expiration = true
      requires_duplicate_detection         = false
    }
    retries = {
      max_delivery_count                      = 20
      dead_lettering_on_message_expiration    = true
      requires_duplicate_detection            = true
      duplicate_detection_history_time_window = "PT15M"
      max_size_in_megabytes                   = 2048
    }
  }
}

⚠️ enable_servicebus_queue and servicebus_queues are members of the feature_flags object, not top-level module arguments. If the module call already has a feature_flags object, merge these members into it.

Configuration Options (each entry in servicebus_queues, keyed by queue name):

Property	Type	Description	Default
max_delivery_count	number	Delivery attempts before a message is dead-lettered as poison	10
lock_duration	string (ISO 8601 duration)	How long a message is locked for a receiving consumer before it becomes available again	PT30S
default_message_ttl	string (ISO 8601 duration)	Default time-to-live for messages on the queue; expired messages are dead-lettered when dead_lettering_on_message_expiration is true, otherwise discarded	P14D
dead_lettering_on_message_expiration	bool	Move expired messages to the dead-letter queue instead of discarding	true
requires_duplicate_detection	bool	Detect and drop duplicate messages sent within duplicate_detection_history_time_window	false
duplicate_detection_history_time_window	string (ISO 8601 duration)	Duplicate-detection window; only applies when requires_duplicate_detection is true	PT10M
max_size_in_megabytes	number	Maximum queue size in megabytes	1024
dlq_reader_identities	map(string)	Identity key → Entra ID object ID granted read access to this queue's DLQ	{}
dlq_manager_identities	map(string)	Identity key → Entra ID object ID granted manage access to this queue's DLQ	{}

⚠️ Hardcode object IDs — don't resolve them with a data source. Same convention as blob storage's contributor_group_id: a data "azuread_group" lookup executes at plan time and fails the Organization Policy Check. Look the object ID up once and paste the literal GUID.

⚠️ requires_duplicate_detection cannot be changed after the queue is created. Azure Service Bus does not support toggling duplicate detection on an existing queue, so Terraform must destroy and recreate the queue to apply a change — discarding any live and dead-lettered messages still on it. Decide this setting up front. See Changing requires_duplicate_detection below if you do need to change it on a queue already in use.

⚠️ Queue keys (the servicebus_queues map keys) must be lowercase alphanumerics separated by single hyphens, such as orders or order-retries. Each key determines both the Azure queue name ({project_id}-{key}-queue) and a ServiceBusQueueName__{key} app setting, so it has to be safe in both namespaces at once. Azure queue names and .NET's IConfiguration lookups are both case-insensitive, so keys differing only by case (e.g. Orders and orders) would resolve to the same remote queue and the same configuration key even though Terraform treats them as distinct map entries. Underscores are rejected as well: App Service surfaces app settings as environment variables and .NET translates __ into the : configuration-path separator, so a key of a__b would emit ServiceBusQueueName__a__b and bind to ServiceBusQueueName:a:b, which the documented ServiceBusQueueName:{key} lookup could never resolve. The module rejects invalid keys with a variable validation error before planning.

Namespace

Queues are provisioned on the shared, per-environment/per-domain Service Bus namespace resolved from platform context. You don't configure a namespace at all — each environment has its own namespace, and the module reads it from context.org_services.

The namespace's fully-qualified host is published centrally to App Configuration as ConnectionStrings:messaging (see Generated Resources below), which AddAzureServiceBusClient(connectionName: "messaging") reads. messaging is the universal connection name used across all consumers of this module, so it isn't configurable.

Changing requires_duplicate_detection

Because requires_duplicate_detection forces queue recreation, do not just flip the value on an existing, in-use queue — that destroys the live queue (and its dead-letter sub-queue) along with any unprocessed messages. Instead, migrate onto a new queue:

1. Add a new entry to servicebus_queues under a new key (e.g. intake-v2) with the desired requires_duplicate_detection value — this provisions a second, independent queue alongside the existing one; it does not touch the original.
2. Update producers to send to the new queue's ServiceBusQueueName__{new_key} app setting.
3. Drain the old queue: let existing consumers finish processing whatever is already on it (and triage anything already in its dead-letter sub-queue) before retiring them.
4. Once the old queue is empty and no longer receiving traffic, remove its entry from servicebus_queues to delete it.

This avoids any window where in-flight or dead-lettered messages are silently discarded by a destroy/recreate of the original queue.

Generated Resources

When enable_servicebus_queue = true, the following resources are created for each key in servicebus_queues:

• 📬 Azure Service Bus Queue — named {project_id}-{key}-queue on the shared Service Bus namespace
• 🔐 RBAC Role Assignments, scoped to that queue only (not the namespace):
• Azure Service Bus Data Sender + Azure Service Bus Data Receiver → the app's Entra App Registration service principal (the app authenticates via EnvironmentCredential; the UAMI is not granted access)
• Optional Azure Service Bus Data Receiver / Azure Service Bus Data Owner → that queue's dlq_reader_identities / dlq_manager_identities
• ⚙️ App Setting: ServiceBusQueueName__{key} — the queue name your app code should send/receive on for that queue

ℹ️ The namespace's fully-qualified host is not emitted per-app. It's published centrally to shared App Configuration as ConnectionStrings:messaging (cloud-foundations#134) — bind with AddAzureServiceBusClient(connectionName: "messaging").

This requires SAIF.Platform.Azure's AddAzureDefaults() to select the ConnectionStrings:* prefix from Azure App Configuration. Upgrade SAIF.Platform.Azure to the release containing this default, or explicitly include ConnectionStrings:* in AzureConfigurationOptions.Prefixes, and call builder.AddAzureDefaults() before AddAzureServiceBusClient().

🖥️ Using the Queue in Your Service

Add the Aspire Service Bus client package to your app project (it's not part of the golden-path template's default package set):

dotnet add package Aspire.Azure.Messaging.ServiceBus

// Program.cs
builder.AddAzureServiceBusClient(connectionName: "messaging");

using Azure.Messaging.ServiceBus;
using System.Text.Json;

// App settings are named ServiceBusQueueName__{key} for each queue key in servicebus_queues
// (e.g. "ServiceBusQueueName__intake", "ServiceBusQueueName__retries"). .NET configuration
// normalizes the "__" App Service setting separator to ":", so look values up with IConfiguration
// using ":" — the "__" form only exists at the App Service / environment-variable layer.
var intakeQueueName = builder.Configuration["ServiceBusQueueName:intake"];

// ServiceBusSender is a thread-safe, long-lived client — register one per queue instead of
// creating a new sender on every request, and let it be disposed on application shutdown.
builder.Services.AddSingleton(sp =>
{
    var client = sp.GetRequiredService<ServiceBusClient>();
    var queueName = sp.GetRequiredService<IConfiguration>()["ServiceBusQueueName:intake"];
    return client.CreateSender(queueName);
});

app.MapPost("/api/documents", async (DocumentRequest request, ServiceBusSender sender) =>
{
    var message = new ServiceBusMessage(JsonSerializer.Serialize(request));
    await sender.SendMessageAsync(message);
    return Results.Accepted();
});

A background worker (hosted service or a second app in the same solution) reads from the same queue with client.CreateReceiver(queueName) or ServiceBusProcessor.

📊 Monitoring & Alerting

• Queue depth (ActiveMessages metric) — alert when messages back up, indicating a stalled or under-scaled consumer.
• Dead-letter message count (DeadletteredMessages metric) — alert on any non-zero value; poison messages need triage, not silent accumulation.
• Both metrics are available on the Service Bus namespace in Azure Monitor, scoped to the entity name (EntityName dimension = your queue name).

📋 Configuration Checklist

• Add enable_servicebus_queue = true to the feature_flags block in infra/api/app.generated.tf (merge into an existing object if one is already generated)
• Define one entry in servicebus_queues per queue your app needs, keyed by a stable name
• Decide max_delivery_count and whether duplicate detection is needed, per queue
• Apply Terraform configuration
• Add the Aspire.Azure.Messaging.ServiceBus package reference to your app project
• Add builder.AddAzureServiceBusClient("messaging") to Program.cs
• Read each queue name from its ServiceBusQueueName__{key} app setting — don't hardcode it
• Set up queue-depth and DLQ-count alerts, per queue

🔗 Related Resources

• Blob Storage — analogous feature-flag pattern for Blob Storage
• Event Service / Event Subscriptions — pub/sub for cross-app events
• Azure Service Bus Documentation
• Aspire Service Bus Integration