@message-queue-toolkit/sns
Version:
SNS adapter for message-queue-toolkit
1,207 lines (1,012 loc) • 35.1 kB
Markdown
# @message-queue-toolkit/sns
AWS SNS (Simple Notification Service) implementation for the message-queue-toolkit. Provides a robust, type-safe abstraction for publishing messages to SNS topics and consuming them via SQS queue subscriptions, with support for both standard and FIFO topics.
## Table of Contents
- [Installation](#installation)
- [Features](#features)
- [Core Concepts](#core-concepts)
- [Quick Start](#quick-start)
- [Standard Topic Publisher](#standard-topic-publisher)
- [SNS-to-SQS Consumer](#sns-to-sqs-consumer)
- [FIFO Topic Publisher](#fifo-topic-publisher)
- [FIFO Topic Consumer](#fifo-topic-consumer)
- [Configuration](#configuration)
- [Topic Creation](#topic-creation)
- [Topic Locator](#topic-locator)
- [Publisher Options](#publisher-options)
- [Consumer Options](#consumer-options)
- [SNS-Specific Features](#sns-specific-features)
- [Topic Subscriptions](#topic-subscriptions)
- [Fan-Out Pattern](#fan-out-pattern)
- [Message Filtering](#message-filtering)
- [Cross-Account Publishing](#cross-account-publishing)
- [Advanced Features](#advanced-features)
- [FIFO Topics](#fifo-topics)
- [FIFO Topic Requirements](#fifo-topic-requirements)
- [Message Groups](#message-groups)
- [FIFO-Specific Configuration](#fifo-specific-configuration)
- [Testing](#testing)
- [API Reference](#api-reference)
## Installation
```bash
npm install @message-queue-toolkit/sns @message-queue-toolkit/sqs @message-queue-toolkit/core
```
**Peer Dependencies:**
- `@aws-sdk/client-sns` - AWS SDK for SNS
- `@aws-sdk/client-sqs` - AWS SDK for SQS (required for consumers)
- `@aws-sdk/client-sts` - AWS SDK for STS (for ARN resolution)
- `zod` - Schema validation
## Features
- ✅ **Type-safe message handling** with Zod schema validation
- ✅ **Standard and FIFO topic support**
- ✅ **Automatic topic and subscription creation**
- ✅ **Fan-out pattern** - publish once, consume from multiple queues
- ✅ **Message filtering** - subscribe to specific message types
- ✅ **Message deduplication** (publisher and consumer level)
- ✅ **Payload offloading** for large messages (S3 integration)
- ✅ **Automatic retry logic** with exponential backoff (inherited from SQS consumer)
- ✅ **Dead Letter Queue (DLQ)** support
- ✅ **Handler spies** for testing
- ✅ **Pre-handlers and barriers** for complex message processing
- ✅ **Cross-account and cross-region publishing**
## Core Concepts
### Publishers
SNS publishers send messages to **topics** (not queues). A topic is a logical access point that acts as a communication channel. Publishers are responsible for:
- Message validation against Zod schemas
- Automatic serialization
- Optional deduplication (preventing duplicate sends)
- Optional payload offloading (for messages > 256KB)
- FIFO-specific concerns (MessageGroupId, MessageDeduplicationId)
### Consumers
SNS consumers subscribe to topics via **SQS queues** (SNS-to-SQS pattern). This provides:
- **Decoupling** - topics don't know about subscribers
- **Fan-out** - one message published to multiple queues
- **Persistence** - messages queued until processed
- **Retry logic** - inherited from SQS consumer capabilities
- **Message filtering** - only receive relevant messages
Consumers are actually `AbstractSnsSqsConsumer` which extends `AbstractSqsConsumer` from the SQS package, inheriting all SQS consumer capabilities.
### Message Flow
```text
Publisher → SNS Topic → [Subscriptions] → SQS Queues → Consumers
↓
(optional filtering)
```
**Key Difference from SQS:**
- **SQS**: Direct queue-to-queue communication (1:1)
- **SNS**: Pub/Sub pattern with fan-out (1:N)
### Message Schemas
Messages use the same schema requirements as SQS. Each message must have:
- A unique message type field (discriminator for routing) - configurable via `messageTypeField` (required)
- A message ID field (for tracking and deduplication) - configurable via `messageIdField` (default: `'id'`)
- A timestamp field (added automatically if missing) - configurable via `messageTimestampField` (default: `'timestamp'`)
See the [SQS README - Message Schemas section](../sqs/README.md#message-schemas) for full details.
## Quick Start
### Standard Topic Publisher
```typescript
import { AbstractSnsPublisher } from '@message-queue-toolkit/sns'
import { SNSClient } from '@aws-sdk/client-sns'
import { STSClient } from '@aws-sdk/client-sts'
import z from 'zod'
// Define your message schemas
const UserCreatedSchema = z.object({
id: z.string(),
messageType: z.literal('user.created'),
userId: z.string(),
email: z.string().email(),
timestamp: z.string().optional(),
})
const UserUpdatedSchema = z.object({
id: z.string(),
messageType: z.literal('user.updated'),
userId: z.string(),
changes: z.record(z.unknown()),
timestamp: z.string().optional(),
})
type UserCreated = z.infer<typeof UserCreatedSchema>
type UserUpdated = z.infer<typeof UserUpdatedSchema>
type SupportedMessages = UserCreated | UserUpdated
// Create your publisher class
class UserEventsPublisher extends AbstractSnsPublisher<SupportedMessages> {
constructor(snsClient: SNSClient, stsClient: STSClient) {
super(
{
snsClient,
stsClient,
logger: console,
errorReporter: { report: (error) => console.error(error) },
},
{
messageSchemas: [UserCreatedSchema, UserUpdatedSchema],
messageTypeField: 'messageType',
creationConfig: {
topic: {
Name: 'user-events-topic',
},
},
deletionConfig: {
deleteIfExists: false,
},
}
)
}
}
// Use the publisher
const snsClient = new SNSClient({ region: 'us-east-1' })
const stsClient = new STSClient({ region: 'us-east-1' })
const publisher = new UserEventsPublisher(snsClient, stsClient)
await publisher.init()
// Publish to the topic - all subscribers will receive this message
await publisher.publish({
id: '123',
messageType: 'user.created',
userId: 'user-456',
email: 'user@example.com',
})
await publisher.close()
```
### SNS-to-SQS Consumer
Consumers subscribe to SNS topics via SQS queues:
```typescript
import { AbstractSnsSqsConsumer } from '@message-queue-toolkit/sns'
import { MessageHandlerConfigBuilder } from '@message-queue-toolkit/core'
import type { Either } from '@lokalise/node-core'
type ExecutionContext = {
userService: UserService
}
class UserEventsConsumer extends AbstractSnsSqsConsumer<
SupportedMessages,
ExecutionContext
> {
constructor(
snsClient: SNSClient,
sqsClient: SQSClient,
stsClient: STSClient,
userService: UserService
) {
super(
{
snsClient,
sqsClient,
stsClient,
logger: console,
errorReporter: { report: (error) => console.error(error) },
consumerErrorResolver: {
resolveError: () => ({ resolve: 'retryLater' as const }),
},
transactionObservabilityManager: {
start: () => {},
stop: () => {},
},
},
{
messageTypeField: 'messageType',
handlers: new MessageHandlerConfigBuilder<SupportedMessages, ExecutionContext>()
.addConfig(
UserCreatedSchema,
async (message, context): Promise<Either<'retryLater', 'success'>> => {
await context.userService.createUser(message.userId, message.email)
return { result: 'success' }
}
)
.addConfig(
UserUpdatedSchema,
async (message, context): Promise<Either<'retryLater', 'success'>> => {
await context.userService.updateUser(message.userId, message.changes)
return { result: 'success' }
}
)
.build(),
// Queue configuration (SQS queue that subscribes to SNS topic)
creationConfig: {
queue: {
QueueName: 'user-events-consumer-queue',
},
topic: {
Name: 'user-events-topic', // Must match publisher's topic name
},
},
// Subscription configuration
subscriptionConfig: {
updateAttributesIfExists: false,
},
deletionConfig: {
deleteIfExists: false,
},
},
{ userService } // Execution context
)
}
}
// Use the consumer
const consumer = new UserEventsConsumer(snsClient, sqsClient, stsClient, userService)
await consumer.start() // Creates queue, subscribes to topic, starts consuming
// Later, to stop
await consumer.close()
```
**What happens during `start()`:**
1. Creates SNS topic (if using `creationConfig`)
2. Creates SQS queue (if using `creationConfig`)
3. Subscribes queue to topic
4. Configures queue permissions to allow SNS to publish
5. Starts consuming messages from the queue
### FIFO Topic Publisher
FIFO topics guarantee message ordering and exactly-once delivery:
```typescript
class UserEventsFifoPublisher extends AbstractSnsPublisher<SupportedMessages> {
constructor(snsClient: SNSClient, stsClient: STSClient) {
super(
{
snsClient,
stsClient,
logger: console,
errorReporter: { report: (error) => console.error(error) },
},
{
messageSchemas: [UserCreatedSchema, UserUpdatedSchema],
messageTypeField: 'messageType',
fifoTopic: true, // Enable FIFO mode
// Option 1: Use a field from the message as MessageGroupId
messageGroupIdField: 'userId',
// Option 2: Use a default MessageGroupId for all messages
// defaultMessageGroupId: 'user-events',
creationConfig: {
topic: {
Name: 'user-events-topic.fifo', // Must end with .fifo
Attributes: {
FifoTopic: 'true',
ContentBasedDeduplication: 'false', // or 'true' for automatic deduplication
},
},
},
}
)
}
}
// Publishing to FIFO topic
const fifoPublisher = new UserEventsFifoPublisher(snsClient, stsClient)
await fifoPublisher.init()
// Messages with the same userId will be processed in order
await fifoPublisher.publish({
id: '123',
messageType: 'user.created',
userId: 'user-456', // Used as MessageGroupId
email: 'user@example.com',
})
await fifoPublisher.publish({
id: '124',
messageType: 'user.updated',
userId: 'user-456', // Same group - processed after the first message
changes: { name: 'John Doe' },
})
// You can also explicitly provide MessageGroupId
await fifoPublisher.publish(
{
id: '125',
messageType: 'user.created',
userId: 'user-789',
email: 'other@example.com',
},
{
MessageGroupId: 'custom-group-id',
MessageDeduplicationId: 'unique-dedup-id', // Optional
}
)
```
### FIFO Topic Consumer
```typescript
class UserEventsFifoConsumer extends AbstractSnsSqsConsumer<
SupportedMessages,
ExecutionContext
> {
constructor(
snsClient: SNSClient,
sqsClient: SQSClient,
stsClient: STSClient,
userService: UserService
) {
super(
{
snsClient,
sqsClient,
stsClient,
logger: console,
errorReporter: { report: (error) => console.error(error) },
consumerErrorResolver: {
resolveError: () => ({ resolve: 'retryLater' as const }),
},
transactionObservabilityManager: {
start: () => {},
stop: () => {},
},
},
{
fifoQueue: true, // Enable FIFO mode for SQS queue
messageTypeField: 'messageType',
handlers: new MessageHandlerConfigBuilder<SupportedMessages, ExecutionContext>()
.addConfig(UserCreatedSchema, handleUserCreated)
.addConfig(UserUpdatedSchema, handleUserUpdated)
.build(),
creationConfig: {
queue: {
QueueName: 'user-events-consumer-queue.fifo', // Must end with .fifo
Attributes: {
FifoQueue: 'true',
ContentBasedDeduplication: 'false',
},
},
topic: {
Name: 'user-events-topic.fifo', // Must match publisher's FIFO topic
Attributes: {
FifoTopic: 'true',
ContentBasedDeduplication: 'false',
},
},
},
subscriptionConfig: {
updateAttributesIfExists: false,
},
// Optional: Configure concurrent consumers for parallel processing of different groups
concurrentConsumersAmount: 3, // Process 3 different message groups in parallel
},
{ userService }
)
}
}
```
## Configuration
### Topic Creation
When using `creationConfig`, the topic will be created automatically if it doesn't exist:
```typescript
{
creationConfig: {
topic: {
Name: 'my-topic',
Attributes: {
// Standard Topic attributes
DisplayName: 'My Notification Topic',
// FIFO Topic attributes (only for .fifo topics)
FifoTopic: 'true', // Must be 'true' for FIFO topics
ContentBasedDeduplication: 'false', // Automatic deduplication based on message body
// Encryption
KmsMasterKeyId: 'alias/aws/sns', // KMS key for encryption
// Delivery policy
DeliveryPolicy: JSON.stringify({
healthyRetryPolicy: {
minDelayTarget: 20,
maxDelayTarget: 20,
numRetries: 3,
numMaxDelayRetries: 0,
numNoDelayRetries: 0,
numMinDelayRetries: 0,
backoffFunction: 'linear'
}
}),
},
Tags: [
{ Key: 'Environment', Value: 'production' },
{ Key: 'Team', Value: 'backend' },
],
},
queue: {
QueueName: 'my-consumer-queue',
// See SQS README for full queue configuration options
},
updateAttributesIfExists: true, // Update attributes if topic/queue exists
forceTagUpdate: false, // Force tag update even if unchanged
// Queue permissions for SNS publishing (automatically configured)
queueUrlsWithSubscribePermissionsPrefix: 'https://sqs.us-east-1.amazonaws.com/123456789012/',
},
}
```
### Topic Locator
When using `locatorConfig`, you connect to an existing topic without creating it:
```typescript
{
locatorConfig: {
// Option 1: By topic ARN
topicArn: 'arn:aws:sns:us-east-1:123456789012:my-topic',
// Option 2: By topic name (ARN will be resolved using STS)
// topicName: 'my-topic',
// Optional: Existing queue URL or name
queueUrl: 'https://sqs.us-east-1.amazonaws.com/123456789012/my-queue',
// or
// queueName: 'my-queue',
// Optional: Existing subscription ARN
subscriptionArn: 'arn:aws:sns:us-east-1:123456789012:my-topic:uuid',
},
}
```
### Publisher Options
```typescript
{
// Required - Message Schema Configuration
messageSchemas: [Schema1, Schema2], // Array of Zod schemas
messageTypeField: 'messageType', // Field containing message type discriminator
// Topic Configuration (one of these required)
creationConfig: { /* ... */ }, // Create topic if doesn't exist
locatorConfig: { /* ... */ }, // Use existing topic
// Optional - FIFO Configuration
fifoTopic: false, // Set to true for FIFO topics
messageGroupIdField: 'userId', // Field to use as MessageGroupId
defaultMessageGroupId: 'default', // Default MessageGroupId if field not present
// Optional - Message Field Configuration (same as SQS)
messageIdField: 'id', // Default: 'id'
messageTimestampField: 'timestamp', // Default: 'timestamp'
messageDeduplicationIdField: 'deduplicationId', // Default: 'deduplicationId'
messageDeduplicationOptionsField: 'deduplicationOptions', // Default: 'deduplicationOptions'
// Optional - Features
logMessages: false, // Log all published messages
handlerSpy: true, // Enable handler spy for testing
// Optional - Deduplication (same as SQS)
enablePublisherDeduplication: false,
messageDeduplicationConfig: { /* ... */ },
// Optional - Payload Offloading (same as SQS)
payloadStoreConfig: { /* ... */ },
// Optional - Deletion
deletionConfig: { /* ... */ },
}
```
See the [SQS README - Publisher Options section](../sqs/README.md#publisher-options) for full details on shared options.
### Consumer Options
SNS consumers use the same options as SQS consumers, plus SNS-specific subscription configuration:
```typescript
{
// All SQS consumer options are supported
// See SQS README for full consumer options
// Required - Message Handling Configuration
handlers: MessageHandlerConfigBuilder.build(),
messageTypeField: 'messageType',
// Topic & Queue Configuration
creationConfig: {
topic: { /* SNS topic config */ },
queue: { /* SQS queue config */ },
},
// or
locatorConfig: {
topicArn: 'arn:aws:sns:...',
queueUrl: 'https://sqs...',
subscriptionArn: 'arn:aws:sns:...',
},
// SNS-Specific - Subscription Configuration
subscriptionConfig: {
updateAttributesIfExists: false, // Update subscription attributes if exists
// Optional: Message filtering
filterPolicy: {
messageType: ['user.created', 'user.updated'], // Only receive these types
},
// Optional: Raw message delivery (disable SNS envelope)
rawMessageDelivery: false,
// Optional: Redrive policy (DLQ for undeliverable messages)
redrivePolicy: {
deadLetterTargetArn: 'arn:aws:sqs:us-east-1:123456789012:my-dlq',
},
},
// Optional - FIFO Configuration
fifoQueue: false,
// Optional - Other options inherited from SQS
concurrentConsumersAmount: 1,
maxRetryDuration: 345600, // 4 days
deadLetterQueue: { /* ... */ },
consumerOverrides: { /* ... */ },
// ... see SQS README for full list
}
```
See the [SQS README - Consumer Options section](../sqs/README.md#consumer-options) for full details on shared options.
## SNS-Specific Features
### Topic Subscriptions
SNS topics can have multiple subscribers. Each subscriber receives a copy of every message published to the topic:
```typescript
// Publisher publishes to one topic
class NotificationPublisher extends AbstractSnsPublisher<NotificationMessage> {
constructor(snsClient: SNSClient, stsClient: STSClient) {
super(dependencies, {
messageSchemas: [NotificationSchema],
creationConfig: {
topic: { Name: 'notifications' },
},
})
}
}
// Multiple consumers subscribe to the same topic
class EmailConsumer extends AbstractSnsSqsConsumer<NotificationMessage, EmailContext> {
constructor(deps) {
super(deps, {
handlers: buildHandlers(sendEmail),
creationConfig: {
queue: { QueueName: 'email-notifications' },
topic: { Name: 'notifications' }, // Same topic
},
})
}
}
class SMSConsumer extends AbstractSnsSqsConsumer<NotificationMessage, SMSContext> {
constructor(deps) {
super(deps, {
handlers: buildHandlers(sendSMS),
creationConfig: {
queue: { QueueName: 'sms-notifications' },
topic: { Name: 'notifications' }, // Same topic
},
})
}
}
class PushConsumer extends AbstractSnsSqsConsumer<NotificationMessage, PushContext> {
constructor(deps) {
super(deps, {
handlers: buildHandlers(sendPush),
creationConfig: {
queue: { QueueName: 'push-notifications' },
topic: { Name: 'notifications' }, // Same topic
},
})
}
}
// One publish reaches all three consumers
await publisher.publish({
id: '123',
messageType: 'notification.created',
userId: 'user-456',
message: 'Your order has shipped!',
})
// → Email sent
// → SMS sent
// → Push notification sent
```
### Fan-Out Pattern
The fan-out pattern enables broadcasting messages to multiple independent processing pipelines:
```typescript
// Single event publisher
class OrderEventPublisher extends AbstractSnsPublisher<OrderEvent> {
constructor(snsClient: SNSClient, stsClient: STSClient) {
super(dependencies, {
messageSchemas: [OrderCreatedSchema, OrderUpdatedSchema],
creationConfig: {
topic: { Name: 'order-events' },
},
})
}
}
// Different services consume the same events independently
// Inventory service - updates stock levels
class InventoryConsumer extends AbstractSnsSqsConsumer<OrderEvent, InventoryContext> {
constructor(deps) {
super(deps, {
handlers: buildInventoryHandlers(),
creationConfig: {
queue: { QueueName: 'inventory-order-events' },
topic: { Name: 'order-events' },
},
})
}
}
// Analytics service - tracks order metrics
class AnalyticsConsumer extends AbstractSnsSqsConsumer<OrderEvent, AnalyticsContext> {
constructor(deps) {
super(deps, {
handlers: buildAnalyticsHandlers(),
creationConfig: {
queue: { QueueName: 'analytics-order-events' },
topic: { Name: 'order-events' },
},
})
}
}
// Shipping service - prepares shipments
class ShippingConsumer extends AbstractSnsSqsConsumer<OrderEvent, ShippingContext> {
constructor(deps) {
super(deps, {
handlers: buildShippingHandlers(),
creationConfig: {
queue: { QueueName: 'shipping-order-events' },
topic: { Name: 'order-events' },
},
})
}
}
// Benefits:
// - Each service processes independently
// - Failure in one doesn't affect others
// - Easy to add new consumers without changing publisher
// - Each consumer can have its own retry/DLQ configuration
```
### Message Filtering
Subscribers can filter messages to receive only specific types:
```typescript
class UserCreatedConsumer extends AbstractSnsSqsConsumer<UserEvent, UserContext> {
constructor(deps) {
super(deps, {
handlers: buildHandlers(),
creationConfig: {
queue: { QueueName: 'user-created-processor' },
topic: { Name: 'user-events' },
},
subscriptionConfig: {
// Only receive user.created events
filterPolicy: {
messageType: ['user.created'],
},
},
})
}
}
class UserModificationConsumer extends AbstractSnsSqsConsumer<UserEvent, UserContext> {
constructor(deps) {
super(deps, {
handlers: buildHandlers(),
creationConfig: {
queue: { QueueName: 'user-modification-processor' },
topic: { Name: 'user-events' },
},
subscriptionConfig: {
// Only receive update and delete events
filterPolicy: {
messageType: ['user.updated', 'user.deleted'],
},
},
})
}
}
// Advanced filtering with message attributes
class HighPriorityConsumer extends AbstractSnsSqsConsumer<OrderEvent, OrderContext> {
constructor(deps) {
super(deps, {
handlers: buildHandlers(),
creationConfig: {
queue: { QueueName: 'high-priority-orders' },
topic: { Name: 'order-events' },
},
subscriptionConfig: {
filterPolicy: {
messageType: ['order.created'],
priority: ['high', 'critical'], // Filters on message attribute
},
},
})
}
}
```
**Benefits:**
- Reduces unnecessary message processing
- Lowers SQS costs (fewer messages received)
- Filtering happens at SNS level (before queuing)
- Each subscriber can have different filters
### Cross-Account Publishing
SNS supports publishing from one AWS account to topics in another:
```typescript
// Account A - Publisher
class CrossAccountPublisher extends AbstractSnsPublisher<Message> {
constructor(snsClient: SNSClient, stsClient: STSClient) {
super(dependencies, {
messageSchemas: [MessageSchema],
locatorConfig: {
// Topic in Account B
topicArn: 'arn:aws:sns:us-east-1:222222222222:shared-topic',
},
})
}
}
// Account B - Consumer (topic owner)
// Topic policy must allow Account A to publish:
{
creationConfig: {
topic: {
Name: 'shared-topic',
Attributes: {
Policy: JSON.stringify({
Version: '2012-10-17',
Statement: [
{
Effect: 'Allow',
Principal: {
AWS: 'arn:aws:iam::111111111111:root', // Account A
},
Action: 'SNS:Publish',
Resource: 'arn:aws:sns:us-east-1:222222222222:shared-topic',
},
],
}),
},
},
},
}
```
## Advanced Features
SNS consumers inherit all advanced features from SQS consumers. See the SQS README for detailed documentation on:
- **[Custom Message Field Names](../sqs/README.md#custom-message-field-names)** - Adapt to existing schemas
- **[Dead Letter Queue (DLQ)](../sqs/README.md#dead-letter-queue-dlq)** - Handle permanently failing messages
- **[Message Retry Logic](../sqs/README.md#message-retry-logic)** - Exponential backoff and retry limits
- **[Message Deduplication](../sqs/README.md#message-deduplication)** - Publisher and consumer-level deduplication
- **[Payload Offloading](../sqs/README.md#payload-offloading)** - S3 storage for large messages
- **[Message Handlers](../sqs/README.md#message-handlers)** - Type-safe handler configuration
- **[Pre-handlers and Barriers](../sqs/README.md#pre-handlers-and-barriers)** - Middleware and message dependencies
- **[Handler Spies](../sqs/README.md#handler-spies)** - Testing async message flows
All these features work identically for SNS consumers since they extend the SQS consumer implementation.
## FIFO Topics
FIFO (First-In-First-Out) topics provide message ordering and exactly-once delivery, similar to FIFO queues.
### FIFO Topic Requirements
1. **Topic name must end with `.fifo`**
```typescript
Name: 'my-topic.fifo' // ✅ Valid
Name: 'my-topic' // ❌ Invalid for FIFO
```
2. **FifoTopic attribute must be 'true'**
```typescript
Attributes: {
FifoTopic: 'true',
}
```
3. **Subscribed queues must also be FIFO**
```typescript
creationConfig: {
topic: {
Name: 'events.fifo',
Attributes: { FifoTopic: 'true' },
},
queue: {
QueueName: 'consumer.fifo', // Must be FIFO
Attributes: { FifoQueue: 'true' },
},
}
```
4. **MessageGroupId required for all messages**
```typescript
// Option 1: From message field
messageGroupIdField: 'userId'
// Option 2: Default value
defaultMessageGroupId: 'default-group'
// Option 3: Explicit in publish call
await publisher.publish(message, {
MessageGroupId: 'custom-group',
})
```
5. **DLQ must also be FIFO**
```typescript
deadLetterQueue: {
creationConfig: {
queue: {
QueueName: 'my-dlq.fifo', // Must be FIFO
Attributes: { FifoQueue: 'true' },
},
},
}
```
### Message Groups
Message groups work the same way as in FIFO queues. See the [SQS README - Message Groups section](../sqs/README.md#message-groups) for detailed information on:
- Group assignment and parallelism
- Best practices for group design
- Balancing group sizes
- Sizing concurrent consumers
**SNS FIFO Fan-Out Example:**
```typescript
// FIFO publisher publishes to FIFO topic
const fifoPublisher = new OrderEventsFifoPublisher(snsClient, stsClient)
await fifoPublisher.publish({
id: '123',
messageType: 'order.created',
customerId: 'customer-A',
orderId: 'order-1',
}, {
MessageGroupId: 'customer-A', // All messages for customer-A ordered
})
// Multiple FIFO consumers subscribe to the same FIFO topic
const inventoryConsumer = new InventoryFifoConsumer(deps)
const analyticsConsumer = new AnalyticsFifoConsumer(deps)
const shippingConsumer = new ShippingFifoConsumer(deps)
// Each consumer receives messages in order within each group
// Different consumers can process different groups in parallel
```
### FIFO-Specific Configuration
```typescript
// Publisher
{
fifoTopic: true,
// Choose one or more:
messageGroupIdField: 'userId', // Use field from message
defaultMessageGroupId: 'default', // Fallback value
// or provide in publish call
}
// Consumer
{
fifoQueue: true,
concurrentConsumersAmount: 3, // Process 3 groups in parallel
// Note: Retry behavior inherits from SQS FIFO queues
// - No DelaySeconds support (AWS limitation)
// - Messages retry immediately
// - Order is preserved
}
// Topic Configuration
{
creationConfig: {
topic: {
Name: 'my-topic.fifo',
Attributes: {
FifoTopic: 'true',
// Optional: Automatic deduplication based on message body
ContentBasedDeduplication: 'false', // or 'true'
},
},
},
}
// Queue Configuration
{
creationConfig: {
queue: {
QueueName: 'my-queue.fifo',
Attributes: {
FifoQueue: 'true',
// Optional: Queue-level deduplication settings
ContentBasedDeduplication: 'false',
DeduplicationScope: 'queue', // or 'messageGroup'
FifoThroughputLimit: 'perQueue', // or 'perMessageGroupId'
},
},
},
}
```
**FIFO Limitations:**
- **Throughput**: 3,000 messages/second per topic (or 300/second per message group)
- **No delays**: FIFO queues don't support `DelaySeconds`
- **Strict ordering**: Within a message group, messages are delivered in exact order
- **FIFO-to-FIFO only**: FIFO topics can only fan out to FIFO queues
See the [SQS README - FIFO Queues section](../sqs/README.md#fifo-queues) for comprehensive FIFO documentation.
## Testing
SNS testing works the same as SQS testing. Handler spies enable testing of async pub/sub flows:
```typescript
import { describe, it, expect, beforeEach, afterEach } from 'vitest'
describe('SNS Publisher and Consumer', () => {
let publisher: UserEventsPublisher
let consumer: UserEventsConsumer
beforeEach(async () => {
publisher = new UserEventsPublisher(snsClient, stsClient, { handlerSpy: true })
consumer = new UserEventsConsumer(snsClient, sqsClient, stsClient, userService, {
handlerSpy: true
})
await publisher.init()
await consumer.start()
})
afterEach(async () => {
await consumer.close()
await publisher.close()
})
it('publishes to topic and consumes from subscribed queue', async () => {
// Publish to SNS topic
await publisher.publish({
id: '123',
messageType: 'user.created',
userId: 'user-456',
email: 'test@example.com',
})
// Wait for publisher spy
await publisher.handlerSpy.waitForMessageWithId('123', 'published')
// Wait for consumer spy
const consumedMessage = await consumer.handlerSpy.waitForMessageWithId('123', 'consumed')
expect(consumedMessage.userId).toBe('user-456')
expect(userService.createUser).toHaveBeenCalledWith('user-456', 'test@example.com')
})
})
```
See the [SQS README - Testing section](../sqs/README.md#testing) for comprehensive testing documentation including:
- Integration tests with LocalStack
- Unit tests with handler spies
- Testing indirect message publishing
- Complex workflow testing
## API Reference
### AbstractSnsPublisher
```typescript
class AbstractSnsPublisher<MessagePayloadType extends object> {
constructor(
dependencies: SNSDependencies,
options: SNSPublisherOptions<MessagePayloadType>
)
async init(): Promise<void>
async close(): Promise<void>
async publish(
message: MessagePayloadType,
options?: SNSMessageOptions
): Promise<void>
readonly handlerSpy: HandlerSpy<MessagePayloadType>
readonly topicArn: string
}
```
### AbstractSnsSqsConsumer
```typescript
class AbstractSnsSqsConsumer<
MessagePayloadType extends object,
ExecutionContext,
PrehandlerOutput = undefined
> extends AbstractSqsConsumer<...> {
constructor(
dependencies: SNSSQSConsumerDependencies,
options: SNSSQSConsumerOptions<MessagePayloadType, ExecutionContext, PrehandlerOutput>,
executionContext: ExecutionContext
)
async init(): Promise<void>
async start(): Promise<void>
async close(abort?: boolean): Promise<void>
readonly handlerSpy: HandlerSpy<MessagePayloadType>
readonly topicArn: string
readonly subscriptionArn: string
readonly queueUrl: string
readonly queueName: string
}
```
### Types
```typescript
// Message options for publishing
type SNSMessageOptions = {
MessageGroupId?: string // Required for FIFO topics
MessageDeduplicationId?: string // Optional for FIFO topics
}
// Dependencies
type SNSDependencies = {
snsClient: SNSClient
stsClient: STSClient
logger: Logger
errorReporter: ErrorReporter
messageMetricsManager?: MessageMetricsManager
}
// Consumer dependencies (extends SNSDependencies and SQSDependencies)
type SNSSQSConsumerDependencies = SNSDependencies & SQSDependencies & {
consumerErrorResolver: ErrorResolver
transactionObservabilityManager: TransactionObservabilityManager
}
// Subscription options
type SNSSubscriptionOptions = {
updateAttributesIfExists?: boolean
filterPolicy?: Record<string, string[]>
rawMessageDelivery?: boolean
redrivePolicy?: {
deadLetterTargetArn: string
}
}
```
### Utility Functions
```typescript
// Topic validation
function isFifoTopicName(topicName: string): boolean
function validateFifoTopicName(topicName: string, isFifoTopic: boolean): void
// Topic operations
async function assertTopic(
snsClient: SNSClient,
stsClient: STSClient,
topicOptions: CreateTopicCommandInput,
extraParams?: ExtraSNSCreationParams
): Promise<string> // Returns topicArn
async function deleteTopic(
snsClient: SNSClient,
stsClient: STSClient,
topicName: string
): Promise<void>
async function getTopicAttributes(
snsClient: SNSClient,
topicArn: string
): Promise<Either<'not_found', { attributes?: Record<string, string> }>>
// Subscription operations
async function subscribeToTopic(
snsClient: SNSClient,
topicArn: string,
queueArn: string,
options?: SNSSubscriptionOptions
): Promise<string> // Returns subscriptionArn
async function findSubscriptionByTopicAndQueue(
snsClient: SNSClient,
topicArn: string,
queueArn: string
): Promise<Subscription | undefined>
// Message reading
function deserializeSNSMessage(
message: SQSMessage
): Either<MessageInvalidFormatError, SNSMessageBody>
// Message size calculation (same as SQS)
function calculateOutgoingMessageSize(message: unknown): number
```
## License
MIT
## Contributing
Contributions are welcome! Please see the main repository for guidelines.
## Links
- [Main Repository](https://github.com/kibertoad/message-queue-toolkit)
- [Core Package](https://www.npmjs.com/package/@message-queue-toolkit/core)
- [SQS Package](https://www.npmjs.com/package/@message-queue-toolkit/sqs) - SNS consumers extend SQS consumers
- [AWS SNS Documentation](https://docs.aws.amazon.com/sns/)
- [FIFO Topic Documentation](https://docs.aws.amazon.com/sns/latest/dg/sns-fifo-topics.html)
- [SNS Message Filtering](https://docs.aws.amazon.com/sns/latest/dg/sns-message-filtering.html)