smart-saga-pattern/README.md

A library implementing the Saga pattern for microservice architecture

# Smart Saga Pattern

A TypeScript library implementing the Saga pattern for microservice architecture.

## Overview

The Saga pattern is a design pattern used to manage distributed transactions in a microservice architecture. It helps maintain data consistency across multiple services without using distributed transactions.

This library provides:

- Support for both Orchestration and Choreography approaches
- Type-safe API with TypeScript
- Pluggable message broker adapters (Kafka, RabbitMQ, Redis)
- State management for Saga persistence
- Error handling and retry mechanisms
- Logging utilities
- Dependency injection with decorators

## Installation

```bash
npm install smart-saga-pattern
```

## Usage

### Structured Context Approach

The library supports a structured context format that organizes data by step:

```typescript
{
  global: { /* global data available to all steps */ },
  steps: {
    "Step Name": {
      input: { /* input data for the step */ },
      output: { /* output data from the step */ },
      compensation: { /* data for compensation */ }
    }
  }
}
```

### Orchestration Approach

#### Standard Approach

```typescript
// Define a Saga definition
const sagaDefinition = SagaDefinition.builder<OrderContext>()
  .step(
    'Create Order',
    async (context) => {
      // Transaction logic
      const orderId = 'order-123';
      
      // Initialize input for the next step
      if (!context.steps.ProcessPayment) {
        context.steps.ProcessPayment = {
          input: {
            orderId,
            customerId: context.global.customerId,
            amount: 100
          },
          output: {},
          compensation: {}
        };
      }
      
      return { orderId };
    },
    async (context, transactionOutput) => {
      // Compensation logic
    }
  )
  .step(
    'Process Payment',
    async (context) => {
      // Transaction logic
      return { paymentId: 'payment-456' };
    },
    async (context, transactionOutput) => {
      // Compensation logic
    }
  )
  .build();

// Create a Saga orchestrator
const orchestrator = new StructuredSagaOrchestrator(sagaDefinition);

// Execute the Saga
const result = await orchestrator.execute({
  global: {
    customerId: 'customer-789'
  },
  steps: {}
});
```

#### Dependency Injection Approach

```typescript
// Define Saga steps using decorators
@SagaStep({ order: 1, description: 'Create order in the system' })
class CreateOrder {
  @Invoke({ description: 'Process order creation' })
  public async processOrder(context: OrderContext, metadata: TransactionMetadata) {
    // Transaction logic
    const orderId = `order-${Date.now()}`;
    
    // Initialize input for the next step
    if (!context.steps.ProcessPayment) {
      context.steps.ProcessPayment = {
        input: {
          orderId,
          customerId: context.global.customerId,
          amount: 100
        },
        output: {},
        compensation: {}
      };
    }
    
    return { orderId };
  }
  
  @Compensation({ description: 'Cancel order' })
  public async cancelOrder(context: OrderContext, transactionOutput: any, metadata: CompensationMetadata) {
    // Compensation logic
  }
}

@SagaStep({ order: 2, description: 'Process payment for the order' })
class ProcessPayment {
  @Invoke({ description: 'Process payment transaction' })
  public async processPayment(context: OrderContext, metadata: TransactionMetadata) {
    // Transaction logic
    return { paymentId: 'payment-456' };
  }
  
  @Compensation({ description: 'Refund payment' })
  public async refundPayment(context: OrderContext, transactionOutput: any, metadata: CompensationMetadata) {
    // Compensation logic
  }
}

// Run the Saga with dependency injection
const result = await SagaRunner.run<OrderContext>(
  [CreateOrder, ProcessPayment],
  {
    global: {
      customerId: 'customer-789'
    },
    steps: {}
  },
  {
    useStructuredContext: true,
    sagaOptions: { timeout: 10000 }
  }
);
```

### Choreography Approach

#### Standard Approach

```typescript
// Create a Saga participant
const orderService = new SagaParticipant(
  { name: 'OrderService', subscribeTopic: 'order-events', publishTopic: 'payment-events' },
  publisher,
  subscriber
);

// Handle events
orderService.on('SAGA_STARTED', async (event) => {
  const context = event.payload;
  
  // Create an order
  const orderId = 'order-123';
  
  // Update context
  context.steps.CreateOrder = {
    input: {
      customerId: context.global.customerId
    },
    output: {
      orderId
    },
    compensation: {}
  };
  
  // Initialize input for the next step
  context.steps.ProcessPayment = {
    input: {
      orderId,
      customerId: context.global.customerId,
      amount: 100
    },
    output: {},
    compensation: {}
  };
  
  // Publish event
  await orderService.publish('ORDER_CREATED', context, event.sagaId);
});

// Create a Saga coordinator
const coordinator = new StructuredSagaCoordinator(
  orderService,
  { name: 'OrderSagaCoordinator', subscribeTopic: 'saga-events', publishTopic: 'order-events' }
);

// Execute the Saga
const result = await coordinator.execute({
  global: {
    customerId: 'customer-789'
  },
  steps: {}
});
```

#### Dependency Injection Approach

```typescript
// Define Saga steps using decorators
@SagaStep({ order: 1, description: 'Create order in the system' })
class CreateOrder {
  @Invoke({ description: 'Process order creation' })
  public async processOrder(context: OrderContext, metadata: TransactionMetadata) {
    // Transaction logic
    const orderId = `order-${Date.now()}`;
    
    // Initialize input for the next step
    if (!context.steps.ProcessPayment) {
      context.steps.ProcessPayment = {
        input: {
          orderId,
          customerId: context.global.customerId,
          amount: 100
        },
        output: {},
        compensation: {}
      };
    }
    
    return { orderId };
  }
  
  @Compensation({ description: 'Cancel order' })
  public async cancelOrder(context: OrderContext, transactionOutput: any, metadata: CompensationMetadata) {
    // Compensation logic
  }
}

// Create a Saga participant
const orderService = new SagaParticipant(
  { name: 'OrderService', subscribeTopic: 'order-events', publishTopic: 'payment-events' },
  publisher,
  subscriber
);

// Handle events using the decorated class
orderService.on('SAGA_STARTED', async (event) => {
  const context = event.payload;
  
  // Create an instance of the CreateOrder step
  const createOrderStep = new CreateOrder();
  
  // Execute the step
  const output = await createOrderStep.processOrder(context, {
    id: `tx-${Date.now()}`,
    name: 'CreateOrder',
    timestamp: Date.now()
  });
  
  // Update step output
  if (context.steps.CreateOrder) {
    context.steps.CreateOrder.output = output;
  }
  
  // Publish event
  await orderService.publish('ORDER_CREATED', context, event.sagaId);
});

// Create a Saga coordinator
const coordinator = new DISagaCoordinator(
  orderService,
  { name: 'OrderSagaCoordinator', subscribeTopic: 'saga-events', publishTopic: 'order-events' }
);

// Execute the Saga
const result = await coordinator.execute({
  global: {
    customerId: 'customer-789'
  },
  steps: {}
});
```

## Documentation

### Key Concepts

#### Saga

A Saga is a sequence of local transactions. Each local transaction updates the database and publishes a message or event to trigger the next local transaction in the saga. If a local transaction fails because it violates a business rule then the saga executes a series of compensating transactions that undo the changes that were made by the preceding local transactions.

#### Transaction

A Transaction is a step in a Saga that performs a specific action. If a transaction fails, the Saga will execute compensating transactions to undo the changes made by previous transactions.

#### Compensation

A Compensation is a step that undoes the changes made by a transaction. Compensations are executed in reverse order when a transaction fails.

#### Orchestration vs Choreography

The library supports two approaches to implementing Sagas:

1. **Orchestration**: A central coordinator (orchestrator) tells the participants what local transactions to execute. The orchestrator is responsible for sequencing the transactions and handling failures.

2. **Choreography**: Each service publishes events that trigger local transactions in other services. There is no central coordinator, and the services communicate directly with each other.

### Examples

Check out the examples directory for usage examples:

#### Structured Context Examples
- [Orchestration Example](examples/structured-context-example.ts) - An example of using the Orchestration approach with structured context
- [Choreography Example](examples/structured-choreography-example.ts) - An example of using the Choreography approach with structured context

#### Dependency Injection Examples
- [DI Orchestration Example](examples/di-orchestration-example.ts) - An example of using the Orchestration approach with dependency injection
- [DI Choreography Example](examples/di-choreography-example.ts) - An example of using the Choreography approach with dependency injection

### Important Notes

1. **Structured Context**: The library supports a structured context format that organizes data by step:
   ```typescript
   {
     global: { /* global data available to all steps */ },
     steps: {
       "Step Name": {
         input: { /* input data for the step */ },
         output: { /* output data from the step */ },
         compensation: { /* data for compensation */ }
       }
     }
   }
   ```
   This makes it clear what data is used by each step and what data is produced.

2. **Strongly Typed Context**: The library supports strongly typed context for transactions and compensations, making it easier to understand what data is required for each step and what data is produced.

3. **Metadata for Transactions and Compensations**: Each transaction and compensation function receives metadata that provides additional information about the step being executed, such as IDs, names, and timestamps.

4. **Compensation Functions**: Compensation functions receive the output of the transaction they are compensating, making it easier to undo the changes made by the transaction.

5. **State Management**: The library provides both in-memory and file system state stores for Saga state. The file system state store includes automatic cleanup of old state files based on TTL and maximum file count.

6. **Dependency Injection**: The library supports a decorator-based approach for defining Saga steps:

   ```typescript
   @SagaStep({ order: 1, description: "Debit money from source account" })
   class DebitAccount {
     @Invoke({ description: "Process debit transaction" })
     public async processDebit(
       context: Context,
       metadata: TransactionMetadata
     ) {
       // Transaction logic

       // Initialize input for the next step
       if (!context.steps.CreditAccount) {
         context.steps.CreditAccount = {
           input: {
             // Input data for the next step
           },
           output: {} as any,
           compensation: {},
         };
       }

       // Return output
       return {
         // Output data
       };
     }

     @Compensation({ description: "Reverse debit transaction" })
     public async reverseDebit(
       context: Context,
       transactionOutput: any,
       metadata: CompensationMetadata
     ) {
       // Compensation logic
     }
   }
   ```

   This makes it easy to organize your Saga steps as classes and use dependency injection. Both Orchestration and Choreography approaches support this pattern.

7. **Message Brokers**: The library provides placeholder implementations for Kafka, RabbitMQ, and Redis adapters. In a production environment, you should implement proper adapters for your message broker of choice.

## License

MIT