dt-common-device/README.md

A secure and robust device management library for IoT applications

# dt-common-device

A comprehensive TypeScript library for device management, supporting both cloud and local device operations with advanced event handling, queue management, and audit logging.

## Installation

```sh
npm install dt-common-device
```

## Environment Variables Required

**The following environment variables are REQUIRED for the library to function:**

### AWS Configuration

- `AWS_SECRET_ACCESS_KEY` — Your AWS secret access key
- `AWS_REGION` — Your AWS region
- `AWS_ACCESS_KEY_ID` — Your AWS access key ID
- `EVENT_BUS_NAME` — Your AWS EventBridge event bus name

### Database Configuration

- `ADMIN_DB_URI` — PostgreSQL database connection URI
- `MONGODB_URI` — MongoDB database connection URI

### Redis Configuration

- `REDIS_HOST` — Redis server host
- `REDIS_PORT` — Redis server port

### Audit Logging

- `POSTHOG_API_KEY` — Your PostHog API key
- `POSTHOG_HOST` — The PostHog host URL

### SQS Configuration

- `SQS_QUEUE_URL` — AWS SQS queue URL (configured during initialization)

Example `.env` file:

```
# AWS Configuration
AWS_SECRET_ACCESS_KEY=your_secret_key
AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=your_access_key
EVENT_BUS_NAME=your-event-bus

# Database Configuration
ADMIN_DB_URI=postgres://username:password@host:port/database
MONGODB_URI=mongodb://username:password@host:port/database

# Redis Configuration
REDIS_HOST=localhost
REDIS_PORT=6379

# Audit Logging
POSTHOG_API_KEY=your_posthog_key
POSTHOG_HOST=https://app.posthog.com

# SQS Configuration (will be set during initialization)
SQS_QUEUE_URL=https://sqs.region.amazonaws.com/account/queue-name
```

The library will throw clear errors if any required environment variables are missing.

---

## Initialization (Required)

Before using any service, you **must** call `initialize()` in your main entry file with the required configuration:

```ts
import { initialize } from "dt-common-device";

// Create a logger instance
const logger = {
  info: (message: string, ...args: any[]) => console.log(message, ...args),
  warn: (message: string, ...args: any[]) => console.warn(message, ...args),
  error: (message: string, ...args: any[]) => console.error(message, ...args),
};

// Initialize the library
await initialize({
  SOURCE: "ADMIN_SERVICE", // or "ACCESS_SERVICE" or "ENERGY_SERVICE"
  SQS_QUEUE_URL: "https://sqs.region.amazonaws.com/account/queue-name",
  DEVICE_SERVICE: "https://api.example.com/device", // Optional
  ADMIN_SERVICE: "https://api.example.com/admin", // Optional
  ACCESS_SERVICE: "https://api.example.com/access", // Optional
  ENERGY_SERVICE: "https://api.example.com/energy", // Optional
  INTERNAL_EVENT_HANDLER: {
    // Your event handler implementation
    handleEvent: async (event) => {
      // Handle internal events
      console.log("Handling event:", event);
    },
  },
  LOGGER: logger,
});
```

### Configuration Options

- `SOURCE`: Required. Must be one of: `"ADMIN_SERVICE"`, `"ACCESS_SERVICE"`, or `"ENERGY_SERVICE"`
- `SQS_QUEUE_URL`: Required. Your AWS SQS queue URL
- `DEVICE_SERVICE`, `ADMIN_SERVICE`, `ACCESS_SERVICE`, `ENERGY_SERVICE`: Optional service URLs
- `INTERNAL_EVENT_HANDLER`: Required. Your event handler implementation
- `LOGGER`: Required. Logger instance with info, warn, and error methods

---

## Available Services

### Local Device Service

```ts
import { LocalDeviceService } from "dt-common-device";

const deviceService = new LocalDeviceService();

// Create a device
const device = await deviceService.createDevice(deviceBody);

// Get a device
const device = await deviceService.getDevice(deviceId);

// Get multiple devices
const devices = await deviceService.getDevices(deviceIds);

// Get devices by property
const propertyDevices = await deviceService.getPropertyDevices(propertyId);

// Update a device
await deviceService.updateDevice(deviceId, updateBody);

// Delete a device
await deviceService.deleteDevice(deviceId);

// State management
const state = await deviceService.getState(deviceId);
await deviceService.setState(deviceId, newState);

// Status management
const status = await deviceService.getStatus(deviceId);
await deviceService.setStatus(
  deviceId,
  newStatus,
  "heartbeat",
  "Device went offline due to network connectivity issues"
);

// Battery management
const battery = await deviceService.getBatteryLevel(deviceId);
await deviceService.setBatteryLevel(deviceId, batteryLevel, "heartbeat");

// Metadata management
const metadata = await deviceService.getMetaData(deviceId);
await deviceService.setMetaData(deviceId, metadata);

// Query operations
const devices = await deviceService.queryDevices(query);
const count = await deviceService.queryCount(query);
await deviceService.deleteDevices(query);
```

### Local Hub Service

```ts
import { LocalHubService } from "dt-common-device";

const hubService = new LocalHubService();

// Add a hub
const hub = await hubService.addHub(hubBody);

// Get hubs
const hubs = await hubService.getHubs(hubIds);

// Get a single hub
const hub = await hubService.getHub(hubId);

// Update a hub
await hubService.updateHub(hubId, updateBody);

// Get hub status
const status = await hubService.getStatus(hubId);

// Delete a hub
await hubService.deleteHub(hubId);

// Delete multiple hubs
await hubService.deleteAllHubs(hubIds);
```

### Local Schedule Service

```ts
import { LocalScheduleService } from "dt-common-device";

const scheduleService = new LocalScheduleService();

// Get a schedule
const schedule = await scheduleService.getSchedule(scheduleId);

// Set a schedule
await scheduleService.setSchedule(scheduleId, schedule);

// Get schedule by zone
const zoneSchedule = await scheduleService.getScheduleByZone(zoneId);
```

### Local Connection Service

```ts
import { LocalConnectionService } from "dt-common-device";

const connectionService = new LocalConnectionService();

// Create a connection
const connection = await connectionService.createConnection({
  connectionName: "My Connection",
  connectionRefId: "ref-123",
  propertyId: "prop-456",
  connectionProvider: "Sensibo",
});

// Get a connection
const connection = await connectionService.getConnection(connectionId);

// Update a connection
await connectionService.updateConnection(connectionId, updateData);
```

### Cloud Device Service

```ts
import { CloudDeviceService, DeviceFactory } from "dt-common-device";

const cloudService = new CloudDeviceService();
const deviceFactory = new DeviceFactory();

// Get cloud devices (must implement in your project)
// await cloudService.getDevices(connection);

// Get device using factory
const device = await deviceFactory.getDevice(deviceId);
```

### Property Service

```ts
import { PropertyService } from "dt-common-device";

const propertyService = new PropertyService();

// Property operations (implementation specific)
// await propertyService.getProperty(propertyId);
```

---

## Event System

The library includes a comprehensive event handling system:

```ts
import {
  EventHandler,
  EventProcessingService,
  DeviceEventHandler,
  EventHandlerOrchestrator,
} from "dt-common-device";

// Event handler for device operations
const eventHandler = new EventHandler();

// Device-specific event handler
const deviceEventHandler = new DeviceEventHandler();

// Event processing service
const eventProcessingService = new EventProcessingService();

// Event handler orchestrator
const orchestrator = new EventHandlerOrchestrator();
```

---

## Queue System

The library provides a hybrid HTTP queue system for managing HTTP requests:

```ts
import { QueueService } from "dt-common-device";

const queueService = new QueueService();

// Make a rate-limited HTTP request
const response = await queueService.request({
  method: "GET",
  url: "https://api.example.com/data",
  headers: { "Content-Type": "application/json" },
  queueOptions: {
    connectionId: "connection-123",
    connectionProvider: "Sensibo",
    microservice: "smart-energy",
  },
});
```

### Features

- **Rate Limiting**: Automatic rate limiting per provider and connection
- **Retry Logic**: Exponential backoff with configurable retry attempts
- **Audit Logging**: Automatic audit logging for all requests
- **Queue Management**: Redis-based queue with BullMQ
- **Error Handling**: Comprehensive error handling and logging

---

## Alert and Issue Management

```ts
import {
  AlertService,
  IssueService,
  AlertBuilder,
  IssueBuilder,
} from "dt-common-device";

// Alert service
const alertService = new AlertService();

// Issue service
const issueService = new IssueService();

// Build alerts
const alert = new AlertBuilder()
  .setTitle("Device Offline")
  .setDescription("Device has been offline for more than 24 hours")
  .setSeverity("HIGH")
  .build();

// Build issues
const issue = new IssueBuilder()
  .setTitle("Connection Failed")
  .setDescription("Failed to connect to device")
  .setPriority("HIGH")
  .build();
```

---

## Graceful Shutdown

```ts
import { shutdown } from "dt-common-device";

// Gracefully shutdown the library
await shutdown();
```

---

## Importing Types and Interfaces

All types and interfaces are available as named exports:

```ts
import {
  IDevice,
  IHub,
  IConnection,
  ISchedule,
  IProperty,
  IAlert,
  IIssue,
} from "dt-common-device";
```

---

## Dependencies

The library requires the following major dependencies:

- `axios`: HTTP client
- `bullmq`: Queue management
- `dt-audit-library`: Audit logging
- `dt-pub-sub`: Event publishing/subscribing
- `ioredis`: Redis client
- `mongoose`: MongoDB ODM
- `pg`: PostgreSQL client
- `typedi`: Dependency injection

---

## Notes

- You **must** call `initialize()` before using any service. If not, you will get a runtime error.
- **You must set `POSTHOG_API_KEY` and `POSTHOG_HOST` in your environment before using any local device service.**
- You do **not** need to call `initializeAudit()`

## Rate Limiting

The queue system now implements intelligent rate limiting that delays requests instead of dropping them when rate limits are exceeded.

### How it works

1. **Rate Limit Check**: Before processing each HTTP request, the system checks if the rate limit for the provider/connection combination has been exceeded.

2. **Delay Instead of Drop**: If the rate limit is exceeded, instead of immediately failing the request, the system:

   - Calculates when the next request can be processed (after the rate limit window expires)
   - Delays the job execution by the calculated time
   - Re-checks the rate limit after the delay
   - Only fails if still rate limited after the delay

3. **Example Scenario**:

   ```
   Rate Limit: 5 requests per 60 seconds

   Timeline:
   0s: Request 1 (processed immediately)
   10s: Request 2 (processed immediately)
   20s: Request 3 (processed immediately)
   30s: Request 4 (processed immediately)
   40s: Request 5 (processed immediately)
   45s: Request 6 (delayed by 15s, processed at 60s)
   50s: Request 7 (delayed by 10s, processed at 60s)
   ```

### Configuration

Rate limits are configured per provider in `src/queue/utils/rateLimit.utils.ts`:

```typescript
configs.set("Sensibo", {
  maxRequests: 5, // Maximum requests allowed
  windowMs: 60000, // Time window in milliseconds
  provider: "Sensibo",
});
```

### Benefits

- **No Lost Requests**: Requests are delayed rather than dropped
- **Automatic Recovery**: System automatically processes delayed requests when rate limits reset
- **Better User Experience**: Users don't see immediate failures due to rate limits
- **Audit Trail**: All rate limit events are logged for monitoring