@hemantwasthere/monitoring-sdk/README.md

Centralized monitoring SDK for Node.js applications with Prometheus, Loki, and Grafana integration

# @hemantwasthere/monitoring-sdk

> Comprehensive monitoring SDK for Node.js applications with Prometheus, Loki, and Grafana integration

## 🚀 Quick Start

```bash
npm install @hemantwasthere/monitoring-sdk
```

```typescript
import { MonitoringSDK } from '@hemantwasthere/monitoring-sdk';

const monitoring = MonitoringSDK.initializeWithDefaults({
  projectName: 'my-app',
  serviceName: 'api-server',
  technology: 'express' // 'nestjs', 'nodejs'
});

// Automatic HTTP monitoring
app.use(monitoring.express.middleware());

// Custom metrics
monitoring.metrics.incrementCounter('orders_total');
monitoring.logger.info('Order processed', { orderId: '123' });
```

## 📊 Features

- ✅ **Easy Integration**: Drop-in for Express, NestJS, Node.js
- ✅ **Prometheus Metrics**: HTTP metrics + custom counters/gauges
- ✅ **Centralized Logging**: Structured logs to Loki via Winston
- ✅ **Cron Monitoring**: Built-in job monitoring decorators
- ✅ **Auto Configuration**: Environment-based setup
- ✅ **Consistent Labels**: Standardized across all metrics/logs

## ⚙️ Configuration

Set environment variables:

```bash
# Required
LOKI_HOST=http://localhost:3100
PROMETHEUS_HOST=http://localhost:9090

# Optional
PROJECT_NAME=my-project
NODE_ENV=development
```
export LOKI_URL=http://your-monitoring-server:3100
export NODE_ENV=production
export CLUSTER_NAME=prod-cluster
```

```typescript
import { MonitoringSDK } from '@hemantwasthere/monitoring-sdk';

// Auto-configures from environment variables
const monitoring = MonitoringSDK.initializeWithDefaults({
  projectName: 'my-project',
  serviceName: 'user-service',
  technology: 'nestjs'
});
```

### Method 2: Global Configuration (Recommended)

Set up global configuration once at your application startup:

```typescript
import { MonitoringSDK } from '@hemantwasthere/monitoring-sdk';

// Set global configuration once
MonitoringSDK.setGlobalConfig({
  lokiHost: 'http://your-monitoring-server:3100',
  environment: process.env.NODE_ENV || 'development',
  logLevel: 'info',
  customLabels: {
    region: 'us-east-1',
    cluster: 'prod-cluster'
  }
});

// Now initialize each service without repeating common configuration
const monitoring = MonitoringSDK.initialize({
  projectName: 'my-project',
  serviceName: 'user-service',
  technology: 'nestjs'
});
```

### Method 3: Full Configuration (Most Control)

Configure everything explicitly for each service:

```typescript
const monitoring = MonitoringSDK.initialize({
  projectName: 'my-project',
  serviceName: 'user-service',
  technology: 'nestjs',
  lokiHost: 'http://your-monitoring-server:3100',
  environment: 'production',
  logLevel: 'info',
  customLabels: { region: 'us-east-1' }
});
```

**Summary:**
- **Method 1**: Perfect for microservices with consistent environment setup
- **Method 2**: Best for applications where you want explicit control over global config
- **Method 3**: Use when each service needs different configuration

## Framework Integration

### NestJS

```typescript
import { MonitoringSDK, MonitoringInterceptor } from '@hemantwasthere/monitoring-sdk';
import { APP_INTERCEPTOR } from '@nestjs/core';

// In your main.ts
MonitoringSDK.setGlobalConfig({
  lokiHost: 'http://monitoring-server:3100',
  environment: process.env.NODE_ENV,
});

const monitoring = MonitoringSDK.initialize({
  projectName: 'ecommerce',
  serviceName: 'user-service',
  technology: 'nestjs'
});

// In your AppModule
@Module({
  providers: [
    {
      provide: APP_INTERCEPTOR,
      useClass: MonitoringInterceptor,
    },
  ],
})
export class AppModule {}

// Add metrics endpoint
app.getHttpAdapter().get('/metrics', async (req, res) => {
  const metrics = await monitoring.getMetrics().getMetrics();
  res.setHeader('Content-Type', monitoring.getMetrics().getRegistry().contentType);
  res.send(metrics);
});

// Add health endpoint
app.getHttpAdapter().get('/health', (req, res) => {
  res.json({ status: 'ok', timestamp: new Date().toISOString() });
});
```

### Express

```typescript
import { MonitoringSDK, expressMetricsMiddleware, MonitoringController } from '@hemantwasthere/monitoring-sdk';

MonitoringSDK.setGlobalConfig({
  lokiHost: 'http://monitoring-server:3100',
  environment: process.env.NODE_ENV,
});

const monitoring = MonitoringSDK.initialize({
  projectName: 'my-project',
  serviceName: 'payment-service',
  technology: 'express'
});

const app = express();

// Add metrics middleware
app.use(expressMetricsMiddleware());

// Add monitoring endpoints
const controller = new MonitoringController();
app.get('/metrics', (req, res) => controller.getMetrics(req, res));
app.get('/health', (req, res) => controller.getHealth(req, res));
```

### Cron Job Monitoring

```typescript
import { CronMonitor } from '@hemantwasthere/monitoring-sdk';

class DataProcessingService {
  @CronMonitor.monitor('daily-report-generation')
  async generateDailyReports() {
    // Your cron job logic here
    // Metrics will be automatically collected:
    // - Execution duration
    // - Success/failure counts
    // - Error details if job fails
  }

  @CronMonitor.monitor('cleanup-old-data', { timeout: 30000 })
  async cleanupOldData() {
    // Job with custom timeout
  }
}
```

## Custom Metrics

```typescript
const monitoring = MonitoringSDK.getInstance();
const metrics = monitoring.getMetrics();

// Counter
const orderCounter = metrics.createCounter('orders_total', 'Total orders processed', ['status', 'payment_method']);
orderCounter.inc({ status: 'completed', payment_method: 'credit_card' });

// Gauge
const activeUsers = metrics.createGauge('active_users', 'Number of active users');
activeUsers.set(150);

// Histogram
const responseTime = metrics.createHistogram('api_response_time', 'API response time', ['endpoint']);
responseTime.observe({ endpoint: '/api/users' }, 0.25);
```

## Logging

```typescript
const monitoring = MonitoringSDK.getInstance();
const logger = monitoring.getLogger();

logger.info('User logged in', { 
  userId: '12345', 
  correlationId: 'abc-123' 
});

logger.error('Database connection failed', { 
  error: 'Connection timeout',
  database: 'users-db' 
});
```

## Configuration Options

### Global Configuration

```typescript
interface GlobalMonitoringConfig {
  lokiHost: string;                    // Loki server URL
  environment?: string;                // Environment name (dev, prod, etc.)
  logLevel?: string;                   // Log level (info, debug, error)
  customLabels?: Record<string, string>; // Labels added to all metrics/logs
}
```

### Service Configuration

```typescript
interface MonitoringConfig {
  projectName: string;                 // Project/application name
  serviceName: string;                 // Service name within the project
  technology: string;                  // Technology stack (nestjs, express, nodejs)
  environment?: string;                // Override global environment
  lokiHost?: string;                   // Override global Loki host
  metricsPath?: string;                // Custom metrics endpoint path
  logLevel?: string;                   // Override global log level
  enableDefaultMetrics?: boolean;      // Enable Node.js default metrics
  prefixDefaultMetrics?: boolean;      // Prefix default metrics with project_service_
  customLabels?: Record<string, string>; // Additional service-specific labels
}
```

## Production Deployment

### 1. Docker Compose (Monitoring Stack)

Deploy the centralized monitoring stack:

```bash
# Clone the monitoring setup
git clone https://github.com/unwrap-labs/monitoring-setup.git
cd monitoring-setup/centralized-stack

# Start the monitoring stack
docker-compose up -d
```

This provides:
- Prometheus (metrics): http://localhost:9090
- Grafana (dashboards): http://localhost:3000
- Loki (logs): http://localhost:3100
- AlertManager (alerts): http://localhost:9093

### 2. Application Configuration

```typescript
// Production configuration
MonitoringSDK.setGlobalConfig({
  lokiHost: process.env.LOKI_URL || 'http://monitoring-cluster:3100',
  environment: 'production',
  logLevel: 'info',
  customLabels: {
    cluster: process.env.CLUSTER_NAME
  }
});
```

### 3. Environment Variables

```bash
# .env
LOKI_URL=http://your-monitoring-cluster:3100
NODE_ENV=production
LOG_LEVEL=info
CLUSTER_NAME=prod-cluster
```

## Collected Metrics

### HTTP Metrics
- `http_request_duration_milliseconds` - Request response times
- `http_requests_total` - Total request count by method, route, status

### Cron Job Metrics
- `cron_job_duration_seconds` - Job execution time
- `cron_job_executions_total` - Total job executions
- `cron_job_failures_total` - Failed job count

### System Metrics (when enabled)
- Node.js process metrics (memory, CPU, GC)
- Event loop lag
- Heap usage

## Best Practices

1. **Set Global Config Once**: Use `setGlobalConfig()` at application startup
2. **Consistent Naming**: Use kebab-case for project and service names
3. **Meaningful Labels**: Add relevant labels to metrics for better filtering
4. **Error Handling**: Always handle monitoring errors gracefully
5. **Resource Cleanup**: Monitoring shouldn't impact application performance

## Troubleshooting

### Common Issues

1. **Loki Connection Failed**: Ensure Loki host is accessible and port 3100 is open
2. **Metrics Not Appearing**: Check if `/metrics` endpoint is properly exposed
3. **High Memory Usage**: Reduce metric cardinality by limiting label values

### Debug Mode

```typescript
MonitoringSDK.setGlobalConfig({
  logLevel: 'debug',
  // ... other config
});
```

## Contributing

We welcome contributions! Please see our [Contributing Guide](https://github.com/unwrap-labs/monitoring-setup/blob/main/CONTRIBUTING.md) for details.

## License

MIT - see [LICENSE](LICENSE) file for details.

## Support

- 📚 [Documentation](https://github.com/unwrap-labs/monitoring-setup)
- 🐛 [Issue Tracker](https://github.com/unwrap-labs/monitoring-setup/issues)
- 💬 [Discussions](https://github.com/unwrap-labs/monitoring-setup/discussions)

## 📝 Examples

Check the `examples/` directory for complete integration examples:

- **Express.js**: `examples/apps/express-example.ts`
- **NestJS**: `examples/apps/nestjs-example.ts`
- **Node.js**: `examples/apps/nodejs-example.ts`

Each example shows how to integrate the monitoring SDK with different frameworks and use cases.

## 📁 Package Structure

```text
monitoring-sdk/
├── src/                    # Source code
│   ├── index.ts           # Main entry point
│   ├── metrics.ts         # Prometheus metrics
│   ├── logging.ts         # Loki logging
│   ├── cron-monitor.ts    # Cron job monitoring
│   └── middleware/        # Framework integrations
├── examples/              # Integration examples
│   └── apps/             # Framework-specific examples
├── dist/                  # Compiled JavaScript
└── package.json          # NPM configuration
```