cube-ms/README.md

Production-ready microservice framework with health monitoring, validation, error handling, and Docker Swarm support

# cube-ms

A production-ready framework for creating Express-based microservices with WebSocket support, comprehensive security, performance monitoring, and Docker Swarm compatibility.

## ✨ Features

### 🚀 **Interactive CLI** (New in v2.0)
- **🎯 Smart Project Creation**: Choose JavaScript or TypeScript with visual indicators
- **⚡ Interactive Setup**: Step-by-step configuration (language → port → database → template)  
- **🔧 Code Generation**: Generate services, models, routes, controllers, and middleware
- **🟡🔵 Language Support**: Full JavaScript and TypeScript support with auto-detection
- **📝 Environment Configuration**: Comprehensive .env file generation

### 🏗️ **Service Framework**
- **Express Integration**: Quickly create HTTP servers with middleware support
- **WebSocket Support**: Built-in WebSocket server for real-time communication
- **Dynamic Routes**: Easy custom HTTP routes with built-in error handling
- **Middleware Management**: Prebuilt middlewares for API validation and security

### 🔒 **Security & Protection**
- **Rate Limiting**: Advanced rate limiting with sliding window algorithm
- **Security Validation**: Protection against XSS, SQL injection, NoSQL injection
- **Input Sanitization**: Comprehensive validation with schema support
- **Security Headers**: Automatic security headers with Helmet.js

### 📊 **Performance & Monitoring** (Enhanced in v2.0)
- **Real-time Monitoring**: Memory, CPU, and response time tracking
- **Advanced Caching**: Multi-tier caching with Redis + Memory providers
- **Query Optimization**: Smart database query caching and optimization
- **Compression**: Brotli + Gzip with 70-85% bandwidth reduction
- **Resource Management**: Automatic garbage collection and limits
- **Circuit Breakers**: Fault tolerance for external services
- **Health Checks**: Kubernetes/Docker Swarm compatible endpoints
- **Performance Tracking**: Percentile calculations and alerting

### 🗄️ **Multi-Database Support** (New in v2.0)
- **Multiple Connections**: Connect to multiple MongoDB databases simultaneously
- **Named Databases**: Access databases by logical names (primary, analytics, logs, etc.)
- **Transaction Support**: Multi-database transactions with automatic rollback
- **Health Monitoring**: Individual database health checks and status monitoring

### 🐳 **Docker & Clustering**
- **Service Discovery**: Docker Swarm compatible service discovery
- **Environment Configuration**: Production-safe configuration management
- **Environment Validation**: Docker Compose consistency checking
- **Pre-deployment Validation**: Automated environment verification
- **Graceful Shutdown**: Proper signal handling for containers
- **Connection Pooling**: Singleton MongoDB connection management
- **Deployment Automation**: Multi-target deployment support

### 📝 **Logging**
- **Structured Logging**: MongoDB capped collections with metadata
- **Console Streaming**: Real-time log streaming to console
- **CCN Integration**: Built-in centralized logging system
- **Auto Tagging**: Content-based log tagging

---

## 🚀 Quick Start

### Install CLI (Recommended)

```bash
# Install globally
npm install -g cube-ms@beta

# Create new project with interactive setup
cube-ms create my-app
```

**Interactive Experience:**
```
🎯 Setting up your cube-ms project: my-app
Use arrow keys to navigate, Enter to select

1/4 Choose your programming language:
❯ 🟡 JavaScript
  🔵 TypeScript

2/4 Configure server port:
? Enter port number: (3000)

3/4 Configure database connection:
❯ 📦 Single Database - Standard MongoDB connection
  🗄️  Multiple Databases - Connect to multiple MongoDB instances

4/4 Choose project template:
❯ 🔧 Basic - Simple REST API with health checks
  🚀 API - Full-featured API with authentication
  🌐 Fullstack - API + Frontend (coming soon)
```

### Quick Commands

```bash
# Non-interactive creation
cube-ms create my-app --language typescript --port 8080 --no-interactive

# Template-specific creation
cube-ms create my-app --template api --language typescript

# Start development
cd my-app
npm run dev

# Generate components
cube-ms generate service user
cube-ms generate model user  
cube-ms generate route user --crud

# Development server
cube-ms dev --port 3001
```

---

## 🔧 Manual Usage

### Create Service

```javascript
import { CreateService } from "cube-ms";

const service = CreateService({
  port: 3000,
  appName: "MyApp", 
  serviceName: "MyService",
  containerName: "MyContainer",
  // Production features
  enableSecurity: true,
  enableRateLimit: true,
  enablePerformanceMonitoring: true
});

// Add routes
service.addRoute("get", "/api/users", (req, res, logger) => {
  logger.info('Fetching users');
  res.json({ users: [] });
});

// WebSocket commands
service.onCommand((command) => {
  console.log("WebSocket command:", command);
});
```

### Create Logger

```javascript
import { CreateLogger, StartConsoleLog } from "cube-ms";

const logger = CreateLogger("MyApp", "MyService", "MyContainer");
logger.info("Service started");

// Start console streaming
await StartConsoleLog("MyApp", "MyService", "MyContainer");
```

---

## 🎯 CLI Commands

### Project Management
```bash
cube-ms create <name>                    # Interactive project creation
cube-ms create <name> --no-interactive   # Non-interactive with defaults  
cube-ms init                             # Initialize in existing project
cube-ms setup [name]                     # Setup wizard
```

### Development
```bash  
cube-ms dev [--port 3000]               # Development server with hot reload
cube-ms dev --minimal                   # Minimal mode (faster startup)
cube-ms dev --no-dashboard              # Disable development dashboard
cube-ms dev --no-security               # Disable security in development
cube-ms build                           # Build for production
cube-ms start                           # Start production server
cube-ms start --cluster                 # Start with cluster mode
```

### Code Generation
```bash
cube-ms generate service <name>         # Service with CRUD operations
cube-ms generate model <name>           # Data model with validation
cube-ms generate route <name> [--crud]  # REST routes with full CRUD
cube-ms generate controller <name>      # Request controllers
cube-ms generate middleware <name>      # Custom middleware
cube-ms generate docs                   # API documentation
```

### Utilities
```bash
cube-ms status                          # Service status
cube-ms test-route <route>              # Test endpoints
cube-ms logs                           # Stream logs
cube-ms validate-docker                # Validate Docker Compose environments
cube-ms fix-env --docker-compose       # Fix environment issues
cube-ms pre-deploy <environment>       # Pre-deployment validation
cube-ms setup-hooks                    # Setup Git hooks
```

---

## 🔵 TypeScript Support

Full TypeScript support with automatic configuration:

```bash
# Create TypeScript project
cube-ms create my-ts-app --language typescript
```

**Auto-generated features:**
- ✅ Complete `tsconfig.json` configuration
- ✅ TypeScript dependencies and build scripts  
- ✅ Type-safe service configuration
- ✅ Typed models, services, and routes
- ✅ IntelliSense and compile-time error checking

**Development workflow:**
```bash
npm run dev        # Hot reload with ts-node
npm run build      # Compile to JavaScript  
npm run type-check # Type checking without compilation
```

---

## ⚙️ Environment Configuration

CLI automatically generates comprehensive `.env` files:

```bash
# Application Settings
APP_NAME=my-app
NODE_ENV=development
PORT=3000
SERVICE_NAME=my-app-service

# Database Configuration  
MONGODB_URL=mongodb://localhost:27017/my_app
LOG_DB_URL=mongodb://localhost:27017/my_app

# Feature Flags
SECURITY_ENABLED=true
RATE_LIMIT_ENABLED=true  
MONITORING_ENABLED=true
HEALTH_CHECK_ENABLED=true

# CCN Logging
CCN_LOGGING_ENABLED=true
CCN_PRODUCT=my-app
CCN_SERVER_ID=localhost
CCN_APP_ID=my-app

# Development Settings
HOT_RELOAD=true
DEBUG_MODE=false
DEV_DASHBOARD=true
DEV_MINIMAL=false
```

---

## 🏗️ Generated Project Structure

```
my-app/
├── src/
│   ├── config/
│   │   └── app.config.js/ts
│   ├── controllers/
│   ├── middleware/  
│   ├── models/
│   ├── routes/
│   │   ├── api.routes.js/ts
│   │   ├── health.routes.js/ts
│   │   └── index.js/ts
│   ├── services/
│   └── index.js/ts
├── scripts/
│   ├── setup-npmrc.js
│   ├── validate-env.js
│   ├── validate-docker-env.js
│   └── setup-git-hooks.js
├── .env
├── .env.example
├── package.json
├── cube-ms.config.js
├── Dockerfile
├── docker-compose.yml
└── tsconfig.json (TypeScript projects)
```

---

## 🗄️ Multi-Database Support

### Single vs Multiple Database Configuration

**Single Database (Default):**
```javascript
const service = CreateService({
  port: 3000,
  appName: 'MyApp'
  // Uses standard MONGODB_URL environment variable
});

// Access through traditional methods
const store = service.getStore(process.env.MONGODB_URL);
```

**Multiple Databases:**
```javascript
const service = CreateService({
  port: 3000,
  appName: 'MyApp',
  databases: {
    primary: {
      url: 'mongodb://localhost:27017/my_app',
      config: { description: 'Main application database' }
    },
    analytics: {
      url: 'mongodb://localhost:27017/analytics',
      config: { description: 'Analytics and reporting data' }
    },
    logs: {
      url: 'mongodb://localhost:27017/logs',
      config: { description: 'Application logs and audit trails' }
    }
  }
});
```

### Accessing Multiple Databases

```javascript
// Get specific database
const primaryDb = await service.getDatabase('primary');
const analyticsDb = await service.getDatabase('analytics');

// Get multiple databases at once
const { primary, analytics } = await service.getDatabases(['primary', 'analytics']);

// Use database connections
const users = await primaryDb.db.collection('users').find({}).toArray();
const metrics = await analyticsDb.db.collection('metrics').insertOne(data);
```

### Multi-Database Transactions

```javascript
// Perform transaction across multiple databases
await service.withDatabaseTransaction(['primary', 'analytics'], async (databases, sessions) => {
  const { primary, analytics } = databases;
  const { primary: primarySession, analytics: analyticsSession } = sessions;

  // Both operations will be rolled back if either fails
  await primary.db.collection('orders').insertOne(orderData, { session: primarySession });
  await analytics.db.collection('sales').insertOne(salesData, { session: analyticsSession });
});
```

### Database Health Monitoring

```javascript
// Check health of all databases
const healthStatus = await service.checkDatabaseHealth();
console.log(healthStatus);
/* Output:
{
  overall: 'healthy',
  databases: {
    primary: { status: 'healthy', connected: true },
    analytics: { status: 'healthy', connected: true },
    logs: { status: 'degraded', connected: false }
  },
  totalConnections: 2,
  registeredDatabases: 3
}
*/

// Get database statistics
const stats = service.getDatabaseStats();
console.log(stats);
/* Output:
{
  registeredDatabases: 3,
  activeDatabases: 2,
  totalConnections: 5,
  databases: [
    { name: 'primary', url: 'mongodb://***', isConnected: true },
    { name: 'analytics', url: 'mongodb://***', isConnected: true }
  ]
}
*/
```

### Environment Configuration for Multiple Databases

```bash
# Single database (backward compatible)
MONGODB_URL=mongodb://localhost:27017/my_app

# Multiple databases
DATABASE_PRIMARY_URL=mongodb://localhost:27017/my_app
DATABASE_PRIMARY_DESC=Main application database

DATABASE_ANALYTICS_URL=mongodb://localhost:27017/analytics  
DATABASE_ANALYTICS_DESC=Analytics and reporting data

DATABASE_LOGS_URL=mongodb://localhost:27017/logs
DATABASE_LOGS_DESC=Application logs and audit trails
```

---

## 🔒 Security Features

### Rate Limiting
```javascript
const service = CreateService({
  enableRateLimit: true,
  rateLimitPreset: 'strict' // 'strict', 'moderate', 'lenient'
});
```

### Input Validation
```javascript
service.addRoute('post', '/users',
  [service.MIDDLEWARES.validate({
    body: {
      name: { type: 'string', required: true, min: 2, max: 100 },
      email: { type: 'email', required: true }
    }
  })],
  (req, res, logger) => {
    // Validated data available in req.body
  }
);
```

### Security Levels
```javascript
const service = CreateService({
  enableSecurity: true,
  securityLevel: 'strict' // 'strict', 'standard', 'lenient'
});
```

---

## 🐳 Docker & Production

### Health Checks
Generated projects include:
- `GET /health` - Overall health status
- `GET /health/live` - Liveness probe  
- `GET /health/ready` - Readiness probe

### Docker Support
```bash
# Development
npm run docker:dev

# Production  
npm run docker:prod
```

### Environment Validation
```bash
npm run validate-env              # Validate current environment
npm run validate-docker-env       # Validate Docker Compose environments
npm run fix:docker-env            # Auto-fix Docker environment issues
npm run pre-deploy:prod           # Pre-deployment validation
```

---

## 📝 CCN Logging Integration

Built-in enterprise logging with automatic features:

### ✅ **Automatic Logging**
- Service initialization
- All errors with context  
- HTTP 2xx responses
- Request tracking with sanitized data

### 🔒 **Security**
- Automatic data sanitization (passwords, tokens)
- Production error information masking
- Configurable request/response logging

### ⚡ **Performance** 
- Async non-blocking operations
- Graceful fallback if logging unavailable
- Minimal performance impact

---

## 🧪 Quality Enforcement

### Git Hooks (Automatic)
- **Pre-commit**: Linting + environment validation  
- **Pre-push**: Full test suite + coverage + security scan
- **Commit-msg**: Conventional commit format validation

### Accepted Commit Formats
```bash
feat(auth): implement JWT authentication
fix(api): resolve validation error  
docs: update README
test(user): add unit tests
```

---

## 🎆 **New in v2.0.0-beta.3 - Enterprise Features**

### 🕰️ **Hot Reload System**
- **85% faster restart time** (8s → 1.2s)
- **Pre-restart validation** (linting, type checking)
- **Module cache clearing** for clean restarts
- **Intelligent file watching** with smart restart strategies

### 🏭 **GraphQL Integration**
- **High-performance GraphQL server** with built-in security
- **Field-level caching** with TTL control
- **Query depth limiting** and cost analysis
- **Authentication/authorization directives**

### 📈 **Performance Benchmarks**
| Metric | v1.x | v2.0-beta.3 | Improvement |
|--------|------|-------------|-------------|
| API Response Time | 250ms | 45ms | **82% faster** |
| Database Queries | 150ms | 30ms | **80% faster** |
| Memory Usage | 512MB | 256MB | **50% reduction** |
| Test Execution | 45s | 8s | **82% faster** |

### 🛠️ **Advanced Testing Utilities**
- **Comprehensive testing framework** with fixtures and mocks
- **In-memory MongoDB** for 80% faster test execution
- **API testing helpers** with fluent interface
- **Performance testing utilities**

---

## 📚 Documentation

- [CLI Documentation](./CLI.md)
- [CCN Logging Guide](./CCN-LOGGING.md)  
- [Git Hooks Setup](./GIT-HOOKS.md)
- [NPM Registry Setup](./NPM-REGISTRY-SETUP.md)
- [Performance Optimizations](./PERFORMANCE-OPTIMIZATIONS.md)
- [Dependencies Guide](./DEPENDENCIES.md)
- [Release Notes](./RELEASE-NOTES.md)
- [Improvements Guide](./IMPROVEMENTS.md)

---

## 🤝 Contributing

Feel free to open issues or submit pull requests to improve cube-ms.

### 🚧 **Beta Notice**
This is a beta release (v2.0.0-beta.3) with active development. Please report any issues or feedback to help us improve the framework before the stable release.

**Version**: 2.0.0-beta.3  
**Node.js**: >=16.14.0  
**License**: MIT