@diagramers/api/README.md

Diagramers API - A comprehensive Node.js API template with TypeScript, Firebase Functions, and Socket.io

# Diagramers API

A comprehensive, modular, and extensible Node.js API framework built with TypeScript, featuring multi-provider authentication, plugin system, unified response handling, Swagger documentation, Socket.IO real-time communication, and MongoDB support.

## 🚀 Features

### Core Features
- **Modular Architecture** - Clean separation of concerns with modules, plugins, and shared components
- **Multi-Provider Authentication** - Internal, Firebase, OAuth, SMS OTP, Email OTP
- **Unified Response Format** - Consistent API responses using the Result model
- **Swagger Documentation** - Auto-generated API documentation
- **Database Seeding** - Comprehensive seeding system with environment-specific data
- **Plugin System** - Extensible plugin architecture for custom functionality
- **Environment Management** - Multi-environment configuration with .env support
- **Audit Logging** - Pluggable audit system supporting multiple backends
- **Socket.IO Integration** - Real-time communication with event handling
- **Auto Database Creation** - Automatically creates collections and indexes on startup

### Database Support
- **MongoDB** - Primary database with Mongoose ODM
- **SQL Databases** - MySQL, PostgreSQL, SQLite, MariaDB (configurable)
- **Connection Pooling** - Optimized database connections
- **Migration Support** - Database schema management
- **Auto-Collection Creation** - Creates collections and indexes automatically

### Authentication & Security
- **JWT Tokens** - Secure token-based authentication
- **Role-Based Access Control** - Granular permission system
- **Rate Limiting** - Built-in request throttling
- **CORS Support** - Configurable cross-origin requests
- **Helmet Security** - Security headers and protection

### Real-Time Communication
- **Socket.IO Server** - WebSocket support with event handling
- **Room Management** - Group-based messaging
- **Event Registration** - Custom event handlers
- **Connection Management** - Client tracking and management

## 📦 Installation

### Option 1: Install API Package Only
```bash
npm install @diagramers/api
```

### Option 2: Install CLI for Project Management
```bash
npm install -g @diagramers/cli
diagramers init api my-new-api
```

### Prerequisites
- Node.js 18+
- npm or yarn
- MongoDB (or other supported database)

## 🚀 Quick Start

### 1. Create New Project
```bash
# Using CLI (recommended)
diagramers init api my-new-api
cd my-new-api

# Or manually
npm install @diagramers/api
```

### 2. Set up Environment Variables
```bash
# Copy the example environment file
cp env.example .env.development

# Edit the development environment file
nano .env.development

# Set NODE_ENV to development
echo "NODE_ENV=development" >> .env.development
```

### 3. Install Dependencies
```bash
npm install
```

### 4. Start Development Server
```bash
npm start
```

## 📁 Project Structure

```
src/
├── core/                     # Core application components
│   ├── config/              # Configuration management
│   │   ├── index.ts         # Config manager
│   │   └── interfaces.ts    # Configuration interfaces
│   ├── database/            # Database management
│   │   ├── connection.ts    # Database connections
│   │   └── seeder.ts        # Database seeding
│   ├── server/              # Server setup
│   │   └── manager.ts       # Server manager with Swagger & Socket.IO
│   ├── logging/             # Logging system
│   └── app.ts               # Application initialization
├── modules/                 # Feature modules
│   ├── users/               # User management module
│   │   ├── entities/        # Data models
│   │   ├── schemas/         # Database schemas
│   │   ├── services/        # Business logic
│   │   ├── controllers/     # HTTP handlers
│   │   └── routes/          # Route definitions
│   ├── auth/                # Authentication module
│   └── audit/               # Audit logging module
├── plugins/                 # Plugin system
│   └── base/                # Base plugin classes
├── shared/                  # Shared utilities
│   ├── types/               # TypeScript types
│   │   ├── result.ts        # Unified response model
│   │   └── base-entity.ts   # Base entity interface
│   └── utils/               # Utility functions
└── main.ts                  # Application entry point
```

## ⚙️ Configuration

### Environment Variables

The API uses environment variables for configuration. Copy `env.example` to `.env.development` and customize:

#### Basic Configuration
```bash
NODE_ENV=development
PORT=4000
HOST=localhost
```

#### Logging Configuration
```bash
# Basic Logging
LOG_LEVEL=info                    # error, warn, info, debug, verbose
LOG_FORMAT=colored               # json, simple, colored
LOG_COLORS=true                  # Enable/disable colored output

# File Logging
LOG_FILE_ENABLED=false           # Enable file logging
LOG_FILE_PATH=./logs/app.log     # Log file path
LOG_FILE_MAX_SIZE=10m            # Max log file size
LOG_FILE_MAX_FILES=5             # Number of log files to keep

# Database Logging
LOG_DATABASE_ENABLED=false       # Enable database logging
LOG_DATABASE_COLLECTION=logs     # MongoDB collection name

# External Logging Services
LOG_EXTERNAL_ENABLED=false       # Enable external logging
LOG_EXTERNAL_TYPE=serilog        # serilog, sentry, loggly, papertrail
LOG_EXTERNAL_URL=                # External service URL
LOG_EXTERNAL_API_KEY=            # API key for external service
LOG_EXTERNAL_APP_NAME=           # Application name for external service
```

**Logging Backends Supported:**
- **Console** - Colored output with configurable format
- **File** - JSON or text format with rotation
- **Database** - MongoDB collection for structured logging
- **External Services** - Serilog, Sentry, Loggly, Papertrail
- **Custom Transports** - Extensible transport system

#### Database Configuration
```bash
# MongoDB (Primary)
MONGODB_URI=mongodb://127.0.0.1:27017/diagramers-api

# SQL Database (Alternative)
DB_HOST=localhost
DB_PORT=3306
DB_USERNAME=root
DB_PASSWORD=your_password
DB_DATABASE=diagramers
DB_DIALECT=mysql
DB_LOGGING=false
```

#### Socket.IO Configuration
```bash
SOCKETIO_ENABLED=true
SOCKETIO_CORS_ORIGIN=*
SOCKETIO_CORS_METHODS=GET,POST
```

#### Authentication
```bash
JWT_SECRET=your-super-secret-jwt-key
JWT_EXPIRES_IN=1h
REFRESH_TOKEN_EXPIRES_IN=7d

# Firebase Authentication
FIREBASE_ENABLED=false
FIREBASE_API_KEY=your-api-key
FIREBASE_AUTH_DOMAIN=your-project.firebaseapp.com
FIREBASE_PROJECT_ID=your-project-id
```

#### Email Configuration
```bash
EMAIL_ENABLED=false
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_USER=your-email@gmail.com
SMTP_PASS=your-app-password
EMAIL_FROM=noreply@yourdomain.com
```

#### SMS Configuration (Twilio)
```bash
TWILIO_ENABLED=false
TWILIO_ACCOUNT_SID=your-account-sid
TWILIO_AUTH_TOKEN=your-auth-token
TWILIO_PHONE_NUMBER=+1234567890
```

#### Security Configuration
```bash
CORS_ENABLED=true
CORS_ORIGIN=*
SECURITY_ENABLED=true
RATE_LIMIT_ENABLED=true
RATE_LIMIT_WINDOW_MS=900000
RATE_LIMIT_MAX_REQUESTS=100
```

## 🛠️ Available Scripts

### Development Scripts
```bash
npm start              # Start development server
npm run dev            # Start development server (alias)
npm run build          # Build for production
npm run build:dev      # Build for development
npm run clean          # Clean build artifacts
```

### Database Scripts
```bash
npm run seed           # Run database seeding
npm run seed:force     # Force seeding (overwrite existing data)
npm run seed:reset     # Reset and reseed database
npm run seed:truncate  # Truncate all data and reseed
```

### Module Generation Scripts
```bash
npm run generate:module product    # Generate new module
npm run process:template my-api    # Process template for new project
npm run cli help                   # Show CLI help
```

### Code Quality Scripts
```bash
npm run lint           # Run ESLint
npm run lint:fix       # Fix ESLint issues
npm run type-check     # Run TypeScript type checking
npm run format         # Format code with Prettier
npm run format:check   # Check code formatting
```

### Documentation Scripts
```bash
npm run docs           # Generate API documentation
npm run docs:serve     # Serve documentation locally
```

### Docker Scripts
```bash
npm run docker:build   # Build Docker image
npm run docker:run     # Run Docker container
npm run docker:compose # Start with Docker Compose
```

### Monitoring Scripts
```bash
npm run health         # Check server health
npm run logs           # Tail application logs
npm run logs:clear     # Clear log files
npm run monitor        # Start monitoring
```

### Deployment Scripts
```bash
npm run deploy         # Deploy to default environment
npm run deploy:staging # Deploy to staging
npm run deploy:production # Deploy to production
```

## 🔌 Socket.IO Usage

### Server-Side Event Registration
```typescript
import { ServerManager } from '@diagramers/api';

const serverManager = new ServerManager(config, pluginManager);

// Register custom events
serverManager.registerSocketEvent({
  event: 'user:join',
  handler: (socket, data) => {
    socket.join(`room:${data.userId}`);
    socket.emit('user:joined', { userId: data.userId });
  },
  description: 'User joins a room'
});

serverManager.registerSocketEvent({
  event: 'message:send',
  handler: (socket, data) => {
    // Broadcast to room
    serverManager.emitToRoom(`room:${data.roomId}`, 'message:received', data);
  },
  description: 'Send message to room'
});
```

### Client-Side Connection
```javascript
import { io } from 'socket.io-client';

const socket = io('http://localhost:4000');

// Listen for events
socket.on('connect', () => {
  console.log('Connected to server');
});

socket.on('message:received', (data) => {
  console.log('New message:', data);
});

// Emit events
socket.emit('user:join', { userId: '123' });
socket.emit('message:send', { roomId: 'room1', message: 'Hello!' });
```

## 🗄️ Database Auto-Creation

The API automatically creates database collections and indexes when the server starts:

```typescript
// Collections are created automatically for all registered models
// Indexes are created based on schema definitions

const userSchema = new mongoose.Schema({
  email: { type: String, required: true, unique: true },
  username: { type: String, required: true, index: true }
}, { timestamps: true });

// This will automatically create:
// - 'users' collection
// - Index on 'email' field (unique)
// - Index on 'username' field
```

## 📚 API Documentation

### Swagger UI
Access the interactive API documentation at: `http://localhost:4000/api-docs`

### Socket.IO Info
Get Socket.IO event information at: `http://localhost:4000/api/socket-info`

### Health Check
Check server status at: `http://localhost:4000/health`

## 🔧 Module Generation

### Using CLI (Recommended)
```bash
# Generate complete module
diagramers extend --module product --crud --fields name,price,category

# Generate table only
diagramers extend --table orders --fields customer_id,amount,status

# Generate relations
diagramers extend --relation user-posts --type mongodb
```

### Using API Scripts
```bash
# Generate module
npm run generate:module product

# Process template
npm run process:template my-api
```

## 🚀 Deployment

### Docker Deployment
```bash
# Build and run with Docker
npm run docker:build
npm run docker:run

# Or use Docker Compose
npm run docker:compose
```

### Manual Deployment
```bash
# Build for production
npm run build

# Start production server
NODE_ENV=production npm start
```

### Environment-Specific Deployment
```bash
# Deploy to staging
npm run deploy:staging

# Deploy to production
npm run deploy:production
```

## 🔍 Monitoring & Logging

### Log Levels
- `error` - Application errors
- `warn` - Warning messages
- `info` - General information
- `debug` - Debug information
- `verbose` - Detailed debugging

### Log Formats
- `json` - Structured JSON logging
- `simple` - Simple text format
- `colored` - Colored console output

### File Logging
```bash
# Enable file logging
LOG_FILE_ENABLED=true
LOG_FILE_PATH=./logs/app.log
LOG_FILE_MAX_SIZE=10m
LOG_FILE_MAX_FILES=5
```

## 🤝 Contributing

1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add some amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request

## 📄 License

This project is licensed under the MIT License.

## 📞 Support

For support and questions:
- 📖 Check the documentation
- 🐛 Open an issue on GitHub
- 💬 Join our community discussions
- 📧 Contact: support@diagramers.com

## 🔗 Related Packages

- **@diagramers/cli** - Command-line interface for project management
- **@diagramers/admin** - Admin dashboard template
- **@diagramers/shared** - Shared utilities and types