@mvp-factory/holy-upload/README.md

File upload processing system extracted from Holy Habit project with security validation and image optimization

# @mvp-factory/holy-upload

File upload processing system extracted from Holy Habit project with advanced security validation, image optimization, and Express.js integration.

## Features

- 🔐 **Advanced Security** - Comprehensive file validation with magic number verification
- 🖼️ **Image Optimization** - Automatic resizing, compression, and format conversion using Sharp
- 📁 **Multiple Storage** - Memory and disk storage with flexible configuration
- 🛡️ **Malicious Content Detection** - Script injection and virus pattern detection
- 📊 **Storage Management** - Usage tracking, cleanup utilities, and quota management
- 🚀 **Express Integration** - Ready-to-use middleware and route handlers
- 📤 **Multiple Upload Types** - Single, multiple, and field-based uploads
- 🎨 **Image Processing** - Thumbnail generation, WebP conversion, and EXIF removal
- 📦 **TypeScript Support** - Full TypeScript definitions and type safety

## Installation

```bash
npm install @mvp-factory/holy-upload
```

## Quick Start

### 1. Basic Setup

```typescript
import { HolyUpload } from '@mvp-factory/holy-upload';

// Create upload handler with default configuration
const uploadHandler = HolyUpload.createHandler({
  uploadDir: './uploads',
  maxSize: 10 * 1024 * 1024, // 10MB
  allowedMimeTypes: ['image/jpeg', 'image/png', 'image/gif', 'image/webp'],
  enableOptimization: true
});

// Upload a file
const result = await uploadHandler.uploadSingle(file, 'user123');
console.log(result); // { success: true, file: {...}, url: '/uploads/...' }
```

### 2. Express.js Integration

```typescript
import express from 'express';
import { HolyUpload } from '@mvp-factory/holy-upload';

const app = express();

// Create middleware
const uploadMiddleware = HolyUpload.createMiddleware({
  uploadDir: './uploads',
  maxSize: 5 * 1024 * 1024, // 5MB
  allowedMimeTypes: ['image/jpeg', 'image/png', 'image/webp']
});

// Single file upload endpoint
const [singleUpload, singleHandler] = uploadMiddleware.createCompleteEndpoint('single', 'image');
app.post('/api/upload', ...singleUpload, singleHandler);

// Multiple files upload endpoint
const [multipleUpload, multipleHandler] = uploadMiddleware.createCompleteEndpoint('multiple', 'images');
app.post('/api/upload/multiple', ...multipleUpload, multipleHandler);

// Serve uploaded files
app.use('/uploads', express.static('./uploads'));

app.listen(3000);
```

### 3. Environment Configuration

Create a `.env` file:

```env
# Upload Configuration
UPLOAD_DIR=./uploads
UPLOAD_MAX_SIZE=10485760
UPLOAD_MAX_WIDTH=1920
UPLOAD_MAX_HEIGHT=1080
UPLOAD_ALLOWED_MIME_TYPES=image/jpeg,image/png,image/gif,image/webp
UPLOAD_ALLOWED_EXTENSIONS=jpg,jpeg,png,gif,webp
UPLOAD_ENABLE_OPTIMIZATION=true
UPLOAD_JPEG_QUALITY=85
UPLOAD_PNG_COMPRESSION=8
UPLOAD_WEBP_QUALITY=85
```

## Core Components

### UploadHandler

Main class for handling file uploads:

```typescript
import { UploadHandler } from '@mvp-factory/holy-upload';

const handler = new UploadHandler({
  uploadDir: './uploads',
  maxSize: 10 * 1024 * 1024,
  enableOptimization: true
});

// Single file upload
const result = await handler.uploadSingle(file, 'user123');

// Multiple files upload
const results = await handler.uploadMultiple(files, 'user123', {
  continueOnError: true
});

// Delete file
await handler.deleteFile('filename.jpg', 'user123');

// Get storage info
const storage = await handler.getStorageInfo('user123');
```

### Express Middleware

Complete Express.js integration:

```typescript
import { ExpressUploadMiddleware } from '@mvp-factory/holy-upload';

const middleware = new ExpressUploadMiddleware({
  uploadDir: './uploads',
  maxSize: 5 * 1024 * 1024
});

// Use middleware in routes
app.post('/upload', 
  middleware.single('image'), 
  async (req, res) => {
    try {
      const result = await req.uploadSingle!();
      res.json(result);
    } catch (error) {
      res.status(400).json({ success: false, error: error.message });
    }
  }
);
```

### File Validation

Advanced security validation:

```typescript
import { FileValidator } from '@mvp-factory/holy-upload';

const validation = await FileValidator.validate(file, {
  maxSize: 5 * 1024 * 1024,
  allowedMimeTypes: ['image/jpeg', 'image/png'],
  allowedExtensions: ['jpg', 'jpeg', 'png'],
  checkMalicious: true
});

if (!validation.isValid) {
  console.log('Validation errors:', validation.errors);
}
```

### Image Optimization

Powerful image processing:

```typescript
import { ImageOptimizer } from '@mvp-factory/holy-upload';

// Optimize image
const result = await ImageOptimizer.optimize('input.jpg', 'output.jpg', {
  maxWidth: 1920,
  maxHeight: 1080,
  jpegQuality: 85
});

// Convert to WebP
const webpResult = await ImageOptimizer.convertToWebP('image.jpg', 'image.webp', 80);

// Generate thumbnails
const thumbnails = await ImageOptimizer.generateThumbnails('image.jpg', [
  { width: 150, height: 150, suffix: 'thumb' },
  { width: 300, height: 300, suffix: 'small' }
], './thumbnails');
```

## Configuration Options

### UploadConfig

| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `uploadDir` | string | `'./uploads'` | Upload directory path |
| `maxSize` | number | `10 * 1024 * 1024` | Maximum file size (10MB) |
| `maxWidth` | number | `1920` | Maximum image width |
| `maxHeight` | number | `1080` | Maximum image height |
| `allowedMimeTypes` | string[] | `['image/jpeg', 'image/png', 'image/gif', 'image/webp']` | Allowed MIME types |
| `allowedExtensions` | string[] | `['jpg', 'jpeg', 'png', 'gif', 'webp']` | Allowed file extensions |
| `enableOptimization` | boolean | `true` | Enable image optimization |
| `jpegQuality` | number | `85` | JPEG compression quality (1-100) |
| `pngCompression` | number | `8` | PNG compression level (0-9) |
| `webpQuality` | number | `85` | WebP quality (1-100) |

## Security Features

### File Validation

- **Magic Number Verification**: Validates file headers against MIME types
- **Malicious Content Detection**: Scans for script injections and suspicious patterns
- **Path Traversal Prevention**: Blocks directory traversal attempts
- **Filename Sanitization**: Removes dangerous characters and patterns
- **Size Limits**: Enforces file size restrictions

### Security Patterns Detected

```typescript
// Automatically detects and blocks:
const maliciousPatterns = [
  /<script[^>]*>/i,           // Script tags
  /javascript:/i,             // JavaScript protocols
  /vbscript:/i,              // VBScript protocols
  /<\?php/i,                 // PHP tags
  /<!--#exec/i,              // Server-side includes
  /\.(exe|bat|cmd|scr)$/i    // Executable extensions
];
```

## Advanced Usage

### Custom Validation

```typescript
const customValidator = async (file: Express.Multer.File) => {
  // Custom validation logic
  if (file.originalname.includes('temp')) {
    return {
      isValid: false,
      errors: ['Temporary files not allowed'],
      mimeType: file.mimetype,
      extension: path.extname(file.originalname),
      size: file.size
    };
  }
  
  return { isValid: true, errors: [] };
};

const middleware = new ExpressUploadMiddleware({
  customValidator
});
```

### Storage Management

```typescript
// Get storage information
const storage = await handler.getStorageInfo('user123');
console.log(`Used: ${storage.totalUsedFormatted}`);
console.log(`Usage: ${storage.usagePercentage}%`);

// Clean up old files
const cleanup = await handler.cleanupFiles({
  olderThanDays: 30,
  dryRun: false
});
console.log(`Deleted ${cleanup.deletedCount} files`);
console.log(`Freed ${cleanup.freedSpaceFormatted}`);
```

### Thumbnail Generation

```typescript
// Generate multiple thumbnail sizes
const thumbnails = await handler.generateThumbnails('image.jpg', [
  { width: 64, height: 64, suffix: 'tiny' },
  { width: 150, height: 150, suffix: 'thumb' },
  { width: 300, height: 300, suffix: 'small' },
  { width: 800, height: 600, suffix: 'medium' }
]);

thumbnails.forEach(thumb => {
  console.log(`${thumb.width}x${thumb.height}: ${thumb.path}`);
});
```

### WebP Conversion

```typescript
// Convert to WebP for better compression
const webpResult = await handler.convertToWebP('image.jpg', 80);

if (webpResult.success) {
  console.log(`Savings: ${webpResult.savings.toFixed(1)}%`);
  console.log(`WebP file: ${webpResult.webpFilename}`);
}
```

## Error Handling

```typescript
import { 
  UploadError, 
  ValidationError, 
  FileSizeError, 
  FileTypeError, 
  StorageError 
} from '@mvp-factory/holy-upload';

try {
  const result = await handler.uploadSingle(file);
} catch (error) {
  if (error instanceof FileSizeError) {
    console.log('File too large');
  } else if (error instanceof FileTypeError) {
    console.log('Invalid file type');
  } else if (error instanceof ValidationError) {
    console.log('Validation failed:', error.message);
  } else if (error instanceof StorageError) {
    console.log('Storage issue:', error.message);
  }
}
```

## Health Monitoring

```typescript
// Check system health
const health = await handler.healthCheck();

if (!health.healthy) {
  console.log('Issues detected:', health.issues);
}

console.log('Storage usage:', health.storageInfo.usagePercentage + '%');
```

## Examples

### Complete Upload API

```typescript
import express from 'express';
import { HolyUpload } from '@mvp-factory/holy-upload';

const app = express();

// Configure upload middleware
const uploadMiddleware = HolyUpload.createMiddleware({
  uploadDir: process.env.UPLOAD_DIR || './uploads',
  maxSize: parseInt(process.env.MAX_FILE_SIZE || '10485760'),
  allowedMimeTypes: ['image/jpeg', 'image/png', 'image/webp'],
  enableOptimization: true
});

// Single image upload
app.post('/api/upload/image', 
  uploadMiddleware.single('image'),
  async (req, res) => {
    try {
      const result = await req.uploadSingle!();
      res.json(result);
    } catch (error) {
      res.status(400).json({ 
        success: false, 
        error: error.message 
      });
    }
  }
);

// Multiple images upload
app.post('/api/upload/gallery',
  uploadMiddleware.multiple('images', 10),
  async (req, res) => {
    try {
      const result = await req.uploadMultiple!();
      res.json(result);
    } catch (error) {
      res.status(400).json({ 
        success: false, 
        error: error.message 
      });
    }
  }
);

// Delete file
app.delete('/api/upload/:filename',
  uploadMiddleware.createDeleteEndpoint()
);

// Get storage info
app.get('/api/upload/storage',
  uploadMiddleware.createStorageInfoEndpoint()
);

// Serve files
app.use('/uploads', 
  uploadMiddleware.createFileServing('/uploads'),
  express.static('./uploads')
);

app.listen(3000, () => {
  console.log('Upload API running on port 3000');
});
```

### Frontend Integration

```javascript
// HTML form
<form id="uploadForm" enctype="multipart/form-data">
  <input type="file" name="image" accept="image/*" required>
  <button type="submit">Upload</button>
</form>

// JavaScript upload
document.getElementById('uploadForm').addEventListener('submit', async (e) => {
  e.preventDefault();
  
  const formData = new FormData(e.target);
  
  try {
    const response = await fetch('/api/upload/image', {
      method: 'POST',
      body: formData
    });
    
    const result = await response.json();
    
    if (result.success) {
      console.log('Upload successful:', result.url);
      // Display uploaded image
      const img = document.createElement('img');
      img.src = result.url;
      document.body.appendChild(img);
    } else {
      console.error('Upload failed:', result.error);
    }
  } catch (error) {
    console.error('Upload error:', error);
  }
});
```

## Development vs Production

### Development Configuration

```typescript
const devConfig = HolyUpload.createDevConfig();
// Uses:
// - ./uploads/dev directory
// - 5MB max file size
// - Optimization disabled for faster uploads
```

### Production Configuration

```typescript
const prodConfig = HolyUpload.createProdConfig();
// Uses:
// - /tmp/uploads directory (cloud-friendly)
// - 10MB max file size
// - Optimization enabled
// - Lower quality settings for smaller files
```

## Testing

```bash
# Run tests
npm test

# Run with coverage
npm run test:coverage
```

## Contributing

1. Fork the repository
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add some amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request

## License

MIT © MVP Factory

## Support

- 📧 Email: support@mvp-factory.dev
- 🐛 Issues: [GitHub Issues](https://github.com/mvp-factory/modules/issues)
- 📖 Documentation: [Full Documentation](https://docs.mvp-factory.dev/modules/holy-upload)

---

**Extracted from Holy Habit project** - Battle-tested upload system used in production with advanced security and optimization features.