qapinterface
Version:
Comprehensive API utilities for Node.js applications including authentication, security, request processing, and response handling with zero external dependencies
522 lines (404 loc) • 15.8 kB
Markdown
# QAPI - Node.js API Utilities
A comprehensive, enterprise-grade npm module providing essential utilities for building robust, secure, and production-ready API applications and services.
## Features
- **🔐 Authentication**: API key validation, extraction, and secure middleware
- **⚡ Request Processing**: ID generation, timing, validation, and intelligent rate limiting
- **🛡️ Security**: Comprehensive headers, monitoring, threat detection, and pattern analysis
- **📡 HTTP Utilities**: Standardized response handling, validation, and error management
- **🎯 Service Validation**: Health monitoring, async validation, and service integration
- **🔗 URL Processing**: Security-first parsing, validation, and construction
- **📊 Logging**: Lightweight console logging with timestamps and source tracking
- **⚙️ Lightweight**: Efficient implementation with essential dependencies only
- **Service Validation**: Async operations, retry logic, and health monitoring
- **URL Processing**: Secure parsing, proxy validation, and domain checking
## Installation
```bash
npm install qapi
```
## Requirements
- Node.js ≥ 18.0.0
- Minimal external dependencies for production use
## Quick Start
```javascript
const {
// Core utilities
validateEmail,
generateId,
validateApiPath,
sanitizeApiData,
formatApiVersion,
// HTTP response utilities
sendJsonResponse,
createSuccessResponse,
createErrorResponse,
throwIfResNotOk,
// API request utilities
apiRequest,
apiGet,
apiPost,
apiPut,
apiDelete,
apiPatch,
// Console logging
log,
logError,
logWarn,
logInfo,
logDebug,
// API authentication
extractApiKey,
createApiKeyValidationMiddleware,
validateApiKeyFormat,
// Request processing
generateRequestId,
validateRequestStructure,
createRequestProcessingMiddleware,
// Security utilities
createSecurityHeadersMiddleware,
generateSecureNonce,
analyzeSecurityPatterns,
// Service validation
validateServiceAsync,
checkServiceHealth
} = require('qapi');
// API response with standardized format and logging
app.get('/api/users', (req, res) => {
const users = [{ id: 1, name: 'John' }];
const response = createSuccessResponse(users, {
requestId: generateRequestId(req),
processingTime: 45,
metadata: { version: formatApiVersion('1.0'), total: users.length }
});
log('API_USERS', `Retrieved ${users.length} users`);
sendJsonResponse(res, 200, response);
});
// Making API calls with built-in validation
async function fetchUserData(userId) {
try {
const userData = await apiGet(`https://api.example.com/users/${userId}`);
log('FETCH_USER', `Successfully retrieved user ${userId}`);
return userData;
} catch (error) {
logError('FETCH_USER', `Failed to fetch user ${userId}: ${error.message}`);
throw error;
}
}
// API path validation and data sanitization
app.get('/api/users/:id', (req, res) => {
const pathValidation = validateApiPath(req.path);
if (!pathValidation.valid) {
return sendJsonResponse(res, 400, createErrorResponse('Invalid API path'));
}
const userData = { id: 1, name: 'John', password: 'secret', token: 'abc123' };
const sanitizedData = sanitizeApiData(userData);
sendJsonResponse(res, 200, createSuccessResponse(sanitizedData));
});
// Comprehensive security and request processing
const securityMiddleware = createSecurityHeadersMiddleware();
const requestMiddleware = createRequestProcessingMiddleware();
const apiKeyMiddleware = createApiKeyValidationMiddleware({
validApiKey: process.env.API_KEY,
logAttempts: true
});
app.use('/api', securityMiddleware, requestMiddleware);
app.use('/api/protected', apiKeyMiddleware);
```
## Core API Utilities
### validateApiPath(path, options)
Validates API endpoint paths with security checks and normalization.
```javascript
const result = validateApiPath('/api/v1/users/123');
// Returns: { valid: true, normalized: '/api/v1/users/123', issues: [] }
const result = validateApiPath('/api/../admin');
// Returns: { valid: false, issues: ['Path traversal not allowed'] }
```
### sanitizeApiData(data, sensitiveFields)
Removes sensitive information from API response data.
```javascript
const userData = {
id: 1,
name: 'John',
password: 'secret',
apiKey: 'key123'
};
const clean = sanitizeApiData(userData);
// Returns: { id: 1, name: 'John' }
// Custom sensitive fields
const clean = sanitizeApiData(userData, ['name', 'id']);
// Returns: { password: 'secret', apiKey: 'key123' }
```
### formatApiVersion(version, prefix)
Formats API version strings for headers and URLs.
```javascript
formatApiVersion('1.0'); // Returns: 'v1.0'
formatApiVersion(2, 'api-v'); // Returns: 'api-v2'
formatApiVersion('v1.2.3'); // Returns: 'v1.2.3'
```
### validateEmail(email)
Validates email addresses with comprehensive format checking.
```javascript
validateEmail('user@example.com'); // Returns: true
validateEmail('invalid-email'); // Returns: false
```
### generateId(prefix, length)
Generates unique identifiers for tracking and correlation.
```javascript
generateId(); // Returns: random ID
generateId('user', 8); // Returns: 'user_12345678'
```
## HTTP Response Utilities
### createSuccessResponse(data, options)
Creates standardized success response objects.
```javascript
const response = createSuccessResponse(
{ users: [...] },
{
requestId: 'req_123',
processingTime: 45,
metadata: { total: 10, page: 1 }
}
);
```
### createErrorResponse(message, options)
Creates standardized error response objects.
```javascript
const error = createErrorResponse(
'Validation failed',
{
statusCode: 400,
issues: [
{ field: 'email', message: 'Invalid format' }
]
}
);
```
### throwIfResNotOk(response, context)
Validates Fetch API responses and throws descriptive errors for failed requests.
```javascript
const response = await fetch('/api/users');
await throwIfResNotOk(response, 'fetchUsers');
// Throws error with status, statusText, and context if response is not ok
```
### sendJsonResponse(res, statusCode, data)
Safely sends JSON responses with proper error handling and headers.
```javascript
sendJsonResponse(res, 200, { success: true, data: users });
sendJsonResponse(res, 404, createErrorResponse('User not found'));
```
## HTTP Request Utilities
### apiGet(url, options), apiPost(url, data, options), etc.
Standardized HTTP methods with automatic response validation and error handling.
```javascript
// GET request with automatic validation
const users = await apiGet('https://api.example.com/users');
// POST request with authentication
const newUser = await apiPost('https://api.example.com/users',
{ name: 'John', email: 'john@example.com' },
{ authCookie: 'session_token_123' }
);
// Custom headers and options
const data = await apiRequest('https://api.example.com/data', {
method: 'PATCH',
data: { status: 'active' },
headers: { 'Custom-Header': 'value' }
});
```
## Console Logging Utilities
### log(source, message), logError(source, message), etc.
Lightweight console logging with formatted timestamps and source tracking.
```javascript
log('API_SERVER', 'Server started on port 3000');
// [2024-08-10 14:30:15] [API_SERVER] Server started on port 3000
logError('DB_CONNECTION', 'Failed to connect to database');
// [2024-08-10 14:30:15] [ERROR] [DB_CONNECTION] Failed to connect to database
logWarn('CACHE', 'Cache hit ratio below threshold');
logInfo('USER_AUTH', 'User authentication successful');
logDebug('QUERY_PERF', 'Query executed in 25ms');
```
### createSourceLogger(source)
Creates a logger bound to a specific source for consistent messaging.
```javascript
const dbLogger = createSourceLogger('DATABASE');
dbLogger.info('Connection established');
dbLogger.error('Query failed');
```
## Security Features
### createSecurityHeadersMiddleware(options)
Comprehensive security headers middleware with CSP, HSTS, and threat protection.
```javascript
const securityMiddleware = createSecurityHeadersMiddleware({
includeCSP: true,
includeHSTS: true,
cspOptions: {
directives: {
'script-src': ["'self'", (req, res) => `'nonce-${res.locals.nonce}'`],
'style-src': ["'self'", "'unsafe-inline'"]
}
}
});
```
### analyzeSecurityPatterns(requestData)
Advanced threat detection for SQL injection, XSS, path traversal, and suspicious patterns.
```javascript
const analysis = analyzeSecurityPatterns({
query: req.query,
body: req.body,
headers: req.headers,
userAgent: req.get('User-Agent')
});
if (analysis.threats.length > 0) {
logSecurityEvent('THREAT_DETECTED', analysis);
return res.status(403).json(createErrorResponse('Request blocked'));
}
```
### generateSecureNonce(length)
Generates cryptographically secure nonces for CSP headers.
```javascript
const nonce = generateSecureNonce(16);
// Returns: cryptographically secure random string
```
### API Key Validation
Secure API key extraction and validation with timing-safe comparison.
```javascript
const apiKeyMiddleware = createApiKeyValidationMiddleware({
validApiKey: process.env.API_KEY,
logAttempts: true,
sources: ['header', 'query', 'body'] // Multiple extraction sources
});
// Manual validation
const isValid = validateApiKeyFormat(extractedKey);
```
### analyzeSecurityPatterns(input, context)
Detects security threats including SQL injection, XSS, and path traversal.
```javascript
const analysis = analyzeSecurityPatterns("'; DROP TABLE users; --");
// Returns: { threats: ['SQL_INJECTION'], severity: 'HIGH', patterns: [...] }
```
## Request Processing
### generateRequestId(req, prefix)
Generates unique request IDs with upstream service correlation.
```javascript
const requestId = generateRequestId(req, 'api');
// Returns: 'api_1640995200000_abc123'
```
### validateRequestStructure(req, options)
Comprehensive request validation with payload limits and security checks.
```javascript
const validation = validateRequestStructure(req, {
maxPayloadSize: 1048576, // 1MB
allowedContentTypes: ['application/json'],
requireUserAgent: true
});
```
## Service Validation
### validateServiceAsync(serviceName, validationFn, fallback, options)
Standardized async service validation with timeout and error handling.
```javascript
const result = await validateServiceAsync(
'Database',
async () => await db.ping(),
{ connected: false },
{ timeout: 5000, logErrors: true }
);
```
### checkServiceHealth(services, options)
Comprehensive health monitoring for multiple services.
```javascript
const health = await checkServiceHealth([
{ name: 'Database', validator: () => db.ping() },
{ name: 'Redis', validator: () => redis.ping(), critical: true }
]);
```
## URL Processing Utilities
### parseUrl(url), validateUrl(url), constructUrl(parts)
Security-first URL parsing, validation, and construction with comprehensive checks.
```javascript
const parsed = parseUrl('https://api.example.com/v1/users?limit=10');
// Returns: { protocol: 'https:', hostname: 'api.example.com', pathname: '/v1/users', ... }
const isValid = validateUrl('https://trusted-api.com/endpoint');
// Returns: true if URL passes security validation
const url = constructUrl({
protocol: 'https:',
hostname: 'api.example.com',
pathname: '/users',
query: { page: 1, limit: 10 }
});
```
## Advanced Features
### Rate Limiting
Memory-efficient sliding window rate limiting with configurable policies.
### Service Health Monitoring
Comprehensive health checks for multiple services with timeout handling.
### Request Context Extraction
Automatic extraction of correlation IDs, timing data, and request metadata.
### Retry Logic
Exponential backoff retry mechanisms with circuit breaker patterns.
## TypeScript Support
While written in JavaScript, the module includes comprehensive JSDoc comments that provide full IntelliSense support in TypeScript and modern IDEs. All functions include detailed type information and usage examples.
## Testing
This project uses the **qtests** framework for comprehensive testing coverage. Unit tests are located next to the modules they validate under `lib/**/__tests__`, while integration tests live in the top-level `tests` directory. Run all tests with:
```bash
npm test
```
The command executes `test-runner.js`, which discovers every `.test.js` file in both directories and runs them together. The test suite covers:
- ✅ All utility functions with edge cases and error conditions
- ✅ HTTP request/response validation and error handling
- ✅ Security threat detection patterns and false positive prevention
- ✅ Request processing workflows and middleware integration
- ✅ Service validation scenarios with timeout and retry logic
- ✅ Console logging functionality and source binding
- ✅ API key validation and timing-safe comparisons
- ✅ URL processing security and validation rules
- ✅ Integration between modules and real-world usage patterns
## Architecture & Design
### Lightweight Architecture
QAPI is built with essential dependencies for production APIs:
- **Minimal footprint**: Carefully selected dependencies for core functionality
- **Security**: Battle-tested packages with active maintenance
- **Compatibility**: Works across different Node.js environments
- **Reliability**: Stable dependency versions with consistent behavior
### Single Responsibility Principle
Each module focuses on a specific concern with clear boundaries:
- **Validation modules**: Input validation and sanitization
- **HTTP modules**: Request/response handling and API communication
- **Security modules**: Threat detection and protection
- **Utility modules**: Core functionality and helpers
### Production-Ready Design
- **Comprehensive error handling**: Graceful failure modes
- **Security-first approach**: Built-in threat protection
- **Performance optimized**: Efficient algorithms and memory usage
- **Monitoring ready**: Extensive logging and health checks
## Contributing
We welcome contributions! Please ensure all new features include comprehensive tests and follow the established patterns:
- Keep modules focused on single responsibilities
- Include JSDoc comments for all functions
- Add unit tests with edge cases
- Update documentation and examples
## License
MIT License - see LICENSE file for details
## Support
- 📖 [Documentation](https://github.com/api-utilities/qapi)
- 🐛 [Issue Tracker](https://github.com/api-utilities/qapi/issues)
- 💬 [Discussions](https://github.com/api-utilities/qapi/discussions)
## Changelog
See the [Recent Changes](./replit.md#recent-changes) section in replit.md for detailed version history and feature additions.
## Security Features
- **Timing attack prevention** using cryptographically secure comparisons
- **Real-time threat detection** for SQL injection, XSS, NoSQL injection, and path traversal
- **Comprehensive security headers** with CSP, HSTS, and content protection
- **Request validation** with payload limits and header injection prevention
- **Rate limiting** with sliding window algorithm and automatic cleanup
## Performance
- **High-precision timing** using Node.js monotonic clock for accurate performance measurement
- **Memory-efficient rate limiting** with automatic cleanup and statistics tracking
- **Optimized pattern matching** for security threat detection
- **Efficient operations** for request processing and data handling
## License
MIT
## Contributing
This module follows strict coding standards:
- Minimal external dependencies for essential functionality
- Comprehensive error handling and logging
- Security-first design principles
- Performance optimization for high-traffic APIs
- Extensive documentation and examples