facezk-core/README.md

ZKP-AI Proof of Humanity: Live Selfie Check with Biometric Template Extraction and Fuzzy Hashing

# FaceZK v1.1.0 - Privacy-First Human Verification

[![npm version](https://badge.fury.io/js/facezk-core.svg)](https://badge.fury.io/js/facezk-core)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![TypeScript](https://img.shields.io/badge/TypeScript-5.7-blue.svg)](https://www.typescriptlang.org/)
[![Security](https://img.shields.io/badge/Security-Audited-green.svg)](https://facezk.com/security)

**Zero-Knowledge Proof of Humanity with Advanced Liveness Detection and Expression State Management**

FaceZK v1.1.0 is a complete rewrite focusing on **privacy-first design**, **modular architecture**, and **enterprise-grade security**. Built with modern cryptographic standards, comprehensive audit logging, and advanced liveness detection with sequential challenge completion.

## 🏢 About Us

### **Our Journey: From VerifiedCreators to VerifiedOnchain**

FaceZK is the culmination of years of research and development, starting with **VerifiedCreators** in 2021 during the Web3 creator economy boom. Our journey began with a simple yet powerful vision: to build trust in the digital world through advanced identity verification.

#### **VerifiedCreators (2021)**
- **Mission**: Public creator profiles with onchain primitives to instill trust and attract investors
- **Innovation**: Non-KYC identity verification technology
- **Impact**: Helped creators build credibility in the emerging Web3 ecosystem
- **Foundation**: Established the core principles of privacy-first verification

#### **VerifiedOnchain Protocol (2024+)**
- **Vision**: The most advanced ONCHAIN and Non-KYC identity verification protocol
- **Technology**: AI-powered verification with Zero-Knowledge Proofs (ZKP)
- **Research**: Backed by comprehensive academic research and real-world testing
- **Goal**: Democratize trust in the decentralized web

### **FaceZK: The Technical Foundation**

FaceZK is **1/3 of the technical steps** toward launching the full VerifiedOnchain protocol. This library represents our commitment to:

- **Privacy-First Design**: Zero-knowledge verification without compromising security
- **Academic Rigor**: Research-backed implementation with peer-reviewed methodologies
- **Open Source**: Transparent, auditable, and community-driven development
- **Enterprise Ready**: Production-grade security and performance

### **Our Vision**

We believe that **trust should be decentralized, privacy should be guaranteed, and verification should be seamless**. In a world where digital identity is increasingly important, we're building the infrastructure that will power the next generation of trusted interactions.

### **Contact & Support**

- **Email**: [support@verifiedcreators.io](mailto:support@verifiedcreators.io)
- **Research**: Academic thesis available for deeper technical understanding
- **Protocol Documentation**: Coming soon (2/3 technical milestone)

---

## 🚀 What's New in v1.1.0

### 🔐 **Security & Privacy Improvements**
- **True Zero-Knowledge**: No raw biometric data storage
- **AES-GCM Encryption**: Military-grade encryption for all sensitive data
- **Secure Key Derivation**: PBKDF2 with 100,000+ iterations
- **Random Salt Generation**: Cryptographically secure random salts
- **Audit Trail**: Complete audit logging for compliance
- **Memory Protection**: Automatic cleanup and monitoring

### 🎯 **Advanced Liveness Detection**
- **Sequential Challenge System**: Enforces correct order: Blink → Head Turn → Smile
- **Expression State Management**: Stable expression persistence across frames
- **Debounced Detection**: Prevents overwriting of expression data
- **Real-time Feedback**: Visual indicators for current challenge step
- **Confidence Tracking**: Per-expression confidence scoring

### 🏗️ **Modular Architecture**
- **Plugin System**: Extensible plugin architecture
- **Dependency Management**: Automatic plugin dependency resolution
- **Hot Reloading**: Plugin updates without restart
- **Type Safety**: Full TypeScript support for all plugins
- **Error Recovery**: Graceful error handling and recovery

### 📊 **Enterprise Features**
- **Performance Monitoring**: Real-time performance metrics
- **Memory Management**: Automatic TensorFlow.js cleanup
- **Audit Logging**: Complete audit trail with export
- **Configuration Validation**: Comprehensive config validation
- **Migration Tools**: Easy migration from v1.x

## 🛡️ Security Features

### **Privacy-Preserving Design**
```typescript
// No raw biometric data storage
const template = await cryptoPlugin.encryptTemplate(biometricData, systemSalt);
const humanityCode = await cryptoPlugin.generateHumanityCode(biometricId, systemSalt);
```

### **Cryptographic Implementation**
- **AES-GCM**: Authenticated encryption for templates
- **PBKDF2**: Secure key derivation (100,000+ iterations)
- **SHA-256**: Secure hashing for humanity codes
- **Random IVs**: Unique initialization vectors for each encryption
- **Salt Generation**: Cryptographically secure random salts

### **Audit & Compliance**
- **Complete Audit Trail**: All operations logged
- **GDPR Compliance**: Data protection by design
- **SOC 2 Ready**: Security controls and monitoring
- **Export Capabilities**: Audit log export for compliance

## 🎯 Liveness Detection Features

### **Sequential Challenge System**
```typescript
// Enforces correct order of completion
const challenges = ['blink', 'head-turn', 'smile'];
const currentStep = session.expressionState?.currentStep; // 'blink' | 'head-turn' | 'smile' | 'completed'
```

### **Expression State Management**
```typescript
// Stable expression persistence across frames
const expressionState = {
  stableExpressions: {
    blink: { detected: boolean; confidence: number; timestamp: number },
    smile: { detected: boolean; confidence: number; timestamp: number },
    headTurn: { detected: boolean; confidence: number; timestamp: number }
  },
  currentStep: 'blink',
  frameCount: 15,
  debounceTimer: NodeJS.Timeout
};
```

### **Real-time Visual Feedback**
- **Current Step Highlighting**: Blue border with pulsing animation
- **Completed Steps**: Green checkmark with confidence scores
- **Waiting Steps**: Grayed out with "⏳ Waiting..." status
- **Progress Tracking**: Real-time confidence updates

## 🏗️ Architecture Overview

### **Core Components**
```
FaceZK v1.1.0
├── Core (FaceZKCore)
│   ├── Plugin Manager
│   ├── Session Management
│   ├── Expression State Management
│   └── Verification Engine
├── Face Detection (Human.js Integration)
│   ├── Face Mesh Extraction
│   ├── Landmark Detection
│   ├── Emotion Analysis
│   └── Anti-spoofing
├── Liveness Detection
│   ├── Sequential Challenge System
│   ├── Expression State Persistence
│   ├── Frame Analysis
│   └── Confidence Scoring
├── Biometric Processing
│   ├── Template Generation
│   ├── Fuzzy Hashing
│   └── Descriptor Extraction
└── Plugins
    ├── Crypto Plugin (AES-GCM, PBKDF2)
    ├── Memory Plugin (TensorFlow.js management)
    ├── Audit Plugin (Compliance logging)
    └── Performance Plugin (Metrics & monitoring)
```

### **Plugin System**
```typescript
interface IPlugin {
  readonly id: string;
  readonly name: string;
  readonly version: string;
  readonly state: PluginState;
  
  initialize(config: any): Promise<void>;
  dispose(): Promise<void>;
  isReady(): boolean;
}
```

## 📦 Installation

```bash
npm install facezk-core@^1.1.0
```

### **Peer Dependencies**
```bash
npm install @tensorflow/tfjs@^4.22.0
npm install @vladmandic/human@^1.3.0
npm install react@^18.0.0 react-dom@^18.0.0  # For React integration
```

## 🎯 Quick Start

### **Basic Usage**
```typescript
import { FaceZK, generateSecureSalt } from 'facezk-core';

// Create with secure configuration
const facezk = await FaceZK.create({
  systemSalt: generateSecureSalt(), // Cryptographically secure
  enableAudit: true,
  maxMemoryMB: 512,
  enableMemoryMonitoring: true,
  minConfidence: 0.8,
  minLivenessScore: 0.7,
});

// Start verification session
const session = facezk.startSession();

// Process video frame
const video = document.getElementById('video');
const result = await facezk.processFrame(video);

if (result?.verified) {
  console.log('Humanity Code:', result.humanityCode);
  console.log('Biometric ID:', result.biometricId);
  console.log('Liveness Score:', result.livenessScore);
  console.log('Completed Challenges:', result.metadata.challengesCompleted);
}

// Cleanup
await facezk.dispose();
```

### **Advanced Configuration**
```typescript
import { FaceZK, DEFAULT_CONFIG } from 'facezk-core';

const config = {
  ...DEFAULT_CONFIG,
  systemSalt: 'your-secure-system-salt-2024',
  enableAudit: true,
  auditRetentionDays: 365,
  maxMemoryMB: 1024,
  enableMemoryMonitoring: true,
  cryptoConfig: {
    algorithm: 'AES-GCM',
    iterations: 200000, // Higher security
    keyLength: 256,
  },
  auditConfig: {
    externalEndpoint: 'https://your-audit-service.com/logs',
    enableCompression: true,
    enableEncryption: true,
  },
  // Liveness detection configuration
  livenessConfig: {
    expressionDebounceMs: 500,
    minStableFrames: 5,
    sequentialChallenges: true,
  },
};

const facezk = await FaceZK.create(config);
```

### **Plugin Management**
```typescript
import { FaceZK, createCryptoPlugin, createMemoryPlugin } from 'facezk-core';

// Create custom plugins
const cryptoPlugin = createCryptoPlugin({
  algorithm: 'AES-GCM',
  iterations: 150000,
});

const memoryPlugin = createMemoryPlugin({
  maxMemoryMB: 1024,
  enableProfiling: true,
});

// Use with FaceZK
const facezk = await FaceZK.create({
  systemSalt: generateSecureSalt(),
  cryptoConfig: cryptoPlugin.getConfig(),
  memoryConfig: memoryPlugin.getConfig(),
});
```

## 🔧 API Reference

### **FaceZK Class**

#### **Factory Method**
```typescript
static async create(config?: FaceZKConfig): Promise<FaceZK>
```

#### **Core Methods**
```typescript
// Session management
startSession(): VerificationSession
getSession(): VerificationSession | null
getCurrentSession(): VerificationSession | null
resetSession(): void

// Frame processing
async processFrame(input: Input, options?: VerificationOptions): Promise<VerificationResult | null>

// Expression state management
getExpressionState(): ExpressionState | null
getCurrentStep(): 'blink' | 'head-turn' | 'smile' | 'completed'

// System information
isReady(): boolean
getConfig(): FaceZKConfig
getSystemStats(): SystemStats
getPluginInfo(pluginId: string): PluginInfo | null
listPlugins(): PluginInfo[]

// Memory management
getMemoryStats(): MemoryStats
async cleanupMemory(): Promise<void>

// Audit logging
async getAuditTrail(sessionId: string): Promise<AuditEvent[]>
async exportAuditLog(startDate: Date, endDate: Date): Promise<AuditLogExport>

// Legacy methods (for v1.x compatibility)
async generateHumanityCode(biometricId: string, additionalData?: string): Promise<string>
async encryptTemplate(templateData: Uint8Array, userSalt?: string): Promise<EncryptedData>
async decryptTemplate(encryptedData: EncryptedData): Promise<Uint8Array>

// Cleanup
async dispose(): Promise<void>
```

### **VerificationSession Interface**
```typescript
interface VerificationSession {
  id: string;
  startTime: Date;
  state: 'initializing' | 'detecting' | 'verifying' | 'completed' | 'failed';
  progress: number;
  result?: VerificationResult;
  error?: string;
  expressionState?: {
    lastStableFrame: number;
    stableExpressions: {
      blink: { detected: boolean; confidence: number; timestamp: number };
      smile: { detected: boolean; confidence: number; timestamp: number };
      headTurn: { detected: boolean; confidence: number; timestamp: number };
    };
    frameCount: number;
    debounceTimer?: NodeJS.Timeout;
    currentStep: 'blink' | 'head-turn' | 'smile' | 'completed';
    stepStartTime: number;
    stepCompletionTime?: number;
  };
}
```

### **VerificationResult Interface**
```typescript
interface VerificationResult {
  verified: boolean;
  humanityCode: string;
  biometricId: string;
  livenessScore: number;
  confidence: number;
  timestamp: Date;
  sessionId: string;
  metadata: {
    processingTime: number;
    frameCount: number;
    challengesCompleted: string[];
    stableExpressions: {
      blink: { detected: boolean; confidence: number };
      smile: { detected: boolean; confidence: number };
      headTurn: { detected: boolean; confidence: number };
    };
    boundingBox?: [number, number, number, number];
    progress: number;
  };
}
```

## 🎮 Demo & Testing

### **Interactive Demo**
```bash
npm run demo
```

The demo includes:
- **Real-time Face Detection**: Live video processing with Human.js
- **Sequential Liveness Challenges**: Step-by-step expression detection
- **Visual Feedback**: Current step highlighting and progress tracking
- **Expression State Display**: Real-time confidence scores
- **Verification Results**: Humanity codes and biometric IDs

### **Demo Features**
- **Step-by-step Guidance**: Clear instructions for each challenge
- **Visual Indicators**: Color-coded challenge cards
- **Progress Tracking**: Real-time confidence updates
- **Error Handling**: Graceful fallbacks for detection failures
- **Performance Monitoring**: Memory usage and processing time

### **Testing**
```bash
# Unit tests
npm test

# Integration tests
npm run test:integration

# Security tests
npm run test:security

# Demo server
npm run demo
```

## 🔌 Creating Custom Plugins

### **Plugin Interface**
```typescript
interface IPlugin {
  readonly id: string;
  readonly name: string;
  readonly version: string;
  readonly state: PluginState;
  
  initialize(config: any): Promise<void>;
  dispose(): Promise<void>;
  isReady(): boolean;
}
```

### **Custom Plugin Example**
```typescript
import { IPlugin, PluginState } from 'facezk-core';

class CustomFaceDetectionPlugin implements IPlugin {
  public readonly id = 'custom-face-detector';
  public readonly name = 'Custom Face Detector';
  public readonly version = '1.0.0';
  public state: PluginState = 'disposed';

  async initialize(config: any): Promise<void> {
    // Initialize your custom face detection logic
    this.state = 'ready';
  }

  async dispose(): Promise<void> {
    // Cleanup resources
    this.state = 'disposed';
  }

  isReady(): boolean {
    return this.state === 'ready';
  }

  async detect(input: Input): Promise<FaceDetectionResult | null> {
    // Your custom detection logic
    return null;
  }
}

// Register and use
const plugin = new CustomFaceDetectionPlugin();
pluginManager.registry.register(plugin);
```

## 📊 Monitoring & Debugging

### **System Statistics**
```typescript
const stats = facezk.getSystemStats();
console.log('Memory Usage:', stats.memory);
console.log('Plugin Status:', stats.plugins);
console.log('Performance Metrics:', stats.performance);
```

### **Expression State Monitoring**
```typescript
const session = facezk.getCurrentSession();
const expressionState = session?.expressionState;

if (expressionState) {
  console.log('Current Step:', expressionState.currentStep);
  console.log('Frame Count:', expressionState.frameCount);
  console.log('Stable Expressions:', expressionState.stableExpressions);
}
```

### **Audit Trail**
```typescript
const auditTrail = await facezk.getAuditTrail(sessionId);
console.log('Session Events:', auditTrail);

const auditExport = await facezk.exportAuditLog(
  new Date('2024-01-01'),
  new Date('2024-12-31')
);
console.log('Audit Export:', auditExport);
```

### **Plugin Monitoring**
```typescript
const plugins = facezk.listPlugins();
plugins.forEach(plugin => {
  console.log(`${plugin.name}: ${plugin.state}`);
});

const cryptoInfo = facezk.getPluginInfo('crypto-plugin');
console.log('Crypto Plugin:', cryptoInfo);
```

## 🔒 Security Best Practices

### **Configuration Security**
```typescript
// ✅ Good: Use secure random salt
const config = {
  systemSalt: generateSecureSalt(),
  enableAudit: true,
  auditRetentionDays: 90,
};

// ❌ Bad: Hardcoded salt
const config = {
  systemSalt: 'my-secret-salt',
  enableAudit: false,
};
```

### **Memory Management**
```typescript
// ✅ Good: Enable memory monitoring
const config = {
  maxMemoryMB: 512,
  enableMemoryMonitoring: true,
};

// ✅ Good: Regular cleanup
await facezk.cleanupMemory();
```

### **Audit Logging**
```typescript
// ✅ Good: Enable comprehensive audit logging
const config = {
  enableAudit: true,
  auditRetentionDays: 365,
  auditConfig: {
    logLevel: 'security',
    enableEncryption: true,
  },
};
```

## 🚀 Performance Optimization

### **Memory Optimization**
```typescript
const config = {
  maxMemoryMB: 1024,
  enableMemoryMonitoring: true,
  memoryConfig: {
    cleanupInterval: 30000, // 30 seconds
    enableProfiling: true,
  },
};
```

### **Backend Selection**
```typescript
const config = {
  backend: 'webgl', // Best performance for most cases
  // backend: 'wasm', // Good for CPU-only environments
  // backend: 'cpu', // Fallback option
};
```

### **Liveness Detection Optimization**
```typescript
const config = {
  livenessConfig: {
    expressionDebounceMs: 300, // Faster response
    minStableFrames: 3, // Lower threshold for faster completion
    sequentialChallenges: true,
  },
};
```

## 🔄 Migration from v1.x

### **Automatic Migration**
```typescript
import { migrateFromV1 } from 'facezk-core';

const v1Config = {
  backend: 'webgl',
  minConfidence: 0.8,
  challenges: ['blink', 'head-turn'],
};

const v2Config = migrateFromV1(v1Config);
const facezk = await FaceZK.create(v2Config);
```

### **Manual Migration**
```typescript
// v1.x initialization
const facezk = new FaceZK(config);
await facezk.initialize();

// v2.0 initialization
const facezk = await FaceZK.create(config);
```

## 📚 Examples

### **React Integration**
```typescript
import { useFaceZK } from 'facezk-core/react';

function FaceVerification() {
  const { facezk, isReady, processFrame } = useFaceZK({
    systemSalt: generateSecureSalt(),
    enableAudit: true,
  });

  const handleVideoFrame = async (video: HTMLVideoElement) => {
    const result = await processFrame(video);
    if (result?.verified) {
      console.log('Verification successful!');
      console.log('Completed challenges:', result.metadata.challengesCompleted);
    }
  };

  return (
    <div>
      {isReady ? 'Ready for verification' : 'Initializing...'}
    </div>
  );
}
```

### **Node.js Server**
```typescript
import { FaceZK, generateSecureSalt } from 'facezk-core';

class VerificationService {
  private facezk: FaceZK;

  async initialize() {
    this.facezk = await FaceZK.create({
      systemSalt: generateSecureSalt(),
      enableAudit: true,
      auditRetentionDays: 365,
    });
  }

  async verifyUser(imageBuffer: Buffer) {
    const session = this.facezk.startSession();
    const result = await this.facezk.processFrame(imageBuffer);
    
    if (result?.verified) {
      return {
        success: true,
        humanityCode: result.humanityCode,
        biometricId: result.biometricId,
        livenessScore: result.livenessScore,
        challengesCompleted: result.metadata.challengesCompleted,
      };
    }
    
    return { success: false };
  }
}
```

## 🐛 Troubleshooting

### **Model Loading Issues**
If you encounter 404 errors for model files:
1. Ensure the `models` directory exists in your project root
2. Verify the demo server is configured to serve `/models` directory
3. Check that all required `.json` and `.bin` files are present

### **Expression Detection Issues**
If expressions are not being detected:
1. Ensure good lighting conditions
2. Position face clearly in camera view
3. Complete challenges in the correct order: Blink → Head Turn → Smile
4. Check browser console for detailed error messages

### **Performance Issues**
If experiencing slow performance:
1. Reduce `maxMemoryMB` in configuration
2. Enable memory monitoring and cleanup
3. Use WebGL backend for better GPU acceleration
4. Adjust `expressionDebounceMs` for faster response

## 🤝 Contributing

We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.

### **Development Setup**
```bash
git clone https://github.com/facezk/facezk-lib.git
cd facezk-lib
npm install
npm run dev
```

### **Code Quality**
```bash
npm run lint
npm run type-check
npm run test:coverage
```

## 📄 License

MIT License - see [LICENSE](LICENSE) for details.

## 🔗 Links

- **Documentation**: [https://facezk.com/docs](https://facezk.com/docs)
- **API Reference**: [https://facezk.com/api](https://facezk.com/api)
- **Security**: [https://facezk.com/security](https://facezk.com/security)
- **Migration Guide**: [MIGRATION_GUIDE.md](MIGRATION_GUIDE.md)

## 🆘 Support

- **Issues**: [GitHub Issues](https://github.com/facezk/facezk-lib/issues)
- **Discussions**: [GitHub Discussions](https://github.com/facezk/facezk-lib/discussions)
- **Email**: [support@verifiedcreators.io](mailto:support@verifiedcreators.io)

---

**FaceZK v1.1.0** - Privacy-First Human Verification with Advanced Liveness Detection

*Built by the team behind VerifiedCreators and VerifiedOnchain Protocol*