UNPKG

claude-flow-novice

Version:

Claude Flow Novice - Advanced orchestration platform for multi-agent AI workflows with CFN Loop architecture Includes Local RuVector Accelerator and all CFN skills for complete functionality.

374 lines (316 loc) 25.6 kB
# Agent Execution Architecture - Quick Reference ## One-Page Comparison ``` ╔════════════════════════════════════════════════════════════════════════════╗ THREE AGENT ORCHESTRATION MODELS ╚════════════════════════════════════════════════════════════════════════════╝ ┌─────────────────────────────────────────────────────────────────────────┐ 1. QuDAG: Decentralized P2P Network ├─────────────────────────────────────────────────────────────────────────┤ Runtime: Docker containers Isolation: Container-level (best) Communication: P2P gossip protocol (50-500ms latency) Scaling: Horizontal (add nodes, geographic distribution) Fault Tolerance: Byzantine robust (survives node failures) Agents: 10-agent permanent swarm Convergence: Test-driven (TDD is primary criterion) Complexity: Very High Best For: Decentralized systems, untrusted environments Pros: Cons: - Geographic distribution - High operational overhead - No single point of failure - Slow consensus (network bound) - Byzantine fault tolerance - Complex custom protocol - Test-driven quality focus - Long startup time (docker) - Continuous improvement (post) - Requires consensus agreement └────────────────────────────────────┴────────────────────────────────────┘ ┌─────────────────────────────────────────────────────────────────────────┐ 2. daa: Rust Async with MCP Integration ├─────────────────────────────────────────────────────────────────────────┤ Runtime: Tokio async processes (native Rust) Isolation: Process-level (good) Communication: MCP + Tokio channels (<1ms latency) Scaling: Vertical (threadpool, many agents per machine) Fault Tolerance: Coordinator-based (detected + respawned) Agents: 100+ agents per orchestrator Convergence: Workflow-based (DAG execution) Complexity: Medium-High Best For: Performance-critical, single-machine deployments Pros: Cons: - Very low latency (in-process) - Requires Rust ecosystem - High throughput (many agents) - Single machine only (default) - Type safety (Rust) - MCP still evolving - MCP AI agent integration - Complex async debugging - Workflow engine structuring - Larger binary sizes └────────────────────────────────────┴────────────────────────────────────┘ ┌─────────────────────────────────────────────────────────────────────────┐ 3. claude-flow-novice: Redis-Coordinated CLI ├─────────────────────────────────────────────────────────────────────────┤ Runtime: Bash subprocess / Node.js CLI Isolation: Process-level (good) Communication: Redis Pub/Sub + key-value (10-50ms latency) Scaling: Sequential CLI spawning (limited parallelism) Fault Tolerance: Orchestrator health checks (restart failed agents) Agents: 10-20 per iteration (CFN Loop) Convergence: Confidence-gated iteration + validator consensus Complexity: Low-Medium Best For: Rapid iteration, human-in-the-loop, development Pros: Cons: - Simple coordination (Redis) - Redis single point of failure - Easy to debug (inspect keys) - Not geographically distributed - Human-readable (JSON in Redis) - Limited to local deployment - Fast development (minimal deps) - Network bound by Redis latency│ - Bash simplicity (shell scripts) - Requires Redis running - Confidence-based pass/fail - No Byzantine fault tolerance - Iterative refinement built-in - Slower than async natives └────────────────────────────────────┴────────────────────────────────────┘ ``` --- ## Performance Scorecard ``` ╔═══════════════════════════════════════════════════════════════════════╗ Metric QuDAG daa cf-novice Winner ╠═══════════════════╪═════════╪═════════╪═════════════╪═════════════════╣ Startup/Agent 5-30s 100ms 50ms cf-novice Message Latency 50-500ms│ <1ms 10-50ms daa Max Agents 100 1000+ 20 daa Geographic Scale Global Single Single QuDAG Fault Tolerance BFT Restart│ Restart QuDAG Setup Complexity Very H Med-H Low cf-novice Debugging Hard Medium Easy cf-novice Docker Ready Yes Via K8s│ No QuDAG Type Safety Medium High Low daa Test-Driven Yes No Yes QuDAG & cf ╚═══════════════════╧═════════╧═════════╧═════════════╧═════════════════╝ ``` --- ## Decision Matrix: Which to Use? ``` ┌────────────────────────────────────────────────────────────────────────┐ Question Answer Use This ├────────────────────────────────────────────────────────────────────────┤ Need to distribute across regions? QuDAG (P2P network) Building untrusted/permissionless sys? QuDAG (Byzantine tolerance) Maximizing throughput/performance? daa (Tokio async) Need 1000+ agents on single machine? daa (threadpool scales) Doing rapid development/iteration? cf-novice (Redis simple) Human in the loop needed? cf-novice (CFN Loop design) Require Rust type safety? daa (Rust native) Prefer shell scripts/simplicity? cf-novice (Bash-based) Building fintech/consensus app? QuDAG (BFT) Building ML training system? daa (async native + compute) Just testing agent ideas? cf-novice (lowest barrier) Need guaranteed delivery? QuDAG (consensus protocol) Want operator-friendly system? cf-novice (Redis introspect) Need sub-millisecond coordination? daa (in-process async) Building testnet/lab environment? QuDAG (docker-compose ready) └────────────────────────────────────────────────────────────────────────┘ ``` --- ## Communication Architecture Comparison ``` ╔════════════════════════════════════════════════════════════════════════════╗ COMMUNICATION LAYERS ╚════════════════════════════════════════════════════════════════════════════╝ QuDAG: 5-Layer Protocol Stack ┌─────────────────────────────────────────┐ Consensus (Byzantine Fault Tolerance) ├─────────────────────────────────────────┤ P2P Network (Custom Protocol) - Peer discovery (bootstrap) - Message routing - State sync (gossip) ├─────────────────────────────────────────┤ Transport (TCP/UDP) ├─────────────────────────────────────────┤ Docker Network (DNS Resolution) └─────────────────────────────────────────┘ Latency: 50-500ms | Complexity: Very High | Scope: Geographic daa: 3-Layer Architecture ┌─────────────────────────────────────────┐ MCP Agent Protocol (JSON-RPC) ├─────────────────────────────────────────┤ Tokio Async Channels (MPSC) - No serialization (in-process) - Fast message passing - Backpressure handling ├─────────────────────────────────────────┤ Optional: QuDAG P2P (multi-node) └─────────────────────────────────────────┘ Latency: <1ms | Complexity: Medium | Scope: Single Machine claude-flow-novice: 2-Layer Architecture ┌─────────────────────────────────────────┐ Redis Pub/Sub + Hash/Set/Sorted Set - Simple key-value stores - BLPOP for blocking (notifications) - SUBSCRIBE for signals ├─────────────────────────────────────────┤ Network (Redis Protocol) └─────────────────────────────────────────┘ Latency: 10-50ms | Complexity: Low | Scope: Single Machine ``` --- ## Agent Lifecycle Comparison ``` QuDAG AGENT LIFECYCLE (Containerized) ┌──────────────────────────────────────────────────────────┐ 1. Image Build (Rust compilation) 2. Container Spawn (docker run with env vars) 3. Peer Discovery (connect to bootstrap peers) 4. Network Sync (consensus participation) 5. Task Polling (claim work from queue) 6. Work Execution (isolated in container) 7. Result Merge (integration agent validates) 8. Persistent (continues post-release) ⏱️ Total: 5-30 seconds per agent startup └──────────────────────────────────────────────────────────┘ daa AGENT LIFECYCLE (Tokio Async) ┌──────────────────────────────────────────────────────────┐ 1. Configuration (AIAgentConfig created) 2. Instantiation (AIAgent::new() async) 3. MCP Connect (initialize MCP client) 4. Tokio Spawn (task spawned in threadpool) 5. Tool Execution (MCP requests/responses) 6. Result Return (async result via channel) 7. Cleanup (orchestrator collects results) ⏱️ Total: 100-500ms per agent startup └──────────────────────────────────────────────────────────┘ claude-flow-novice AGENT LIFECYCLE (CLI-based) ┌──────────────────────────────────────────────────────────┐ 1. Agent Selection (via skill templates) 2. Environment Setup (AGENT_ID, TASK_ID, context) 3. Process Spawn (CLI: npx claude-flow-spawn) 4. Redis Connect (subscribe to task channel) 5. Work Execution (run task in subprocess) 6. Completion Signal (report-completion.sh) 7. Orchestrator Collects (invoke-waiting-mode.sh) ⏱️ Total: 50-200ms per agent startup └──────────────────────────────────────────────────────────┘ ``` --- ## Key Architectural Innovations ``` QuDAG's Key Strength: TEST-DRIVEN CONVERGENCE ┌─────────────────────────────────────────────────────────┐ Tests are the source of truth for completion No "agent thinks it's done" opinions If tests pass deliverable is correct Enables: Automatic agent retry on test failure Benefit: Objective quality criteria └─────────────────────────────────────────────────────────┘ daa's Key Strength: ASYNC/AWAIT NATIVE ┌─────────────────────────────────────────────────────────┐ Tokio enables 1000+ agents per machine <1ms latency (no network serialization) MCP integration for structured AI calls Workflow engine for complex orchestration Benefit: Maximum throughput & performance └─────────────────────────────────────────────────────────┘ claude-flow-novice's Key Strength: CONFIDENCE GATING ┌─────────────────────────────────────────────────────────┐ Objective pass/fail based on confidence scores Not subjective agent opinions ("I think it's done") Automatic retry when confidence < threshold Blind validator review prevents bias Benefit: Transparent decision-making └─────────────────────────────────────────────────────────┘ ``` --- ## Adoptable Patterns: Impact Summary ``` ┌──────────────────────────────────────────────────────────────┐ PATTERN IMPACT COMPLEXITY TIME ├──────────────────────────┼──────────┼────────────┼───────────┤ Confidence Gating HIGH Easy 1-2 days (objective pass/fail) ├──────────────────────────┼──────────┼────────────┼───────────┤ Blind Validator Review HIGH Medium 3-5 days (removes bias) ├──────────────────────────┼──────────┼────────────┼───────────┤ Test-Driven Convergence VERY HGH Medium 5-7 days (tests = proof) ├──────────────────────────┼──────────┼────────────┼───────────┤ Service Registry HC MEDIUM Medium 5-10 days (dynamic discovery) ├──────────────────────────┼──────────┼────────────┼───────────┤ MCP Protocol MEDIUM Complex 10-14 days│ (structured tools) └──────────────────────────┴──────────┴────────────┴───────────┘ RECOMMENDED ADOPTION ORDER: 1. Confidence Gating (foundation) 2. Test-Driven Convergence (objective validation) 3. Blind Review (consensus quality) 4. Service Registry (operational improvement) 5. MCP Protocol (long-term standardization) ``` --- ## Critical Insights ``` INSIGHT 1: No Single Winner Each architecture wins in its domain: QuDAG = Decentralization + Byzantine fault tolerance daa = Performance + Type safety cf-novice = Developer velocity + Simplicity RECOMMENDATION: Use the right tool for the job, not the "best" tool. 🎯 INSIGHT 2: Three Patterns Can Be Combined Confidence Gating + Blind Review + Test-Driven = Highest quality with objective criteria and no bias RECOMMENDATION: Implement all three in sequence for maximum impact. ⚠️ INSIGHT 3: Communication is Often the Bottleneck QuDAG: 50-500ms per message (network bound by consensus) daa: <1ms per message (no network serialization) cf-novice: 10-50ms per message (Redis network round-trip) RECOMMENDATION: Choose communication model before architecture. 📊 INSIGHT 4: Convergence Criteria Matter More Than Speed A slower system with good convergence beats a fast system with ambiguous completion criteria. RECOMMENDATION: Design convergence (tests? confidence? quorum?) before choosing execution model. 🔄 INSIGHT 5: Iteration is Better Than Perfection All three systems support iteration: QuDAG: Swarm iterates until tests pass daa: Workflow retries failed steps cf-novice: CFN Loop explicitly designed for iteration RECOMMENDATION: Expect and design for multiple iterations. ``` --- ## Next Steps 1. **Read Full Analysis:** `/home/user/claude-flow-novice/docs/ARCHITECTURAL_COMPARISON_QUDAG_DAA.md` 2. **Implementation Guide:** `/home/user/claude-flow-novice/docs/ADOPTABLE_PATTERNS_IMPLEMENTATION_GUIDE.md` 3. **Quick Decision:** Use the decision matrix above to pick a model 4. **Start Adopting:** Begin with Confidence Gating pattern (1-2 days) 5. **Measure Impact:** Track decision clarity, iteration count, consensus quality --- ## Repository References - **QuDAG**: https://github.com/ruvnet/QuDAG - File: `qudag-exchange/plans/swarm-orchestration.md` - File: `docker-compose.yml` - **daa**: https://github.com/ruvnet/daa - File: `daa-orchestrator/src/lib.rs` - File: `crates/daa-ai/src/agent.rs` - **claude-flow-novice**: `/home/user/claude-flow-novice` - File: `.claude/skills/cfn-loop-orchestration/orchestrate.sh` - File: `.claude/skills/cfn-redis-coordination/` --- **Analysis Confidence: 0.92** | **Date: November 15, 2025**