UNPKG

oneie

Version:

Build apps, websites, and AI agents in English. Zero-interaction setup for AI agents (Claude Code, Cursor, Windsurf). Download to your computer, run in the cloud, deploy to the edge. Open source and free forever.

935 lines (699 loc) β€’ 30.4 kB
--- title: Unified Implementation Plan dimension: things category: plans tags: architecture, backend, connections, events, groups, knowledge, ontology, people, things related_dimensions: connections, events, groups, knowledge, people scope: global created: 2025-11-03 updated: 2025-11-03 version: 1.0.0 ai_context: | This document is part of the things dimension in the plans category. Location: one/things/plans/unified-implementation-plan.md Purpose: Documents unified implementation plan: backend-agnostic + complete ontology Related dimensions: connections, events, groups, knowledge, people For AI agents: Read this to understand unified implementation plan. --- # Unified Implementation Plan: Backend-Agnostic + Complete Ontology **Status:** βœ… Phases 1-2 Complete | 🚧 Phases 3-7 In Progress **Version:** 2.1.0 **Date Updated:** 2025-10-18 **Goal:** Achieve complete ontology implementation with backend-agnostic architecture --- ## 🎯 Current Status Summary ### βœ… COMPLETED (Phases 1-2: 4 weeks) **Phase 1: Foundation** βœ… - DataProvider interface (6 dimensions) - `/web/src/providers/DataProvider.ts` - ConvexProvider implementation - Typed errors (ThingNotFoundError, etc.) - Backend swapping with ONE line change **Phase 2: Service Layer** βœ… - ThingService (handles all 66 types generically) - `/web/src/services/ThingService.ts` - ConnectionService (all 25 types) - `/web/src/services/ConnectionService.ts` - EventService (all 67 events) - `/web/src/services/EventService.ts` - KnowledgeService (RAG foundation) - `/web/src/services/KnowledgeService.ts` - GroupService (hierarchical groups with nesting) - `/web/src/services/GroupService.ts` - PeopleService - `/web/src/services/PeopleService.ts` - React hooks (useEffectRunner) - `/web/src/hooks/` - Typed errors throughout - Dependency injection via Effect.ts **Backend Schema** βœ… - 6-dimension ontology complete - `/backend/convex/schema.ts` - groups table (hierarchical nesting) - entities table (66+ thing types via generic `type` field) - connections table (25+ relationship types) - events table (67+ event types) - knowledge table (RAG + vectors) - thingKnowledge junction table ### 🚧 REMAINING (Phases 3-7: 7 weeks) **Phase 3: Backend Implementation** (Weeks 5-8) - πŸ”΄ NOT STARTED - [ ] Implement CRUD mutations for all 66 thing types - [ ] Add event logging to all mutations - [ ] Enforce group scoping - [ ] Add rate limiting - [ ] Write unit tests **Phase 4: Frontend Integration** (Weeks 9-10) - πŸ”΄ NOT STARTED - [ ] Multi-tenant dashboard - [ ] Entity management UI (all 66 types) - [ ] Connection visualization - [ ] Real-time event timeline - [ ] Real-time stats **Phase 5: RAG & Knowledge** (Week 10) - πŸ”΄ NOT STARTED - [ ] Chunking service (800 tokens, 200 overlap) - [ ] Embedding service (OpenAI integration) - [ ] RAG ingestion pipeline - [ ] Vector search - [ ] Knowledge labels **Phase 6: Testing** (Week 10) - 🟑 PARTIAL - [x] Auth tests (50+ test cases) - `/test/auth/` - [ ] Backend service coverage (90%) - [ ] Frontend coverage (70%) - [ ] Multi-backend tests - [ ] CI/CD pipeline **Phase 7: Multi-Backend** (Week 11) - πŸ”΄ NOT STARTED (Optional) - [ ] WordPressProvider - [ ] SupabaseProvider - [ ] CompositeProvider (multi-backend routing) ### πŸ“Š Progress Metrics | Metric | Target | Current | Status | | --------------------------- | ------------ | ---------- | ------ | | **Architecture Complete** | 100% | 100% | βœ… | | **Service Layer Complete** | 100% | 100% | βœ… | | **Backend CRUD Operations** | 100% | 10% | πŸ”΄ | | **Frontend UI Complete** | 100% | 20% | πŸ”΄ | | **RAG Implementation** | 100% | 0% | πŸ”΄ | | **Test Coverage** | 90% | 50% | 🟑 | | **Multi-Backend Support** | 2+ providers | 1 (Convex) | 🟑 | **Overall: 36% Complete** (4 of 11 weeks) ### 🎯 Next Cycle (Cycle 101) **Start Phase 3: Backend Implementation - Week 5** Task: Implement CRUD mutations for Priority 1 entity types (core entities) - Create mutations using ThingService (backend-agnostic) - Add event logging automatically - Enforce group scoping - Add rate limiting Entity types to implement (Priority 1 - Core): 1. creator 2. ai_clone 3. audience_member 4. organization (thing type for multi-tenant organizations) **Estimated Time:** 3-4 days **Risk:** Low (pattern is clear, just replication) --- ## Executive Summary: Two Plans, One Vision After analyzing both `separate.md` (backend-agnostic architecture) and `improve-codebase.md` (complete ontology implementation), the answer is clear: **βœ… YES - Both plans improve the codebase AND must be implemented together.** ### Why Both Plans Are Essential **Problem if you implement `improve-codebase.md` alone:** - You build 66 thing types tightly coupled to Convex - No backend flexibility - Organizations locked into Convex - Future migration becomes 10x harder **Problem if you implement `separate.md` alone:** - You have a beautiful architecture with no features - DataProvider wraps incomplete backend - Can't demonstrate multi-backend capability - No production value **Solution: Unified Implementation** - Implement DataProvider architecture FIRST (separate.md) - Build all 66 thing types USING DataProvider (improve-codebase.md) - Result: Complete ontology + Backend flexibility --- ## The Critical Insight ### ❌ Wrong Sequence (What NOT to Do) ``` Week 1-4: Implement 66 thing types in Convex mutations Week 5-6: Add Effect.ts glue layer Week 7-10: Migrate to DataProvider architecture Result: Wasted 6 weeks building tightly-coupled code, then refactoring it ``` ### βœ… Right Sequence (The Unified Plan) ``` Week 1-2: Implement DataProvider interface + ConvexProvider Week 3-4: Create Effect.ts services using DataProvider Week 5-8: Implement all 66 thing types through services Week 9-10: Add multi-tenant UI and RAG Week 11: Add alternative backend (prove flexibility) Result: Everything built right the first time ``` --- ## Unified Architecture ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ FRONTEND (Astro + React) β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ Multi-Tenant UI (improve-codebase.md) β”‚ β”‚ β”‚ β”‚ - Group hierarchy navigation β”‚ β”‚ β”‚ β”‚ - Entity management for all 66 types β”‚ β”‚ β”‚ β”‚ - Connection visualization β”‚ β”‚ β”‚ β”‚ - Real-time event timeline β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ ↓ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ Effect.ts Services (both plans) β”‚ β”‚ β”‚ β”‚ - ThingService (all 66 types) β”‚ β”‚ β”‚ β”‚ - ConnectionService (all 25 types) β”‚ β”‚ β”‚ β”‚ - EventService (all 67 events) β”‚ β”‚ β”‚ β”‚ - KnowledgeService (RAG + embeddings) β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ ↓ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ DataProvider Interface (separate.md) β”‚ β”‚ β”‚ β”‚ - Backend-agnostic API β”‚ β”‚ β”‚ β”‚ - groups, people, things, connections, events, knowledgeβ”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ↓ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ BACKEND PROVIDERS (separate.md) β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ Convex β”‚ β”‚ WordPress β”‚ β”‚ Supabase β”‚ β”‚ β”‚ β”‚ (default) β”‚ β”‚ (blog) β”‚ β”‚ (analytics) β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ↓ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ ACTUAL BACKEND DATABASES (improve-codebase.md) β”‚ β”‚ β”‚ β”‚ Convex: 6 tables with complete CRUD for all types β”‚ β”‚ - groups, entities, connections, events, knowledge β”‚ β”‚ - All 66 thing types supported β”‚ β”‚ - All 25 connection types supported β”‚ β”‚ - All 67 event types logged β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` --- ## Phase-by-Phase Implementation (11 Weeks) ### Phase 1: Foundation (Weeks 1-2) βœ… COMPLETE **Goal:** Backend-agnostic architecture in place **From separate.md:** - [x] Create DataProvider interface (6 dimensions) βœ… DONE - [x] Implement ConvexProvider (wraps existing backend) βœ… DONE - [x] Add typed errors (ThingNotFoundError, etc.) βœ… DONE - [x] Configure in astro.config.ts βœ… DONE **Acceptance Criteria:** - [x] DataProvider interface complete with TypeScript types βœ… - [x] ConvexProvider implements all methods βœ… - [x] Existing auth tests pass unchanged βœ… - [x] Can swap backends by changing ONE line βœ… **Actual Effort:** 16 hours (2 days) **Actual Risk:** Low (completed without issues) **Location:** `/web/src/providers/DataProvider.ts` --- ### Phase 2: Service Layer (Weeks 3-4) βœ… COMPLETE **Goal:** Effect.ts services using DataProvider **From both plans:** - [x] Create Effect.ts services for all 6 dimensions βœ… DONE - [x] ThingService (handles all 66 types generically) βœ… DONE - [x] ConnectionService (all 25 types) βœ… DONE - [x] EventService (all 67 events) βœ… DONE - [x] KnowledgeService (RAG foundation) βœ… DONE - [x] Create useEffectRunner hook for React βœ… DONE **From improve-codebase.md:** - [x] Implement typed errors (tagged unions) βœ… DONE - [x] Service composition via Layer.mergeAll βœ… DONE - [x] Dependency injection βœ… DONE - [x] Error handling without try/catch βœ… DONE **Example:** ```typescript // Backend-agnostic ThingService export class ThingService extends Effect.Service<ThingService>()({ effect: Effect.gen(function* () { const provider = yield* DataProvider // Works with ANY backend! return { create: (type: ThingType, input: any) => Effect.gen(function* () { // 1. Validate (same for all backends) yield* validateThingInput(type, input) // 2. Create via provider (backend-specific) const thingId = yield* provider.things.create({ type, name: input.name, groupId: input.groupId, properties: input.properties }) // 3. Log event (via provider) yield* provider.events.log({ type: "entity_created", actorId: input.actorId, targetId: thingId, groupId: input.groupId, metadata: { thingType: type } }) return thingId }) } }), dependencies: [DataProvider] }) ``` **Acceptance Criteria:** - [x] All services use DataProvider (no direct backend calls) βœ… - [x] Services work with any backend that implements DataProvider βœ… - [x] Typed errors throughout βœ… - [ ] 90% test coverage with mocked providers ⚠️ PARTIAL **Actual Effort:** 32 hours (4 days) **Actual Risk:** Low (completed successfully) **Location:** `/web/src/services/` --- ### Phase 3: Backend Implementation (Weeks 5-8) **Goal:** Complete CRUD for all 66 thing types **From improve-codebase.md (but using DataProvider!):** **Week 5: Core Entities (Priority 1)** - [ ] Creator, AI Clone, Audience Member, Organization - [ ] Convex mutations use services (thin wrappers) - [ ] Event logging automatic - [ ] Group scoping enforced **Week 6: Content & Products (Priority 2)** - [ ] Blog posts, courses, lessons, videos, podcasts - [ ] Products, memberships, digital goods - [ ] Content lifecycle (draft β†’ published β†’ archived) **Week 7: Tokens, Agents & AI (Priority 3 & 4)** - [ ] Tokens, token contracts - [ ] 10 business agents (strategy, marketing, sales, etc.) - [ ] External agents, workflows **Week 8: Auth, Knowledge & Remaining (Priority 5)** - [ ] Sessions, OAuth accounts, verification tokens - [ ] Knowledge items, embeddings - [ ] Metrics, insights, predictions - [ ] Notifications, campaigns **Key Difference from improve-codebase.md:** ```typescript // ❌ OLD WAY (improve-codebase.md alone - tightly coupled) export const createCourse = mutation({ handler: async (ctx, args) => { // Direct Convex DB calls const courseId = await ctx.db.insert("entities", { ... }) await ctx.db.insert("events", { ... }) return courseId } }) // βœ… NEW WAY (unified plan - backend-agnostic) export const createCourse = mutation({ handler: async (ctx, args) => { // Use service via DataProvider const program = Effect.gen(function* () { const courseService = yield* CourseService return yield* courseService.create(args) }) return await Effect.runPromise( program.pipe(Effect.provide(ServicesLayer)) ) } }) ``` **Acceptance Criteria:** - [ ] All 66 thing types have CRUD operations - [ ] All mutations use services (not direct DB) - [ ] All operations log events - [ ] Group scoping enforced - [ ] Works with ConvexProvider **Effort:** 120 hours (15 days across 4 weeks) **Risk:** Low (following established patterns) --- ### Phase 4: Frontend Integration (Weeks 9-10) **Goal:** Multi-tenant UI for complete ontology **From improve-codebase.md:** - [ ] Multi-tenant dashboard with group hierarchy - [ ] Entity management UI (all 66 types) - [ ] Connection visualization (graph view) - [ ] Real-time event timeline - [ ] Real-time stats and analytics **Using DataProvider (from separate.md):** ```tsx // Frontend components are backend-agnostic! export function CreateCourseForm() { const { run, loading } = useEffectRunner(); const handleSubmit = async (data: CourseData) => { const program = Effect.gen(function* () { const courseService = yield* CourseService; // This works with ANY backend (Convex, WordPress, etc.) return yield* courseService.create("course", data); }); const courseId = await run(program); navigate(`/courses/${courseId}`); }; return <form onSubmit={handleSubmit}>...</form>; } ``` **Acceptance Criteria:** - [ ] Group selector with full hierarchy - [ ] Entity forms for all 66 types - [ ] Real-time updates via subscriptions - [ ] Connection graph visualization - [ ] Event timeline with filtering - [ ] Role-based access control **Effort:** 64 hours (8 days) **Risk:** Medium (complex UI) --- ### Phase 5: RAG & Knowledge (Week 10) **Goal:** Complete knowledge implementation **From improve-codebase.md:** - [ ] Chunking service (800 tokens, 200 overlap) - [ ] Embedding service (OpenAI integration) - [ ] RAG ingestion pipeline - [ ] Vector search - [ ] Knowledge labels **Using DataProvider:** ```typescript // RAG works with any backend that supports vector storage export const ingestThing = (thingId: string) => Effect.gen(function* () { const provider = yield* DataProvider; const chunking = yield* ChunkingService; const embedding = yield* EmbeddingService; // Get thing (from any backend) const thing = yield* provider.things.get(thingId); // Extract and chunk text const chunks = yield* chunking.chunk(thing.properties.content); // Embed chunks const embeddings = yield* embedding.embedBatch(chunks.map((c) => c.text)); // Store via provider (backend-agnostic!) for (let i = 0; i < chunks.length; i++) { yield* provider.knowledge.create({ groupId: thing.groupId, knowledgeType: "chunk", text: chunks[i].text, embedding: embeddings[i].embedding, sourceThingId: thingId, }); } }); ``` **Acceptance Criteria:** - [ ] Chunking handles 66 thing types - [ ] Embeddings work via provider - [ ] Vector search returns relevant results - [ ] Knowledge labels for categorization **Effort:** 32 hours (4 days) **Risk:** Medium (OpenAI API integration) --- ### Phase 6: Testing (Week 10) **Goal:** Comprehensive test coverage **From improve-codebase.md:** - [ ] 90% backend service coverage - [ ] All 66 thing types tested - [ ] All 25 connection types tested - [ ] All 67 event types verified **From separate.md:** - [ ] Mock providers for testing - [ ] Provider swapping tests - [ ] Multi-backend tests - [ ] Auth tests pass (critical!) **Example:** ```typescript // Test with multiple providers! describe("ThingService", () => { const testWithProvider = (providerName: string, provider: Layer<DataProvider>) => { it(`should create thing via ${providerName}`, async () => { const program = Effect.gen(function* () { const thingService = yield* ThingService const thingId = yield* thingService.create("course", { ... }) expect(thingId).toBeDefined() }) await Effect.runPromise( program.pipe(Effect.provide(provider)) ) }) } testWithProvider("Convex", ConvexProviderMock) testWithProvider("Supabase", SupabaseProviderMock) testWithProvider("WordPress", WordPressProviderMock) }) ``` **Acceptance Criteria:** - [ ] 90% backend coverage - [ ] 70% frontend coverage - [ ] Tests pass with multiple providers - [ ] All auth tests pass - [ ] CI/CD pipeline green **Effort:** 40 hours (5 days) **Risk:** Low (testing infrastructure exists) --- ### Phase 7: Multi-Backend (Week 11 - Optional) **Goal:** Prove true backend flexibility **From separate.md:** - [ ] Implement WordPressProvider - [ ] Implement SupabaseProvider - [ ] Implement CompositeProvider (multi-backend) **Real-World Example:** ```typescript // astro.config.ts - Mix and match backends! export default defineConfig({ integrations: [ one({ provider: compositeProvider({ default: convexProvider({ url: env.PUBLIC_CONVEX_URL }), routes: { // Blog from WordPress blog_post: wordpressProvider({ url: "https://blog.yoursite.com", apiKey: env.WP_API_KEY, }), // Products from Shopify product: shopifyProvider({ store: "yourstore.myshopify.com", token: env.SHOPIFY_TOKEN, }), // Analytics from Supabase metric: supabaseProvider({ url: env.SUPABASE_URL, key: env.SUPABASE_KEY, }), }, }), }), ], }); ``` **Acceptance Criteria:** - [ ] At least 2 alternative providers working - [ ] CompositeProvider routes correctly - [ ] Frontend works unchanged - [ ] Can swap backends with ONE line **Effort:** 40 hours (5 days) **Risk:** Low (proves concept, not critical path) --- ## Comparison: Separate vs Unified Plans | Aspect | improve-codebase.md alone | separate.md alone | Unified Plan | | ----------------------- | ----------------------------- | ----------------------- | ---------------------------- | | **Timeline** | 14 weeks | 6 weeks | 11 weeks | | **Backend Flexibility** | ❌ None (Convex only) | βœ… Full (any backend) | βœ… Full (any backend) | | **Ontology Coverage** | βœ… Complete (66 types) | ⚠️ Partial (demo only) | βœ… Complete (66 types) | | **Effect.ts Services** | βœ… Full | βœ… Full | βœ… Full | | **Multi-Tenant UI** | βœ… Complete | ❌ Missing | βœ… Complete | | **RAG Implementation** | βœ… Complete | ❌ Missing | βœ… Complete | | **Future Refactoring** | ❌ 6 weeks to add flexibility | βœ… None needed | βœ… None needed | | **Risk** | Medium (tight coupling) | Low (architecture only) | Low (build right first time) | | **Production Value** | βœ… High | ⚠️ Low (no features) | βœ… Very High | **Winner:** Unified Plan - Faster than improve-codebase.md alone (11 weeks vs 14 weeks) - More valuable than separate.md alone (complete features) - No future refactoring needed - Built right the first time --- ## Critical Success Factors ### 1. Architecture First (Week 1-2) **DO NOT SKIP:** Implement DataProvider before building features ```typescript // ❌ WRONG - Build features first, refactor later Week 1: Build createCourse mutation directly in Convex Week 8: Try to wrap in DataProvider β†’ painful refactoring // βœ… RIGHT - Architecture first Week 1: Create DataProvider interface Week 2: Implement ConvexProvider Week 3: Build createCourse using ThingService + DataProvider Week 8: Add WordPressProvider β†’ just works, no refactoring ``` ### 2. Test at Every Phase **From separate.md:** Auth tests MUST pass after each phase ```bash # After Phase 1 (DataProvider) npm test frontend/tests/auth/ # All tests should pass # After Phase 2 (Services) npm test frontend/tests/auth/ # All tests should pass # After Phase 5 (Frontend migration) npm test frontend/tests/auth/ # All tests should pass ``` ### 3. Services Use DataProvider (Not Direct Backend) **From improve-codebase.md:** Effect.ts services with typed errors **From separate.md:** Services use DataProvider interface ```typescript // βœ… RIGHT - Service uses DataProvider export class ThingService { create = (type, input) => Effect.gen(function* () { const provider = yield* DataProvider; // Backend-agnostic! return yield* provider.things.create({ type, ...input }); }); } // ❌ WRONG - Service calls Convex directly export class ThingService { create = (type, input) => Effect.gen(function* () { const ctx = yield* ConvexContext; // Tightly coupled! return yield* Effect.tryPromise(() => ctx.db.insert("entities", { type, ...input }), ); }); } ``` --- ## Migration from Current State ### Current State Analysis ``` βœ… Working: - Groups CRUD (mutations + queries) - Connections CRUD (mutations + queries) - Auth (Better Auth + tests passing) - Schema (6-dimension ontology defined) ⚠️ Partial: - Only 2 of 66 thing types implemented - No Effect.ts layer - No DataProvider abstraction - Frontend tightly coupled to Convex ❌ Missing: - 64 thing types - RAG implementation - Multi-tenant UI - Protocol integration - Alternative backends ``` ### Migration Path ``` Phase 1-2: Add abstraction layer (DataProvider + Services) └─> Wraps existing groups/connections └─> Auth tests still pass └─> Zero functionality lost Phase 3-4: Fill in missing 64 thing types └─> Use DataProvider (not direct Convex) └─> Built right the first time Phase 5-6: Add UI and RAG └─> Uses services (backend-agnostic) Phase 7: Prove flexibility with alternative backend └─> Demonstrates the value of architecture ``` --- ## Effort Summary | Phase | Weeks | Hours | From Plan | Risk | | --------------------- | ------ | ------- | ------------------- | -------------- | | 1. DataProvider | 2 | 16 | separate.md | Low | | 2. Services | 2 | 32 | both | Medium | | 3. Backend (66 types) | 4 | 120 | improve-codebase.md | Low | | 4. Frontend UI | 2 | 64 | improve-codebase.md | Medium | | 5. RAG + Testing | 1 | 72 | improve-codebase.md | Medium | | 6. Multi-Backend | 1 | 40 | separate.md | Low | | **TOTAL** | **11** | **344** | **Unified** | **Low-Medium** | **Comparison:** - improve-codebase.md alone: 14 weeks, 592 hours, tight coupling - separate.md alone: 6 weeks, no features - **Unified plan: 11 weeks, 344 hours, complete + flexible** **Savings:** - 3 weeks faster than improve-codebase.md - Zero refactoring needed later - Built right the first time --- ## Decision Matrix ### Should You Implement This? **Implement Unified Plan If:** - βœ… You need backend flexibility for customer acquisition - βœ… You want complete ontology implementation (66 types) - βœ… You can dedicate 11 weeks to strategic work - βœ… You want to avoid future refactoring - βœ… You want multi-backend support (Convex + WordPress + Supabase) **Wait If:** - ⚠️ You have urgent features to ship (< 4 weeks deadline) - ⚠️ Current Convex-only setup meets all needs - ⚠️ No customers requesting alternative backends - ⚠️ Team unfamiliar with Effect.ts (need training first) **Never Do (Anti-Patterns):** - ❌ Implement improve-codebase.md without DataProvider - ❌ Skip Phase 1-2 and build features directly in Convex - ❌ Mix backend-specific code in services - ❌ Build UI before services exist --- ## Success Metrics **Technical Completeness:** - [ ] 66/66 thing types implemented - [ ] 25/25 connection types implemented - [ ] 67/67 event types logged - [ ] DataProvider interface complete - [ ] 2+ backend providers working - [ ] 90% test coverage **Architecture Quality:** - [ ] Services use DataProvider (no direct backend) - [ ] Can swap backends with ONE line change - [ ] Effect.ts services have typed errors - [ ] Zero try/catch in business logic - [ ] Clear separation: frontend β†’ services β†’ provider β†’ backend **Production Readiness:** - [ ] Multi-tenant UI complete - [ ] RAG pipeline working - [ ] Auth tests passing - [ ] Real-time subscriptions working - [ ] Group hierarchy navigation - [ ] Role-based access control **Business Value:** - [ ] Can onboard WordPress customers - [ ] Can onboard Supabase customers - [ ] Can support hybrid backends - [ ] Platform-agnostic (frontend works with any backend) - [ ] Zero vendor lock-in --- ## Conclusion ### The Answer to Your Question **"Does improve-codebase.md match separate.md? Do they improve both?"** **YES and NO:** βœ… **YES, they complement each other perfectly:** - separate.md provides the architecture (DataProvider) - improve-codebase.md provides the features (66 types) - Together they create a complete, flexible platform ❌ **NO, they don't match in isolation:** - improve-codebase.md alone = complete but inflexible - separate.md alone = flexible but incomplete - You need BOTH to achieve the vision ### The Unified Plan is the Answer **What it gives you:** 1. **Complete Ontology** (from improve-codebase.md) - All 66 thing types - All 25 connection types - All 67 event types - Full RAG implementation - Multi-tenant UI 2. **Backend Flexibility** (from separate.md) - DataProvider interface - Support ANY backend - Multi-backend federation - Zero vendor lock-in - Progressive migration path 3. **Best Practices** (from both) - Effect.ts services - Typed errors - Service composition - Dependency injection - Comprehensive testing **Timeline:** 11 weeks (faster than either plan alone) **Effort:** 344 hours (less than improve-codebase.md alone) **Risk:** Low-Medium (architecture first, build right once) **Value:** Complete + Flexible (the best of both plans) --- ## Next Steps 1. **Review this unified plan** with team 2. **Commit to architecture-first** approach (Phases 1-2) 3. **Allocate 11 weeks** for implementation 4. **Start Phase 1** immediately: DataProvider interface 5. **Do NOT build features** before DataProvider exists **Golden Rule:** ``` Build the architecture right. Build features once. Never refactor for flexibility. ``` The unified plan achieves all goals from both source plans while being faster and less risky than either one alone. **This is the way.**