UNPKG

oneie

Version:

Build apps, websites, and AI agents in English. Zero-interaction setup for AI agents (Claude Code, Cursor, Windsurf). Download to your computer, run in the cloud, deploy to the edge. Open source and free forever.

1,769 lines (1,448 loc) 88.2 kB
--- title: 2 2 Config System dimension: things category: features tags: backend, frontend related_dimensions: connections, events, groups, knowledge, people scope: global created: 2025-11-03 updated: 2025-11-03 version: 1.0.0 ai_context: | This document is part of the things dimension in the features category. Location: one/things/features/2-2-config-system.md Purpose: Documents feature 2-2: configuration system Related dimensions: connections, events, groups, knowledge, people For AI agents: Read this to understand 2 2 config system. --- # Feature 2-2: Configuration System **Feature ID:** `feature_2_2_config_system` **Plan:** `plan_2_backend_agnostic_frontend` **Owner:** Backend Specialist **Status:** Complete Specification **Priority:** P0 (Critical - Required for Provider Switching) **Effort:** 3 days **Dependencies:** Feature 2-1 (DataProvider Interface) --- ## Executive Summary The Configuration System enables ONE Platform to support multiple backend providers (Convex, WordPress, Notion, Supabase) with runtime provider switching, multi-tenant isolation, encrypted credential storage, and <30-second switchover times. Organizations configure their backend provider via environment variables or runtime API, with all credentials encrypted at rest and provider factories instantiating the correct implementation on demand. **Key Features:** - Environment-based configuration with Zod validation - Runtime provider switching per organization - Encrypted API key storage using AES-256-GCM - Provider factory pattern with lazy initialization - Multi-tenant isolation (each org has independent provider) - Provider registration system for extensibility - Event logging for all configuration changes - Fast switchover (<30 seconds) --- ## 1. Ontology Mapping (6 Dimensions) ### 1.1 Organizations **Purpose:** Each organization has independent backend provider configuration. **Schema Addition:** ```typescript // entities table (type: "organization") properties: { // ... existing org properties backendProvider?: { type: "convex" | "wordpress" | "notion" | "supabase", configId?: Id<"entities">, // Reference to external_connection switchedAt?: number, // Last provider switch timestamp previousProvider?: string, // For rollback } } ``` **Usage:** - `org.properties.backendProvider.type` determines which provider to use - `org.properties.backendProvider.configId` links to encrypted credentials - Platform owner can switch providers for testing - Org owners can switch their own provider (with proper permissions) ### 1.2 People **Purpose:** Only org_owner and platform_owner can change backend providers. **Roles with Config Access:** - `platform_owner`: Can configure ANY organization's provider (for support/migration) - `org_owner`: Can configure THEIR organization's provider - `org_user`: Read-only access to provider type (not credentials) - `customer`: No access to provider configuration **Permission Check Pattern:** ```typescript // Before allowing provider switch const person = await getPerson(userId); const org = await getOrganization(orgId); // Check authorization if (person.role !== "platform_owner") { // Must be org_owner of THIS org const membership = await getMembership(userId, orgId); if (!membership || membership.metadata.role !== "org_owner") { throw new Error("Forbidden: Only org owners can change providers"); } } ``` ### 1.3 Things (Entities) **New Entity Type: `external_connection`** ```typescript type: "external_connection" name: "Convex Production Backend" | "WordPress Staging" | etc. properties: { platform: "convex" | "wordpress" | "notion" | "supabase" | "custom", connectionType: "backend_provider", // Distinguishes from other external connections // Connection details (NOT encrypted here - see encryption section) baseUrl?: string, // API base URL apiKey?: string, // Encrypted API key (AES-256-GCM) apiKeyIv?: string, // Initialization vector for encryption apiKeyAuthTag?: string, // Auth tag for GCM mode // Additional provider-specific config config: { // Convex deploymentUrl?: string, deploymentName?: string, // WordPress wpJsonEndpoint?: string, username?: string, applicationPassword?: string, // Encrypted // Notion notionToken?: string, // Encrypted databaseId?: string, // Supabase supabaseUrl?: string, supabaseAnonKey?: string, // Encrypted supabaseServiceKey?: string, // Encrypted (admin operations) }, // Metadata status: "active" | "inactive" | "error", lastConnectedAt?: number, lastError?: string, healthCheckUrl?: string, rateLimits?: { requestsPerMinute: number, requestsPerDay: number, }, } status: "active" | "inactive" createdAt: number updatedAt: number ``` **Why This Design:** - Reuses existing `external_connection` entity type (no new type needed) - Encrypted credentials stored in properties (never in plaintext) - Supports multiple configs per organization (dev, staging, prod) - Can health-check provider availability - Rate limits prevent quota exhaustion ### 1.4 Connections **Organization → Backend Config:** ```typescript { fromEntityId: organizationId, toEntityId: externalConnectionId, relationshipType: "configured_by", metadata: { configType: "backend_provider", activeProvider: true, // Only one can be active per org switchedAt: Date.now(), switchedBy: personId, }, validFrom: Date.now(), validTo?: undefined, // Current active config has no end date createdAt: Date.now(), } ``` **Person → Backend Config (Who Created):** ```typescript { fromEntityId: personId, toEntityId: externalConnectionId, relationshipType: "created_by", metadata: { role: "platform_owner" | "org_owner", }, createdAt: Date.now(), } ``` ### 1.5 Events **Configuration Events:** ```typescript // settings_updated (existing consolidated event) { type: "settings_updated", actorId: personId, targetId: organizationId, timestamp: Date.now(), metadata: { settingType: "backend_provider", fromProvider: "convex", toProvider: "wordpress", configId: externalConnectionId, switchDuration: 28500, // milliseconds (<30 seconds target) } } // communication_event (for provider health checks) { type: "communication_event", actorId: systemAgentId, targetId: externalConnectionId, timestamp: Date.now(), metadata: { protocol: "http", messageType: "health_check", endpoint: "https://shocking-falcon-870.convex.cloud/_system/health", status: 200, responseTime: 142, // ms } } // entity_created (when new config added) { type: "entity_created", actorId: personId, targetId: externalConnectionId, timestamp: Date.now(), metadata: { entityType: "external_connection", platform: "wordpress", organizationId: orgId, } } // entity_archived (when config deactivated) { type: "entity_archived", actorId: personId, targetId: externalConnectionId, timestamp: Date.now(), metadata: { reason: "provider_switch", newConfigId: newExternalConnectionId, } } ``` ### 1.6 Knowledge **Configuration Labels:** ```typescript // Label knowledge items for provider configs { knowledgeType: "label", text: "backend:convex", labels: ["capability:realtime", "capability:auth", "capability:storage"], createdAt: Date.now(), } { knowledgeType: "label", text: "backend:wordpress", labels: ["capability:cms", "capability:rest_api", "capability:plugins"], createdAt: Date.now(), } // Link labels to external_connection entities via thingKnowledge { thingId: externalConnectionId, knowledgeId: labelId, role: "label", createdAt: Date.now(), } ``` **Why Labels:** - Enables capability-based provider discovery - Supports queries like "find backends with realtime capability" - Allows filtering in UI (show only providers with auth support) - Future-proof for new provider types --- ## 2. User Stories with Acceptance Criteria ### Story 2.1: Platform Owner Configures Default Provider **As a** platform owner (Anthony) **I want to** set the default backend provider via environment variables **So that** new organizations automatically use the configured backend **Acceptance Criteria:** - [ ] `.env.local` contains `BACKEND_PROVIDER=convex` variable - [ ] `.env.local` contains provider-specific credentials (e.g., `CONVEX_DEPLOYMENT_URL`) - [ ] Zod schema validates all required environment variables on startup - [ ] Missing or invalid config throws error with helpful message - [ ] Default provider is used when org has no custom config - [ ] Provider switch creates `settings_updated` event with platform_owner actorId **Example Config:** ```bash # .env.local BACKEND_PROVIDER=convex CONVEX_DEPLOYMENT_URL=https://shocking-falcon-870.convex.cloud CONVEX_DEPLOYMENT_NAME=prod:shocking-falcon-870 # Alternative for WordPress # BACKEND_PROVIDER=wordpress # WORDPRESS_URL=https://example.com/wp-json # WORDPRESS_USERNAME=admin # WORDPRESS_APP_PASSWORD=xxxx-xxxx-xxxx-xxxx ``` ### Story 2.2: Org Owner Switches Backend Provider **As an** org owner **I want to** switch my organization's backend from Convex to WordPress **So that** I can use my existing WordPress infrastructure **Acceptance Criteria:** - [ ] Org owner can access "Backend Settings" page - [ ] Page lists available provider types (Convex, WordPress, Notion, Supabase) - [ ] Org owner enters WordPress credentials in form - [ ] System validates credentials by testing connection - [ ] On success, system creates `external_connection` entity with encrypted credentials - [ ] System updates organization's `backendProvider` property - [ ] System creates `settings_updated` event - [ ] Switch completes in <30 seconds - [ ] Next API call uses new provider automatically **Security Requirements:** - [ ] API keys encrypted at rest using AES-256-GCM - [ ] Credentials never logged or exposed in events - [ ] Only org_owner and platform_owner can see credentials - [ ] Failed credential validation does not save config ### Story 2.3: Multi-Tenant Isolation **As the** platform **I want to** ensure each organization uses its own provider independently **So that** Org A using WordPress doesn't affect Org B using Convex **Acceptance Criteria:** - [ ] Each organization has independent `backendProvider` config - [ ] Provider registry maintains per-org provider instances - [ ] Queries automatically route to correct provider based on org context - [ ] Provider failure in Org A does not affect Org B - [ ] Event logs clearly show which org triggered which provider operation - [ ] Health checks run per organization, not globally **Example:** ```typescript // Org A uses Convex const orgAProvider = await getProviderForOrg("org_a"); await orgAProvider.query("entities", { type: "creator" }); // Convex // Org B uses WordPress const orgBProvider = await getProviderForOrg("org_b"); await orgBProvider.query("entities", { type: "creator" }); // WordPress REST API ``` ### Story 2.4: Provider Health Monitoring **As a** platform owner **I want to** monitor health of all configured providers **So that** I can detect and fix provider issues before users are affected **Acceptance Criteria:** - [ ] Health check runs every 5 minutes per active provider - [ ] Health check creates `communication_event` with status - [ ] Dashboard shows provider health status (online, degraded, offline) - [ ] Failed health checks trigger alerts - [ ] Health metrics include response time and error rate - [ ] Admins can manually trigger health check **Health Check Endpoint Examples:** - Convex: `/_system/health` - WordPress: `/wp-json/` - Notion: `/v1/users/me` - Supabase: `/rest/v1/` ### Story 2.5: Provider Factory Registration **As a** developer **I want to** register a new backend provider (e.g., Supabase) **So that** users can choose from more backend options **Acceptance Criteria:** - [ ] Provider implements `DataProvider` interface - [ ] Provider registered via `registerProvider()` function - [ ] Factory creates provider instance with validated config - [ ] New provider appears in UI dropdown automatically - [ ] Provider-specific Zod schema validates config - [ ] Provider factory lazy-initializes (only creates instance when needed) **Registration Pattern:** ```typescript // backend/convex/providers/supabase.ts export const SupabaseProviderFactory: ProviderFactory = { type: "supabase", create: (config: ProviderConfig) => new SupabaseProvider(config), validate: SupabaseConfigSchema, capabilities: ["auth", "storage", "realtime", "edge_functions"], }; // Register in registry registerProvider(SupabaseProviderFactory); ``` ### Story 2.6: Configuration Rollback **As an** org owner **I want to** rollback to previous provider if new one fails **So that** my organization isn't stuck with a broken backend **Acceptance Criteria:** - [ ] Organization properties store `previousProvider` reference - [ ] UI shows "Rollback to Convex" button when provider changed recently - [ ] Rollback creates new connection with `validFrom` = now - [ ] Old provider marked `inactive` but not deleted - [ ] Rollback completes in <10 seconds - [ ] Rollback creates `settings_updated` event with rollback metadata **Example Rollback:** ```typescript await rollbackProvider({ orgId: "org_123", targetConfigId: previousConfigId, // Points to old external_connection }); // Event logged: { type: "settings_updated", actorId: orgOwnerId, targetId: orgId, metadata: { settingType: "backend_provider_rollback", fromProvider: "wordpress", toProvider: "convex", reason: "connection_failure", } } ``` ### Story 2.7: Encrypted Credential Management **As a** security engineer **I want to** ensure API keys are never stored in plaintext **So that** compromised database doesn't expose credentials **Acceptance Criteria:** - [ ] Encryption key stored in environment variable `ENCRYPTION_KEY` - [ ] AES-256-GCM used for encryption (authenticated encryption) - [ ] Each credential has unique IV (initialization vector) - [ ] Auth tag validates integrity on decryption - [ ] Failed decryption throws error and logs security event - [ ] Credentials never appear in logs, events, or error messages - [ ] Encryption key rotation supported (future feature) **Encryption Flow:** ```typescript // On save const encrypted = await encrypt(apiKey, process.env.ENCRYPTION_KEY); await db.patch(externalConnectionId, { properties: { ...properties, apiKey: encrypted.ciphertext, apiKeyIv: encrypted.iv, apiKeyAuthTag: encrypted.authTag, }, }); // On load const decrypted = await decrypt({ ciphertext: properties.apiKey, iv: properties.apiKeyIv, authTag: properties.apiKeyAuthTag, key: process.env.ENCRYPTION_KEY, }); ``` ### Story 2.8: Configuration Validation **As a** developer **I want to** validate provider configs with Zod schemas **So that** invalid configs are caught early with helpful errors **Acceptance Criteria:** - [ ] Each provider type has dedicated Zod schema - [ ] Schema validates required fields (e.g., `CONVEX_DEPLOYMENT_URL`) - [ ] Schema validates field formats (URLs, API key patterns) - [ ] Validation errors show field name and expected format - [ ] Runtime config override validates before applying - [ ] Invalid config prevents provider initialization **Schema Examples:** ```typescript // Convex config schema export const ConvexConfigSchema = z.object({ type: z.literal("convex"), deploymentUrl: z.string().url(), deploymentName: z.string().regex(/^(dev|prod):.+$/), }); // WordPress config schema export const WordPressConfigSchema = z.object({ type: z.literal("wordpress"), baseUrl: z.string().url(), username: z.string().min(1), applicationPassword: z.string().regex(/^[a-zA-Z0-9 ]{24}$/), }); ``` ### Story 2.9: Provider Switching Performance **As an** org owner **I want** provider switches to complete quickly **So that** my team isn't waiting for backend changes **Acceptance Criteria:** - [ ] Provider switch completes in <30 seconds (99th percentile) - [ ] Switch includes: validate → encrypt → save → initialize → health check - [ ] Progress indicator shows switch steps in UI - [ ] Failed switch rolls back automatically - [ ] Switch duration logged in `settings_updated` event - [ ] Performance metrics tracked (P50, P95, P99) **Target Timings:** - Validate credentials: <2s - Encrypt and save: <1s - Initialize provider: <5s - Health check: <3s - Total: <15s (average), <30s (P99) ### Story 2.10: Configuration UI **As an** org owner **I want** a simple UI to manage backend configuration **So that** I don't need technical knowledge to switch providers **Acceptance Criteria:** - [ ] Settings page at `/org/[orgId]/settings/backend` - [ ] Current provider shown with status badge (online/offline) - [ ] Dropdown to select new provider type - [ ] Dynamic form based on selected provider (WordPress shows username field, Convex doesn't) - [ ] "Test Connection" button validates credentials before saving - [ ] Success notification shows "Provider switched to WordPress" - [ ] Error messages are user-friendly (not raw error logs) - [ ] Credentials masked in UI (show **\*\*\*** for API keys) **UI Flow:** 1. Org owner visits `/org/acme/settings/backend` 2. Sees "Current Provider: Convex (Online)" 3. Clicks "Change Provider" 4. Selects "WordPress" from dropdown 5. Form shows: Base URL, Username, Application Password fields 6. Fills in credentials 7. Clicks "Test Connection" → success notification 8. Clicks "Save and Switch" → progress indicator 9. After 15 seconds → "Successfully switched to WordPress" 10. Page shows "Current Provider: WordPress (Online)" --- ## 3. Implementation Steps (50 Steps) ### Phase 1: Schema and Types (Steps 1-10) **Step 1:** Define `ProviderConfig` type ```typescript // backend/convex/providers/types.ts export type ProviderType = "convex" | "wordpress" | "notion" | "supabase"; export interface ProviderConfig { type: ProviderType; url: string; credentials: Record<string, string>; options?: Record<string, any>; } export interface EncryptedCredentials { ciphertext: string; iv: string; authTag: string; } ``` **Step 2:** Add Zod schemas for each provider ```typescript // backend/convex/providers/schemas.ts import { z } from "zod"; export const ConvexConfigSchema = z.object({ type: z.literal("convex"), deploymentUrl: z.string().url(), deploymentName: z.string(), }); export const WordPressConfigSchema = z.object({ type: z.literal("wordpress"), baseUrl: z.string().url().endsWith("/wp-json"), username: z.string().min(1), applicationPassword: z.string().length(27), // WordPress app password format }); export const NotionConfigSchema = z.object({ type: z.literal("notion"), token: z.string().startsWith("secret_"), databaseId: z.string(), }); export const SupabaseConfigSchema = z.object({ type: z.literal("supabase"), url: z.string().url(), anonKey: z.string(), serviceKey: z.string().optional(), }); export const ProviderConfigSchema = z.discriminatedUnion("type", [ ConvexConfigSchema, WordPressConfigSchema, NotionConfigSchema, SupabaseConfigSchema, ]); ``` **Step 3:** Update organization schema ```typescript // backend/convex/schema.ts entities: defineTable({ type: v.string(), name: v.string(), properties: v.any(), // Includes backendProvider: { type, configId, switchedAt } status: v.optional(v.union(/* ... */)), createdAt: v.number(), updatedAt: v.number(), }); ``` **Step 4:** Add indexes for external_connection queries ```typescript // backend/convex/schema.ts entities: defineTable({ /* ... */ }) .index("by_type", ["type"]) .index("by_type_status", ["type", "status"]); // NEW: Fast external_connection queries ``` **Step 5:** Define `ProviderFactory` interface ```typescript // backend/convex/providers/factory.ts import { z } from "zod"; import type { DataProvider } from "./interface"; export interface ProviderFactory { type: ProviderType; create: (config: ProviderConfig) => Promise<DataProvider>; validate: z.ZodSchema; capabilities: string[]; healthCheckUrl?: (config: ProviderConfig) => string; } ``` **Step 6:** Create encryption utility ```typescript // backend/convex/lib/encryption.ts import { createCipheriv, createDecipheriv, randomBytes } from "crypto"; const ALGORITHM = "aes-256-gcm"; const KEY_LENGTH = 32; // 256 bits const IV_LENGTH = 16; // 128 bits const AUTH_TAG_LENGTH = 16; export async function encrypt( plaintext: string, key: string, ): Promise<EncryptedCredentials> { const keyBuffer = Buffer.from(key, "hex"); if (keyBuffer.length !== KEY_LENGTH) { throw new Error("Encryption key must be 32 bytes (64 hex chars)"); } const iv = randomBytes(IV_LENGTH); const cipher = createCipheriv(ALGORITHM, keyBuffer, iv); let ciphertext = cipher.update(plaintext, "utf8", "hex"); ciphertext += cipher.final("hex"); const authTag = cipher.getAuthTag(); return { ciphertext, iv: iv.toString("hex"), authTag: authTag.toString("hex"), }; } export async function decrypt( encrypted: EncryptedCredentials, key: string, ): Promise<string> { const keyBuffer = Buffer.from(key, "hex"); const ivBuffer = Buffer.from(encrypted.iv, "hex"); const authTagBuffer = Buffer.from(encrypted.authTag, "hex"); const decipher = createDecipheriv(ALGORITHM, keyBuffer, ivBuffer); decipher.setAuthTag(authTagBuffer); let plaintext = decipher.update(encrypted.ciphertext, "hex", "utf8"); plaintext += decipher.final("utf8"); return plaintext; } ``` **Step 7:** Create environment variable loader ```typescript // backend/convex/config/loader.ts import { ProviderConfigSchema } from "../providers/schemas"; export function loadProviderConfig(): ProviderConfig { const type = process.env.BACKEND_PROVIDER as ProviderType; if (!type) { throw new Error("BACKEND_PROVIDER environment variable required"); } const rawConfig = { type, ...(type === "convex" && { deploymentUrl: process.env.CONVEX_DEPLOYMENT_URL, deploymentName: process.env.CONVEX_DEPLOYMENT_NAME, }), ...(type === "wordpress" && { baseUrl: process.env.WORDPRESS_URL, username: process.env.WORDPRESS_USERNAME, applicationPassword: process.env.WORDPRESS_APP_PASSWORD, }), ...(type === "notion" && { token: process.env.NOTION_TOKEN, databaseId: process.env.NOTION_DATABASE_ID, }), ...(type === "supabase" && { url: process.env.SUPABASE_URL, anonKey: process.env.SUPABASE_ANON_KEY, serviceKey: process.env.SUPABASE_SERVICE_KEY, }), }; // Validate with Zod return ProviderConfigSchema.parse(rawConfig); } ``` **Step 8:** Add encryption key validation ```typescript // backend/convex/config/loader.ts (continued) export function validateEncryptionKey(): void { const key = process.env.ENCRYPTION_KEY; if (!key) { throw new Error("ENCRYPTION_KEY environment variable required"); } const keyBuffer = Buffer.from(key, "hex"); if (keyBuffer.length !== 32) { throw new Error("ENCRYPTION_KEY must be 32 bytes (64 hex characters)"); } } // Generate key helper (for docs) export function generateEncryptionKey(): string { return randomBytes(32).toString("hex"); } ``` **Step 9:** Create provider configuration service ```typescript // backend/convex/services/config/provider.ts import { Effect, Context } from "effect"; import type { Id } from "../../_generated/dataModel"; export class ProviderConfigService extends Context.Tag("ProviderConfigService")< ProviderConfigService, { getForOrganization: ( orgId: Id<"entities">, ) => Effect.Effect< ProviderConfig | null, ConfigNotFoundError | DecryptionError >; saveForOrganization: ( orgId: Id<"entities">, config: ProviderConfig, actorId: Id<"entities">, ) => Effect.Effect< Id<"entities">, // externalConnectionId ValidationError | EncryptionError | UnauthorizedError >; testConnection: ( config: ProviderConfig, ) => Effect.Effect< { success: true; responseTime: number }, ConnectionTestError >; } >() {} ``` **Step 10:** Define error types ```typescript // backend/convex/services/config/errors.ts export class ConfigNotFoundError { readonly _tag = "ConfigNotFoundError"; constructor(readonly orgId: string) {} } export class DecryptionError { readonly _tag = "DecryptionError"; constructor(readonly reason: string) {} } export class ValidationError { readonly _tag = "ValidationError"; constructor(readonly errors: z.ZodError) {} } export class EncryptionError { readonly _tag = "EncryptionError"; constructor(readonly reason: string) {} } export class UnauthorizedError { readonly _tag = "UnauthorizedError"; constructor( readonly userId: string, readonly requiredRole: string, ) {} } export class ConnectionTestError { readonly _tag = "ConnectionTestError"; constructor( readonly provider: string, readonly reason: string, readonly httpStatus?: number, ) {} } ``` ### Phase 2: Provider Factory (Steps 11-20) **Step 11:** Create provider registry ```typescript // backend/convex/providers/registry.ts const registeredProviders = new Map<ProviderType, ProviderFactory>(); export function registerProvider(factory: ProviderFactory): void { registeredProviders.set(factory.type, factory); } export function getProviderFactory( type: ProviderType, ): ProviderFactory | undefined { return registeredProviders.get(type); } export function listProviderTypes(): ProviderType[] { return Array.from(registeredProviders.keys()); } export function getProviderCapabilities(type: ProviderType): string[] { return registeredProviders.get(type)?.capabilities ?? []; } ``` **Step 12:** Implement Convex provider factory ```typescript // backend/convex/providers/convex/factory.ts import { ConvexProvider } from "./provider"; import { ConvexConfigSchema } from "../schemas"; export const ConvexProviderFactory: ProviderFactory = { type: "convex", create: async (config: ProviderConfig) => { const validated = ConvexConfigSchema.parse(config); return new ConvexProvider({ deploymentUrl: validated.deploymentUrl, deploymentName: validated.deploymentName, }); }, validate: ConvexConfigSchema, capabilities: ["auth", "realtime", "storage", "functions", "search"], healthCheckUrl: (config) => `${config.url}/_system/health`, }; // Auto-register registerProvider(ConvexProviderFactory); ``` **Step 13:** Implement WordPress provider factory ```typescript // backend/convex/providers/wordpress/factory.ts import { WordPressProvider } from "./provider"; import { WordPressConfigSchema } from "../schemas"; export const WordPressProviderFactory: ProviderFactory = { type: "wordpress", create: async (config: ProviderConfig) => { const validated = WordPressConfigSchema.parse(config); return new WordPressProvider({ baseUrl: validated.baseUrl, username: validated.username, applicationPassword: validated.applicationPassword, }); }, validate: WordPressConfigSchema, capabilities: ["cms", "rest_api", "plugins", "media_library"], healthCheckUrl: (config) => config.baseUrl, }; registerProvider(WordPressProviderFactory); ``` **Step 14:** Create provider instance cache ```typescript // backend/convex/providers/cache.ts const providerCache = new Map<string, DataProvider>(); export function getCachedProvider(orgId: string): DataProvider | undefined { return providerCache.get(orgId); } export function setCachedProvider(orgId: string, provider: DataProvider): void { providerCache.set(orgId, provider); } export function clearCachedProvider(orgId: string): void { providerCache.delete(orgId); } export function clearAllProviders(): void { providerCache.clear(); } ``` **Step 15:** Implement lazy provider initialization ```typescript // backend/convex/providers/initializer.ts import { Effect } from "effect"; import { getCachedProvider, setCachedProvider } from "./cache"; import { getProviderFactory } from "./registry"; export const initializeProvider = ( orgId: Id<"entities">, config: ProviderConfig, ): Effect.Effect<DataProvider, ProviderInitError> => Effect.gen(function* () { // Check cache first const cached = getCachedProvider(orgId); if (cached) return cached; // Get factory const factory = getProviderFactory(config.type); if (!factory) { return yield* Effect.fail( new ProviderInitError(`Unknown provider type: ${config.type}`), ); } // Create provider instance const provider = yield* Effect.tryPromise({ try: () => factory.create(config), catch: (error) => new ProviderInitError(`Failed to initialize ${config.type}: ${error}`), }); // Cache for future use setCachedProvider(orgId, provider); return provider; }); ``` **Step 16:** Create provider switcher service ```typescript // backend/convex/services/config/switcher.ts export const switchProvider = ( orgId: Id<"entities">, newConfig: ProviderConfig, actorId: Id<"entities">, ): Effect.Effect< { switchDuration: number; configId: Id<"entities"> }, ValidationError | EncryptionError | UnauthorizedError | ConnectionTestError > => Effect.gen(function* () { const startTime = Date.now(); // 1. Validate config const factory = getProviderFactory(newConfig.type); if (!factory) { return yield* Effect.fail( new ValidationError(`Unknown provider: ${newConfig.type}`), ); } const validated = factory.validate.parse(newConfig); // 2. Test connection yield* testConnection(validated); // 3. Encrypt credentials const encrypted = yield* encryptCredentials(validated); // 4. Save config as external_connection const configId = yield* saveConfig(orgId, encrypted, actorId); // 5. Update organization yield* updateOrgProvider(orgId, newConfig.type, configId); // 6. Clear cached provider clearCachedProvider(orgId); // 7. Log event const switchDuration = Date.now() - startTime; yield* logProviderSwitch(orgId, actorId, newConfig.type, switchDuration); return { switchDuration, configId }; }); ``` **Step 17:** Implement connection test ```typescript // backend/convex/services/config/tester.ts export const testConnection = ( config: ProviderConfig, ): Effect.Effect< { success: true; responseTime: number }, ConnectionTestError > => Effect.gen(function* () { const factory = getProviderFactory(config.type); if (!factory?.healthCheckUrl) { // No health check available, assume success return { success: true, responseTime: 0 }; } const url = factory.healthCheckUrl(config); const startTime = Date.now(); const response = yield* Effect.tryPromise({ try: () => fetch(url, { method: "GET", timeout: 5000 }), catch: (error) => new ConnectionTestError( config.type, `Health check failed: ${error}`, undefined, ), }); if (!response.ok) { return yield* Effect.fail( new ConnectionTestError( config.type, `Health check returned ${response.status}`, response.status, ), ); } const responseTime = Date.now() - startTime; return { success: true, responseTime }; }); ``` **Step 18:** Implement credential encryption helper ```typescript // backend/convex/services/config/encryption.ts export const encryptCredentials = ( config: ProviderConfig, ): Effect.Effect<EncryptedConfig, EncryptionError> => Effect.gen(function* () { const key = process.env.ENCRYPTION_KEY; if (!key) { return yield* Effect.fail( new EncryptionError("ENCRYPTION_KEY not configured"), ); } const credentials = extractCredentials(config); const encrypted: Record<string, EncryptedCredentials> = {}; for (const [field, value] of Object.entries(credentials)) { encrypted[field] = yield* Effect.tryPromise({ try: () => encrypt(value, key), catch: (error) => new EncryptionError(`Failed to encrypt ${field}: ${error}`), }); } return { ...config, encryptedCredentials: encrypted }; }); function extractCredentials(config: ProviderConfig): Record<string, string> { switch (config.type) { case "convex": return {}; // Convex uses public URLs case "wordpress": return { applicationPassword: config.applicationPassword }; case "notion": return { token: config.token }; case "supabase": return { anonKey: config.anonKey, ...(config.serviceKey && { serviceKey: config.serviceKey }), }; default: return {}; } } ``` **Step 19:** Implement config save mutation ```typescript // backend/convex/mutations/config.ts export const saveProviderConfig = mutation({ args: { orgId: v.id("entities"), config: v.any(), // Validated by service layer }, handler: async (ctx, args) => { // Get actor const identity = await ctx.auth.getUserIdentity(); if (!identity) throw new Error("Not authenticated"); const actor = await ctx.db .query("entities") .filter((q) => q.eq(q.field("properties.email"), identity.email)) .first(); if (!actor) throw new Error("User not found"); // Use Effect service return await Effect.runPromise( switchProvider(args.orgId, args.config, actor._id).pipe( Effect.provide(ConfigServiceLayer), ), ); }, }); ``` **Step 20:** Register all provider factories on startup ```typescript // backend/convex/providers/index.ts import "./convex/factory"; import "./wordpress/factory"; import "./notion/factory"; import "./supabase/factory"; export * from "./registry"; export * from "./factory"; export * from "./initializer"; ``` ### Phase 3: Multi-Tenant Provider Access (Steps 21-30) **Step 21:** Create organization context service ```typescript // backend/convex/services/context/organization.ts export class OrganizationContext extends Context.Tag("OrganizationContext")< OrganizationContext, { getCurrentOrg: () => Effect.Effect<Id<"entities">, OrgNotFoundError>; getOrgProvider: ( orgId: Id<"entities">, ) => Effect.Effect<DataProvider, ConfigNotFoundError | ProviderInitError>; } >() {} ``` **Step 22:** Implement provider resolver ```typescript // backend/convex/services/context/resolver.ts export const resolveProviderForOrg = ( orgId: Id<"entities">, ): Effect.Effect<DataProvider, ConfigNotFoundError | ProviderInitError> => Effect.gen(function* () { // Check cache const cached = getCachedProvider(orgId); if (cached) return cached; // Load org const org = yield* Effect.tryPromise(() => db.get(orgId)); if (!org) { return yield* Effect.fail(new ConfigNotFoundError(orgId)); } // Check if org has custom provider const customConfig = org.properties.backendProvider; if (customConfig?.configId) { const config = yield* loadExternalConnection(customConfig.configId); const decrypted = yield* decryptConfig(config); return yield* initializeProvider(orgId, decrypted); } // Fall back to default provider const defaultConfig = loadProviderConfig(); return yield* initializeProvider(orgId, defaultConfig); }); ``` **Step 23:** Implement config loader from external_connection ```typescript // backend/convex/services/config/loader.ts export const loadExternalConnection = ( configId: Id<"entities">, ): Effect.Effect<ProviderConfig, ConfigNotFoundError | DecryptionError> => Effect.gen(function* () { const config = yield* Effect.tryPromise(() => db.get(configId)); if (!config || config.type !== "external_connection") { return yield* Effect.fail(new ConfigNotFoundError(configId)); } if (config.status !== "active") { return yield* Effect.fail( new ConfigNotFoundError(`Config ${configId} is inactive`), ); } // Decrypt credentials const decrypted = yield* decryptConfigCredentials(config.properties); return { type: config.properties.platform, ...config.properties.config, ...decrypted, }; }); ``` **Step 24:** Create scoped query wrapper ```typescript // backend/convex/queries/scoped.ts export const createScopedQuery = <Args, Result>( fn: (ctx: QueryCtx, args: Args, provider: DataProvider) => Promise<Result>, ) => { return query({ handler: async (ctx, args) => { // Get user identity const identity = await ctx.auth.getUserIdentity(); if (!identity) throw new Error("Not authenticated"); // Get user's organization const user = await ctx.db .query("entities") .filter((q) => q.eq(q.field("properties.email"), identity.email)) .first(); if (!user?.properties.organizationId) { throw new Error("User has no organization"); } // Resolve provider for org const provider = await Effect.runPromise( resolveProviderForOrg(user.properties.organizationId).pipe( Effect.provide(ConfigServiceLayer), ), ); // Execute query with scoped provider return await fn(ctx, args, provider); }, }); }; ``` **Step 25:** Example scoped query usage ```typescript // backend/convex/queries/entities.ts export const list = createScopedQuery( async (ctx, args: { type: string }, provider) => { // Provider is automatically scoped to user's org return await provider.query("entities", { where: { type: args.type }, }); }, ); ``` **Step 26:** Create scoped mutation wrapper ```typescript // backend/convex/mutations/scoped.ts export const createScopedMutation = <Args, Result>( fn: ( ctx: MutationCtx, args: Args, provider: DataProvider, orgId: Id<"entities">, ) => Promise<Result>, ) => { return mutation({ handler: async (ctx, args) => { const identity = await ctx.auth.getUserIdentity(); if (!identity) throw new Error("Not authenticated"); const user = await ctx.db .query("entities") .filter((q) => q.eq(q.field("properties.email"), identity.email)) .first(); if (!user?.properties.organizationId) { throw new Error("User has no organization"); } const provider = await Effect.runPromise( resolveProviderForOrg(user.properties.organizationId).pipe( Effect.provide(ConfigServiceLayer), ), ); return await fn(ctx, args, provider, user.properties.organizationId); }, }); }; ``` **Step 27:** Implement provider health check ```typescript // backend/convex/services/health/checker.ts export const checkProviderHealth = ( configId: Id<"entities">, ): Effect.Effect< { status: "online" | "offline"; responseTime: number }, ConnectionTestError > => Effect.gen(function* () { const config = yield* loadExternalConnection(configId); const result = yield* testConnection(config); return { status: "online", responseTime: result.responseTime, }; }); ``` **Step 28:** Create scheduled health check action ```typescript // backend/convex/crons/health.ts export const healthCheckCron = cronJobs.interval( "provider-health-check", { minutes: 5 }, internal.health.checkAllProviders, ); export const checkAllProviders = internalAction({ handler: async (ctx) => { // Get all active external_connection entities const configs = await ctx.runQuery( internal.queries.config.listActiveConfigs, ); for (const config of configs) { try { const health = await Effect.runPromise( checkProviderHealth(config._id).pipe( Effect.provide(HealthCheckLayer), ), ); // Log health event await ctx.runMutation(internal.mutations.events.logHealthCheck, { configId: config._id, status: health.status, responseTime: health.responseTime, }); } catch (error) { // Log failure event await ctx.runMutation(internal.mutations.events.logHealthCheck, { configId: config._id, status: "offline", error: String(error), }); } } }, }); ``` **Step 29:** Implement provider usage tracking ```typescript // backend/convex/services/usage/tracker.ts export const trackProviderUsage = ( orgId: Id<"entities">, operation: string, duration: number, ): Effect.Effect<void, never> => Effect.gen(function* () { // Create usage event yield* Effect.tryPromise(() => db.insert("events", { type: "communication_event", actorId: orgId, targetId: undefined, timestamp: Date.now(), metadata: { protocol: "provider_usage", operation, duration, }, }), ); // Update organization usage metrics (if needed) // This enables billing based on provider operations }); ``` **Step 30:** Add provider metrics query ```typescript // backend/convex/queries/metrics.ts export const getProviderMetrics = query({ args: { orgId: v.id("entities"), days: v.number() }, handler: async (ctx, args) => { const since = Date.now() - args.days * 24 * 60 * 60 * 1000; const events = await ctx.db .query("events") .withIndex("by_timestamp", (q) => q.gte("timestamp", since)) .filter((q) => q.and( q.eq(q.field("type"), "communication_event"), q.eq(q.field("actorId"), args.orgId), q.eq(q.field("metadata.protocol"), "provider_usage"), ), ) .collect(); return { totalOperations: events.length, averageDuration: events.reduce((sum, e) => sum + (e.metadata.duration || 0), 0) / events.length, operationsByType: groupBy(events, (e) => e.metadata.operation), }; }, }); ``` ### Phase 4: Frontend Integration (Steps 31-40) **Step 31:** Create provider config form component ```typescript // frontend/src/components/features/config/ProviderConfigForm.tsx import { useState } from "react"; import { useMutation } from "convex/react"; import { api } from "@/convex/_generated/api"; import { Button } from "@/components/ui/button"; import { Input } from "@/components/ui/input"; import { Select } from "@/components/ui/select"; import { Card } from "@/components/ui/card"; export function ProviderConfigForm({ orgId }: { orgId: Id<"entities"> }) { const [providerType, setProviderType] = useState<ProviderType>("convex"); const [config, setConfig] = useState<Record<string, string>>({}); const [testing, setTesting] = useState(false); const saveConfig = useMutation(api.mutations.config.saveProviderConfig); const testConnection = useMutation(api.mutations.config.testConnection); const handleTest = async () => { setTesting(true); try { const result = await testConnection({ orgId, config: { type: providerType, ...config } }); alert(`Connection successful! Response time: ${result.responseTime}ms`); } catch (error) { alert(`Connection failed: ${error.message}`); } finally { setTesting(false); } }; const handleSave = async () => { try { await saveConfig({ orgId, config: { type: providerType, ...config } }); alert("Provider configuration saved successfully!"); } catch (error) { alert(`Failed to save configuration: ${error.message}`); } }; return ( <Card className="p-6"> <h2 className="text-2xl font-bold mb-4">Backend Provider Configuration</h2> <div className="space-y-4"> <Select value={providerType} onValueChange={(value) => setProviderType(value as ProviderType)} > <option value="convex">Convex</option> <option value="wordpress">WordPress</option> <option value="notion">Notion</option> <option value="supabase">Supabase</option> </Select> {providerType === "wordpress" && ( <> <Input placeholder="WordPress URL" value={config.baseUrl || ""} onChange={(e) => setConfig({ ...config, baseUrl: e.target.value })} /> <Input placeholder="Username" value={config.username || ""} onChange={(e) => setConfig({ ...config, username: e.target.value })} /> <Input type="password" placeholder="Application Password" value={config.applicationPassword || ""} onChange={(e) => setConfig({ ...config, applicationPassword: e.target.value })} /> </> )} {providerType === "notion" && ( <> <Input type="password" placeholder="Notion Token" value={config.token || ""} onChange={(e) => setConfig({ ...config, token: e.target.value })} /> <Input placeholder="Database ID" value={config.databaseId || ""} onChange={(e) => setConfig({ ...config, databaseId: e.target.value })} /> </> )} <div className="flex gap-2"> <Button onClick={handleTest} disabled={testing}> {testing ? "Testing..." : "Test Connection"} </Button> <Button onClick={handleSave} variant="primary"> Save Configuration </Button> </div> </div> </Card> ); } ``` **Step 32:** Create provider status badge ```typescript // frontend/src/components/features/config/ProviderStatusBadge.tsx import { useQuery } from "convex/react"; import { api } from "@/convex/_generated/api"; import { Badge } from "@/components/ui/badge"; export function ProviderStatusBadge({ orgId }: { orgId: Id<"entities"> }) { const status = useQuery(api.queries.config.getProviderStatus, { orgId }); if (status === undefined) return <Badge>Loading...</Badge>; return ( <Badge variant={status.online ? "success" : "destructive"}> {status.type} ({status.online ? "Online" : "Offline"}) </Badge> ); } ``` **Step 33:** Create settings page ```typescript // frontend/src/pages/org/[orgId]/settings/backend.astro --- import Layout from "@/layouts/Layout.astro"; import ProviderConfigForm from "@/components/features/config/ProviderConfigForm"; import ProviderStatusBadge from "@/components/features/config/ProviderStatusBadge"; const { orgId } = Astro.params; --- <Layout title="Backend Settings"> <div class="container mx-auto py-8"> <div class="flex items-center justify-between mb-6"> <h1 class="text-3xl font-bold">Backend Provider Settings</h1> <ProviderStatusBadge client:load orgId={orgId} /> </div> <ProviderConfigForm client:load orgId={orgId} /> </div> </Layout> ``` **Step 34:** Add provider metrics dashboard ```typescript // frontend/src/components/features/config/ProviderMetrics.tsx import { useQuery } from "convex/react"; import { api } from "@/convex/_generated/api"; import { Card } from "@/components/ui/card"; import { LineChart, Line, XAxis, YAxis, Tooltip } from "recharts"; export function ProviderMetrics({ orgId }: { orgId: Id<"entities"> }) { const metrics = useQuery(api.queries.metrics.getProviderMetrics, { orgId, days: 7, }); if (metrics === undefined) return <div>Loading metrics...</div>; return ( <Card className="p-6"> <h3 className="text-xl font-bold mb-4">Provider Usage (Last 7 Days)</h3> <div className="grid grid-cols-2 gap-4 mb-6"> <div> <p className="text-sm text-gray-500">Total Operations</p> <p className="text-2xl font-bold">{metrics.totalOperations}</p> </div> <div> <p className="text-sm text-gray-500">Average Response Time</p> <p className="text-2xl font-bold">{metrics.averageDuration.toFixed(0)}ms</p> </div> </div> <LineChart width={600} height={300} data={metrics.operationsByDay}> <XAxis dataKey="day" /> <YAxis /> <Tooltip /> <Line type="monotone" dataKey="count" stroke="#8884d8" /> </LineChart> </Card> ); } ``` **Step 35:** Implement rollback UI ```typescript // frontend/src/components/features/config/ProviderRollback.tsx import { useMutation, useQuery } from "convex/react"; import { api } from "@/convex/_generated/api"; import { Button } from "@/components/ui/button"; import { Alert } from "@/components/ui/alert"; export function ProviderRollback({ orgId }: { orgId: Id<"entities"> }) { const org = useQuery(api.queries.entities.get, { id: orgId }); const rollback = useMutation(api.mutations.config.rollbackProvider); if (!org?.properties.backendProvider?.previousProvider) { return null; // No previous provider to rollback to } const handleRollback = async () => { if (!confirm("Are you sure you want to rollback to the previous provider?")) { return; } try { await rollback({ orgId }); alert("Successfully rolled back to previous provider!"); } catch (error) { alert(`Rollback failed: ${error.message}`); } }; return ( <Alert variant="warning" className="mt-4"> <p className="mb-2"> You recently switched from {org.properties.backendProvider.previousProvider}. </p> <Button onClick={handleRollback} variant="outline"> Rollback to {org.properties.backendProvider.previousProvider} </Button> </Alert> ); } ``` **Step 36:** Add provider switch confirmation dialog ```typescript // frontend/src/components/fe