oneie
Version:
Build apps, websites, and AI agents in English. Zero-interaction setup for AI agents (Claude Code, Cursor, Windsurf). Download to your computer, run in the cloud, deploy to the edge. Open source and free forever.
1,590 lines (1,259 loc) • 45 kB
Markdown
title: Architecture Any Backend
dimension: knowledge
category: architecture-any-backend.md
tags: ai, architecture, backend, convex, frontend, groups, ontology, protocol, system-design
related_dimensions: connections, events, groups, people, things
scope: global
created: 2025-11-03
updated: 2025-11-03
version: 1.0.0
ai_context: |
This document is part of the knowledge dimension in the architecture-any-backend.md category.
Location: one/knowledge/architecture-any-backend.md
Purpose: Documents backend-agnostic architecture: use any database
Related dimensions: connections, events, groups, people, things
For AI agents: Read this to understand architecture any backend.
# Backend-Agnostic Architecture: Use Any Database
**Version:** 3.0
**Status:** ✅ Production Ready
**Last Updated:** 2025-11-03
## Overview
The ONE Platform implements a **backend-agnostic architecture** that allows any group to choose ANY backend database or CMS without changing a single line of frontend code. Switch from Convex to WordPress to Notion with **one configuration change**.
This architecture proves that the 6-dimension ontology is truly protocol-agnostic and platform-independent.
**Critical:** This document aligns with the 5-table implementation (groups table + 4 other tables with groupId scoping) as specified in the main architecture.
## The 6-Dimension Ontology: 5-Table Implementation
The ONE Platform uses a **6-dimension ontology** implemented with **5 database tables**:
### Database Schema
```typescript
// TABLE 1: groups (Dimension 1 - Multi-tenancy boundary)
{
_id: Id<"groups">,
name: string,
type: "friend_circle" | "business" | "community" | "dao" | "government" | "organization",
parentGroupId?: Id<"groups">, // Hierarchical nesting (groups contain groups)
properties: {
plan?: "starter" | "pro" | "enterprise",
backendProvider?: "convex" | "wordpress" | "notion" | "supabase",
// ... type-specific fields
},
status: "draft" | "active" | "archived",
createdAt: number,
updatedAt: number
}
// TABLE 2: things (Dimension 3 - All nouns)
{
_id: Id<"things">,
type: ThingType, // 66 types
name: string,
groupId: Id<"groups">, // SCOPED TO GROUP
properties: any, // Flexible type-specific data
status: "draft" | "active" | "published" | "archived",
createdAt: number,
updatedAt: number
}
// TABLE 3: connections (Dimension 4 - All relationships)
{
_id: Id<"connections">,
fromThingId?: Id<"things">,
toThingId?: Id<"things">,
fromPersonId?: Id<"people">,
toPersonId?: Id<"people">,
relationshipType: ConnectionType, // 25 types
groupId: Id<"groups">, // SCOPED TO GROUP
metadata: any,
createdAt: number
}
// TABLE 4: events (Dimension 5 - All actions)
{
_id: Id<"events">,
type: EventType, // 67 types
actorId: Id<"people">, // Person who did it (REQUIRED)
targetId?: Id<"things"> | Id<"people"> | Id<"connections">,
groupId: Id<"groups">, // SCOPED TO GROUP
metadata: any,
timestamp: number
}
// TABLE 5: knowledge (Dimension 6 - AI understanding)
{
_id: Id<"knowledge">,
type: "embedding" | "label" | "category" | "tag",
text?: string,
embedding?: number[], // Vector for semantic search
embeddingModel?: string,
sourceThingId?: Id<"things">,
sourcePersonId?: Id<"people">,
groupId: Id<"groups">, // SCOPED TO GROUP
labels?: string[],
metadata?: any,
createdAt: number,
updatedAt: number
}
```
### Where is Dimension 2 (People)?
**People are represented as things** with `type: 'creator'` and `properties.role` field, OR as a separate `people` table depending on backend requirements. The main architecture uses things, but some providers (like WordPress) may use a separate users table.
**4 Roles:**
- `platform_owner` - Full platform access
- `org_owner` - Group owner (can manage group settings)
- `org_user` - Group member (can use features)
- `customer` - End user (limited access)
### Multi-Tenancy Via groupId
**Every dimension (except groups themselves) includes groupId for data scoping:**
- Things → `groupId: Id<"groups">`
- Connections → `groupId: Id<"groups">`
- Events → `groupId: Id<"groups">`
- Knowledge → `groupId: Id<"groups">`
**This is how multi-tenancy works:** All queries filter by groupId, ensuring perfect data isolation between groups.
### Hierarchical Groups
Groups can contain other groups via `parentGroupId`:
```
Group: "Acme Corp" (organization)
└─ Group: "Engineering Dept" (business)
└─ Group: "Frontend Team" (friend_circle)
```
This enables infinite nesting from friend circles (2 people) to governments (billions of people).
## Architecture Layers
The system uses **5 distinct layers** that separate concerns and enable backend flexibility:
```
┌─────────────────────────────────────────────────────────┐
│ LAYER 5: UI COMPONENTS (React 19) │
│ - Auth forms, dashboards, data displays │
│ - Uses hooks only, never direct backend calls │
└────────────────────┬────────────────────────────────────┘
│
↓ (React hooks)
┌─────────────────────────────────────────────────────────┐
│ LAYER 4: REACT HOOKS │
│ - useThings(), useConnections(), useEvents() │
│ - useLogin(), useSignup(), useCurrentUser() │
│ - Loading states, error handling, optimistic updates │
└────────────────────┬────────────────────────────────────┘
│
↓ (Effect.ts)
┌─────────────────────────────────────────────────────────┐
│ LAYER 3: EFFECT.TS SERVICES │
│ - ThingService, ConnectionService, EventService │
│ - Pure business logic, backend-agnostic │
│ - Validation, authorization, type-specific rules │
└────────────────────┬────────────────────────────────────┘
│
↓ (DataProvider interface)
┌─────────────────────────────────────────────────────────┐
│ LAYER 2: DATAPROVIDER INTERFACE │
│ - Standard CRUD operations for 6 dimensions │
│ - Type-safe errors with tagged unions │
│ - Real-time subscription support │
└────────────────────┬────────────────────────────────────┘
│
↓ (Provider implementations)
┌─────────────────────────────────────────────────────────┐
│ LAYER 1: BACKEND PROVIDERS │
│ - ConvexProvider → Convex real-time database │
│ - NotionProvider → Notion API (pages + relations) │
│ - WordPressProvider → WordPress REST API + ACF │
│ - [Future: Supabase, Firebase, Airtable, etc.] │
└─────────────────────────────────────────────────────────┘
```
**Key Principle:** Each layer only knows about the layer directly below it. Components don't know what backend is being used.
## Layer 1: DataProvider Interface
### The Contract
The `DataProvider` interface defines ALL operations for the 6-dimension ontology:
```typescript
// frontend/src/providers/DataProvider.ts
export interface DataProvider {
// ===== GROUPS (Dimension 1) =====
// Hierarchical containers that partition all other dimensions
// Scales from friend circles (2 people) to governments (billions)
groups: {
get: (id: string) => Effect.Effect<Group, GroupNotFoundError>;
list: (options?: ListOptions) => Effect.Effect<Group[], QueryError>;
create: (input: CreateGroupInput) => Effect.Effect<string, GroupCreateError>;
update: (id: string, input: UpdateGroupInput) => Effect.Effect<void, GroupUpdateError>;
delete: (id: string) => Effect.Effect<void, GroupDeleteError>;
getChildren: (parentId: string) => Effect.Effect<Group[], QueryError>; // Hierarchical nesting
};
// ===== PEOPLE (Dimension 2) =====
// Authorization & governance - who can do what
people: {
getCurrentUser: () => Effect.Effect<User, UserNotFoundError>;
getByRole: (role: Role) => Effect.Effect<User[], QueryError>;
updateRole: (userId: string, role: Role) => Effect.Effect<void, UpdateError>;
inviteMember: (input: InviteInput) => Effect.Effect<void, InviteError>;
};
// ===== THINGS (Dimension 3) =====
// All nouns - users, agents, content, tokens, courses (66 types)
// Every thing belongs to a group (scoped via groupId)
things: {
get: (id: string) => Effect.Effect<Thing, ThingNotFoundError>;
list: (options: ListThingsOptions) => Effect.Effect<Thing[], QueryError>;
create: (input: CreateThingInput) => Effect.Effect<string, ThingCreateError>;
update: (id: string, input: UpdateThingInput) => Effect.Effect<void, ThingUpdateError>;
delete: (id: string) => Effect.Effect<void, ThingDeleteError>;
};
// ===== CONNECTIONS (Dimension 4) =====
// All relationships - owns, follows, purchased, enrolled_in (25 types)
// Every connection belongs to a group (scoped via groupId)
connections: {
get: (id: string) => Effect.Effect<Connection, ConnectionNotFoundError>;
list: (options: ListConnectionsOptions) => Effect.Effect<Connection[], QueryError>;
create: (input: CreateConnectionInput) => Effect.Effect<string, ConnectionCreateError>;
delete: (id: string) => Effect.Effect<void, ConnectionDeleteError>;
};
// ===== EVENTS (Dimension 5) =====
// All actions and state changes - created, updated, purchased (67 types)
// Every event belongs to a group (scoped via groupId)
events: {
create: (input: CreateEventInput) => Effect.Effect<string, EventCreateError>;
list: (options: ListEventsOptions) => Effect.Effect<Event[], QueryError>;
getAuditTrail: (thingId: string) => Effect.Effect<Event[], QueryError>;
};
// ===== KNOWLEDGE (Dimension 6) =====
// Labels, embeddings, and semantic search for AI
// Every knowledge entry belongs to a group (scoped via groupId)
knowledge: {
search: (options: SearchKnowledgeOptions) => Effect.Effect<Knowledge[], QueryError>;
create: (input: CreateKnowledgeInput) => Effect.Effect<string, KnowledgeCreateError>;
linkToThing: (thingId: string, knowledgeId: string) => Effect.Effect<void, LinkError>;
};
// ===== AUTH =====
auth: {
login: (args: LoginArgs) => Effect.Effect<AuthResult, AuthError>;
signup: (args: SignupArgs) => Effect.Effect<AuthResult, AuthError>;
logout: () => Effect.Effect<void, AuthError>;
getCurrentUser: () => Effect.Effect<User, AuthError>;
// ... 8 more auth methods
};
}
```
### Why Effect.ts?
Every operation returns an `Effect.Effect<Success, Error>` type. This provides:
1. **Type-safe errors**: Compiler catches unhandled errors
2. **Composability**: Chain operations together
3. **Dependency injection**: Services declare what they need
4. **Testability**: Mock providers easily
5. **Pure functions**: No side effects until `Effect.runPromise()`
**Example:**
```typescript
// Type-safe error handling
const program = Effect.gen(function* () {
const thing = yield* provider.things.get(id); // Can fail with ThingNotFoundError
const connections = yield* provider.connections.list({ fromEntityId: id }); // Can fail with QueryError
return { thing, connections };
});
// Compiler forces you to handle both error types
await Effect.runPromise(
program.pipe(
Effect.catchTag('ThingNotFoundError', (error) => {
console.error('Thing not found:', error.message);
return Effect.succeed(null);
}),
Effect.catchTag('QueryError', (error) => {
console.error('Query failed:', error.message);
return Effect.succeed(null);
})
)
);
```
## Layer 2: Backend Providers
### Three Providers Implemented
#### 1. ConvexProvider (Default)
Wraps the Convex SDK with the DataProvider interface.
**File:** `frontend/src/providers/ConvexProvider.ts`
**How it works:**
```typescript
export function createConvexProvider(config: ConvexProviderConfig): DataProvider {
const { client } = config;
return {
things: {
get: (id) =>
Effect.tryPromise({
try: () => client.query(api.queries.things.get, { id }),
catch: (error) => new ThingNotFoundError(id, String(error)),
}),
list: (options) =>
Effect.tryPromise({
try: () => client.query(api.queries.things.list, options),
catch: (error) => new QueryError('Failed to list things', error),
}),
create: (input) =>
Effect.tryPromise({
try: () => client.mutation(api.mutations.things.create, input),
catch: (error) => new ThingCreateError('Failed to create thing', error),
}),
// ... more operations
},
// ... other dimensions
};
}
```
**Features:**
- Real-time subscriptions via Convex
- <5ms overhead per operation
- Zero backend changes required
- Supports all 6 dimensions natively
#### 2. NotionProvider
Maps Notion databases and pages to the 6-dimension ontology.
**File:** `frontend/src/providers/notion/NotionProvider.ts` (1,200+ lines)
**Mapping Strategy:**
| Ontology | Notion |
|----------|--------|
| Groups | Page property: `groupId` or separate database |
| People | Person properties + permissions |
| Things | Pages in databases |
| Connections | Relation properties |
| Events | Page updates (delegated to Convex) |
| Knowledge | Database properties as labels |
**Example:**
```typescript
// Notion Page → Thing
const notionPage = {
id: "abc123",
properties: {
Name: { title: [{ text: { content: "My Course" } }] },
Status: { select: { name: "Published" } },
Price: { number: 99 },
}
};
// Becomes:
const thing = {
_id: "notion_abc123",
type: "course",
name: "My Course",
status: "published",
properties: {
price: 99,
notionId: "abc123"
}
};
```
**ID Format:** `notion_<32-char-hex>` (prevents collisions)
**Hybrid Approach:** Notion handles things/connections, Convex handles events/knowledge (for backends without native support)
#### 3. WordPressProvider
Maps WordPress posts and custom post types to the ontology.
**File:** `frontend/src/providers/wordpress/WordPressProviderEnhanced.ts` (1,100+ lines)
**Mapping Strategy:**
| Ontology | WordPress |
|----------|-----------|
| Groups | Custom `wp_groups` table (with parentGroupId) |
| People | WordPress users + roles |
| Things | Posts + Custom Post Types |
| Connections | Custom `wp_connections` table |
| Events | Custom `wp_events` table |
| Knowledge | Custom `wp_knowledge` table |
**Requires:** Custom WordPress plugin `one-platform-connector` (creates tables + REST endpoints)
**Example:**
```typescript
// WordPress Post → Thing
const wpPost = {
id: 123,
title: "My Course",
status: "publish",
type: "course",
meta: {
_group_id: "group_xyz", // Scoped to group
_properties: JSON.stringify({ price: 99 })
}
};
// Becomes:
const thing = {
_id: "wp_course_123",
type: "course",
name: "My Course",
status: "published",
groupId: "group_xyz", // All things scoped to groups
properties: { price: 99 }
};
```
**ID Format:** `wp_<post-type>_<id>` (e.g., `wp_course_123`, `wp_lesson_456`)
## Layer 3: Effect.ts Services
### Business Logic Layer
Services contain pure business logic that works with ANY backend:
**File:** `frontend/src/services/ThingService.ts`
```typescript
import { Effect, Layer } from 'effect';
import type { DataProvider } from '@/providers/DataProvider';
export class ThingService extends Effect.Service<ThingService>()('ThingService', {
effect: Effect.gen(function* () {
const provider = yield* DataProviderService;
return {
// Create thing with validation
create: (input: CreateThingInput) =>
Effect.gen(function* () {
// 1. Validate thing type
if (!VALID_THING_TYPES.includes(input.type)) {
return yield* Effect.fail(new InvalidThingTypeError(input.type));
}
// 2. Validate type-specific rules
if (input.type === 'course' && !input.properties.title) {
return yield* Effect.fail(new ValidationError('Course requires title'));
}
// 3. Create via provider
const thingId = yield* provider.things.create(input);
// 4. Log creation event
yield* provider.events.create({
type: 'thing_created',
thingId,
actorId: input.actorId,
timestamp: Date.now(),
metadata: { thingType: input.type }
});
return thingId;
}),
// Get thing with related data
getWithConnections: (id: string) =>
Effect.gen(function* () {
const thing = yield* provider.things.get(id);
const connections = yield* provider.connections.list({ fromEntityId: id });
return { thing, connections };
}),
// More operations...
};
}),
dependencies: [DataProviderService.Default],
}) {}
```
### Key Features
1. **Type-Specific Validation:** Each of the 66 thing types has custom rules
2. **Authorization:** Checks roles before allowing operations
3. **Event Logging:** Automatically logs all state changes
4. **Backend-Agnostic:** Never imports Convex/Notion/WordPress directly
5. **Composable:** Services can call other services
### All 7 Services
| Service | Purpose | LOC |
|---------|---------|-----|
| ThingService | Manages 66 entity types | 246 |
| ConnectionService | Manages 25 relationship types | 300 |
| EventService | Manages 67 event types | 294 |
| KnowledgeService | RAG, labels, embeddings | 299 |
| OrganizationService | Multi-tenancy, quotas | 409 |
| PeopleService | Authorization, roles | 444 |
| ConfigService | Provider configuration | 511 |
**Total:** 2,503 lines of pure business logic
**Note:** These services work with ANY backend because they only use the DataProvider interface, never backend-specific code.
## Layer 4: React Hooks
### Backend-Agnostic Hooks
React hooks wrap services to provide convenient, type-safe access:
**File:** `frontend/src/hooks/useThings.tsx`
```typescript
import { useDataProvider } from './useDataProvider';
import { useMutation, useQuery } from '@tanstack/react-query';
import { Effect } from 'effect';
export function useThings(options: ListThingsOptions) {
const provider = useDataProvider();
return useQuery({
queryKey: ['things', options],
queryFn: async () => {
return await Effect.runPromise(provider.things.list(options));
},
staleTime: 5000, // 5 seconds
});
}
export function useCreateThing() {
const provider = useDataProvider();
const queryClient = useQueryClient();
return useMutation({
mutationFn: async (input: CreateThingInput) => {
return await Effect.runPromise(provider.things.create(input));
},
onSuccess: () => {
// Invalidate and refetch
queryClient.invalidateQueries({ queryKey: ['things'] });
},
});
}
```
### Auth Hooks
**File:** `frontend/src/hooks/useAuth.ts`
```typescript
export function useLogin() {
const provider = useDataProvider();
const [loading, setLoading] = useState(false);
const [error, setError] = useState<AuthError | null>(null);
const mutate = useCallback(async (args: LoginArgs) => {
setLoading(true);
setError(null);
try {
return await Effect.runPromise(provider.auth.login(args));
} catch (err) {
setError(err as AuthError);
throw err;
} finally {
setLoading(false);
}
}, [provider]);
return { mutate, loading, error };
}
```
### Hook Inventory
**All 45 hooks implemented:**
| Category | Hooks | Purpose |
|----------|-------|---------|
| Organizations | 7 hooks | Org management, members |
| People | 7 hooks | Users, roles, permissions |
| Things | 8 hooks | CRUD + type-specific |
| Connections | 8 hooks | Relationships |
| Events | 6 hooks | Audit trails, activity |
| Knowledge | 6 hooks | RAG, search, labels |
| Auth | 7 hooks | All 6 auth methods |
## Layer 5: UI Components
### Backend-Agnostic Components
Components only use hooks, never backend-specific code:
```typescript
// frontend/src/components/courses/CourseList.tsx
import { useThings } from '@/hooks/useThings';
import { Card } from '@/components/ui/card';
export function CourseList() {
const { data: courses, loading, error } = useThings({ type: 'course' });
if (loading) return <div>Loading courses...</div>;
if (error) return <div>Error: {error.message}</div>;
return (
<div className="grid grid-cols-3 gap-4">
{courses?.map((course) => (
<Card key={course._id}>
<h3>{course.name}</h3>
<p>{course.properties.description}</p>
<span>${course.properties.price}</span>
</Card>
))}
</div>
);
}
```
**This component works identically whether:**
- Backend is Convex
- Backend is Notion
- Backend is WordPress
- Backend is Supabase (future)
**Zero code changes required when switching backends.**
## Configuration System
### Environment-Based Configuration
**File:** `frontend/src/config/providers.ts`
```bash
# .env.local
# Choose your backend (convex, wordpress, notion, supabase)
BACKEND_PROVIDER=convex
# Convex configuration
PUBLIC_CONVEX_URL=https://shocking-falcon-870.convex.cloud
CONVEX_DEPLOYMENT=prod:shocking-falcon-870
# WordPress configuration (if using)
WORDPRESS_URL=https://mysite.com/wp-json
WORDPRESS_USERNAME=admin
WORDPRESS_APP_PASSWORD=xxxx xxxx xxxx xxxx xxxx xxxx
# Notion configuration (if using)
NOTION_API_TOKEN=secret_abc123...
NOTION_DATABASE_ID=def456...
# Encryption key (for secure credential storage)
PROVIDER_ENCRYPTION_KEY=your-32-byte-hex-key
```
### Multi-Tenant Configuration
**Critical:** Groups (not organizations) are the multi-tenancy boundary. Each group can have different backends.
```typescript
// Group 1 uses Convex (type: organization)
{
_id: "group_startup",
name: "Startup Inc",
type: "organization",
parentGroupId: undefined, // Top-level group
properties: {
plan: "pro",
backendProvider: "convex",
backendConfig: {
deploymentUrl: "https://fast-startup-123.convex.cloud"
}
}
}
// Group 2 uses WordPress (type: business)
{
_id: "group_enterprise",
name: "Enterprise Corp",
type: "business",
parentGroupId: "group_holding", // Child of another group
properties: {
plan: "enterprise",
backendProvider: "wordpress",
backendConfig: {
url: "https://enterprise.com/wp-json",
credentials: "encrypted:abc123..."
}
}
}
```
### Runtime Provider Switching
```typescript
// Switch group's backend
await switchProviderForGroup(
'group_123',
{
type: 'notion',
token: 'secret_xyz...',
databaseId: 'abc123...'
},
'user_456' // Actor who made the change
);
// Takes <30 seconds
// - Validates new config
// - Tests connection
// - Saves config to groups table
// - Clears cache
// - Logs provider_switched event
```
## Provider Setup
### AppProviders Component
The root provider that gives all components access to DataProvider:
**File:** `frontend/src/components/providers/AppProviders.tsx`
```typescript
import { ConvexProvider } from 'convex/react';
import { ConvexReactClient } from 'convex/react';
import { DataProviderProvider } from '@/hooks/useDataProvider';
import { createConvexProvider } from '@/providers/ConvexProvider';
const convexClient = new ConvexReactClient(import.meta.env.PUBLIC_CONVEX_URL);
const convexDataProvider = createConvexProvider({ client: convexClient });
export function AppProviders({ children }: { children: ReactNode }) {
return (
<ConvexProvider client={convexClient}>
<DataProviderProvider provider={convexDataProvider}>
{children}
</DataProviderProvider>
</ConvexProvider>
);
}
```
### Usage in Astro Pages
Due to Astro's islands architecture, wrap auth components:
```tsx
// frontend/src/components/auth/SignInPage.tsx
import { AppProviders } from '@/components/providers/AppProviders';
import { SimpleSignInForm } from './SimpleSignInForm';
export function SignInPage() {
return (
<AppProviders>
<SimpleSignInForm />
</AppProviders>
);
}
```
```astro
// frontend/src/pages/account/signin.astro
import { SignInPage } from '@/components/auth/SignInPage';
<Layout title="Sign In">
<SignInPage client:only="react" />
</Layout>
```
**Why this pattern?** Astro's `client:only` creates isolated React islands. The provider must be inside the same island as components that use hooks.
## Adding a New Provider
### 6-Step Process
**Step 1: Create Provider File**
```typescript
// frontend/src/providers/supabase/SupabaseProvider.ts
import { createClient } from '@supabase/supabase-js';
import type { DataProvider } from '@/providers/DataProvider';
export function createSupabaseProvider(config: SupabaseConfig): DataProvider {
const supabase = createClient(config.url, config.anonKey);
return {
things: {
get: (id) =>
Effect.tryPromise({
try: async () => {
const { data, error } = await supabase
.from('things')
.select('*')
.eq('id', id)
.single();
if (error) throw error;
return mapSupabaseRowToThing(data);
},
catch: (error) => new ThingNotFoundError(id, String(error)),
}),
// ... more operations
},
// ... other dimensions
};
}
```
**Step 2: Define ID Format**
```typescript
// Supabase: Use UUIDs
const thingId = "supabase_550e8400-e29b-41d4-a716-446655440000";
// Conversion functions
function supabaseIdToOneId(uuid: string): string {
return `supabase_${uuid}`;
}
function oneIdToSupabaseId(id: string): string {
return id.replace('supabase_', '');
}
```
**Step 3: Map Database Schema**
```sql
-- Supabase tables (5-table implementation)
CREATE TABLE groups (
id UUID PRIMARY KEY,
name VARCHAR(255),
type VARCHAR(50), -- friend_circle, business, community, dao, government, organization
parent_group_id UUID REFERENCES groups(id), -- Hierarchical nesting
properties JSONB,
status VARCHAR(20),
created_at TIMESTAMP,
updated_at TIMESTAMP
);
CREATE TABLE things (
id UUID PRIMARY KEY,
type VARCHAR(50),
name TEXT,
group_id UUID REFERENCES groups(id), -- Scoped to group
properties JSONB,
status VARCHAR(20),
created_at TIMESTAMP,
updated_at TIMESTAMP
);
CREATE TABLE connections (
id UUID PRIMARY KEY,
from_thing_id UUID REFERENCES things(id),
to_thing_id UUID REFERENCES things(id),
relationship_type VARCHAR(50),
group_id UUID REFERENCES groups(id), -- Scoped to group
metadata JSONB,
created_at TIMESTAMP
);
CREATE TABLE events (
id UUID PRIMARY KEY,
type VARCHAR(50),
actor_id VARCHAR(255), -- Person ID
target_id VARCHAR(255), -- Thing/Person/Connection ID
group_id UUID REFERENCES groups(id), -- Scoped to group
metadata JSONB,
timestamp BIGINT
);
CREATE TABLE knowledge (
id UUID PRIMARY KEY,
type VARCHAR(50),
text TEXT,
embedding VECTOR(768), -- Vector for semantic search
source_thing_id UUID REFERENCES things(id),
group_id UUID REFERENCES groups(id), -- Scoped to group
labels JSONB,
metadata JSONB,
created_at TIMESTAMP,
updated_at TIMESTAMP
);
```
**Step 4: Implement All 6 Dimensions**
Implement all methods for groups, people, things, connections, events, knowledge, and auth.
**Critical:** Remember that the 6-dimension ontology is implemented using 5 tables:
1. **groups** table (multi-tenancy boundary, hierarchical nesting)
2. **things** table (with groupId scoping)
3. **connections** table (with groupId scoping)
4. **events** table (with groupId scoping)
5. **knowledge** table (with groupId scoping)
People are represented as things with type: 'creator' and properties.role field, OR as a separate people table depending on backend requirements.
**Step 5: Add to Factory**
```typescript
// frontend/src/providers/factory.ts
export function createProvider(config: ProviderConfig): DataProvider {
switch (config.type) {
case 'convex':
return createConvexProvider(config);
case 'notion':
return createNotionProvider(config);
case 'wordpress':
return createWordPressProvider(config);
case 'supabase':
return createSupabaseProvider(config); // ✨ New
default:
throw new Error(`Unknown provider: ${config.type}`);
}
}
```
**Step 6: Write Tests**
```typescript
// frontend/test/providers/SupabaseProvider.test.ts
describe('SupabaseProvider', () => {
it('should create thing', async () => {
const provider = createSupabaseProvider(testConfig);
const thingId = await Effect.runPromise(
provider.things.create({
type: 'course',
name: 'Test Course',
properties: { price: 99 }
})
);
expect(thingId).toMatch(/^supabase_/);
});
// ... 80 more tests
});
```
**Documentation:** See `one/knowledge/provider-creation-guide.md` for complete guide.
## Performance
### Overhead Measurements
| Operation | Baseline (Direct) | With DataProvider | Overhead | Status |
|-----------|-------------------|-------------------|----------|--------|
| Get Thing | 5ms | 6ms | +1ms | ✅ <10ms |
| List Things | 10ms | 11ms | +1ms | ✅ <10ms |
| Create Thing | 15ms | 16ms | +1ms | ✅ <10ms |
| Update Thing | 10ms | 11ms | +1ms | ✅ <10ms |
| Delete Thing | 8ms | 9ms | +1ms | ✅ <10ms |
**Target:** <10ms overhead per operation ✅ **Achieved**
### Why So Fast?
1. **Thin wrappers:** Providers just map types, no processing
2. **Effect.ts optimized:** Minimal runtime overhead
3. **Direct backend calls:** No middleware, no proxies
4. **Smart caching:** React Query handles data caching
## Migration Guide
### From Direct Convex to DataProvider
**Before:**
```typescript
import { useMutation, useQuery } from 'convex/react';
import { api } from '@/convex/_generated/api';
function CourseList() {
const courses = useQuery(api.queries.things.list, { type: 'course' });
const create = useMutation(api.mutations.things.create);
return <div>{courses?.map(c => <div>{c.name}</div>)}</div>;
}
```
**After:**
```typescript
import { useThings, useCreateThing } from '@/hooks/useThings';
function CourseList() {
const { data: courses } = useThings({ type: 'course' });
const { mutate: create } = useCreateThing();
return <div>{courses?.map(c => <div>{c.name}</div>)}</div>;
}
```
**Changes:**
1. Replace `useQuery(api.*)` with `useThings()`
2. Replace `useMutation(api.*)` with `useCreateThing()`
3. Wrap page with `AppProviders`
**Benefits:**
- Can now switch to WordPress/Notion/Supabase without code changes
- Better error handling with typed errors
- Built-in loading states
- Optimistic updates
## Multi-Tenant Support
### Organization-Level Configuration
Each organization can use a different backend:
```typescript
// Organization 1: Convex (fast, real-time)
const org1 = {
_id: "org_startup",
properties: {
backendProvider: "convex",
backendConfig: {
deploymentUrl: "https://fast-startup.convex.cloud"
}
}
};
// Organization 2: WordPress (existing content)
const org2 = {
_id: "org_enterprise",
properties: {
backendProvider: "wordpress",
backendConfig: {
url: "https://enterprise.com/wp-json",
credentials: "encrypted:..."
}
}
};
// Organization 3: Notion (collaborative docs)
const org3 = {
_id: "org_agency",
properties: {
backendProvider: "notion",
backendConfig: {
token: "secret_...",
databaseId: "abc123..."
}
}
};
```
### Data Isolation
Perfect isolation between groups (multi-tenancy):
1. **Convex:** groupId in every document (things, connections, events, knowledge all scoped)
2. **Notion:** Separate databases per group OR groupId page property
3. **WordPress:** Post meta `_group_id` filter OR separate wp_groups table
4. **Supabase:** Row-level security (RLS) policies based on groupId
**Key Principle:** ALL dimensions (except groups themselves) are scoped via groupId. This is the universal multi-tenancy pattern.
## Real-World Use Cases
### Use Case 1: Startup (Convex)
**Profile:**
- New SaaS company
- Needs real-time features
- Fast iteration
**Solution:**
```bash
BACKEND_PROVIDER=convex
PUBLIC_CONVEX_URL=https://fast-startup-123.convex.cloud
```
**Benefits:**
- Real-time subscriptions
- Instant deploys
- Free tier generous
- Zero devops
### Use Case 2: Agency (WordPress)
**Profile:**
- Existing WordPress site
- 10,000+ blog posts
- Client content expertise
**Solution:**
```bash
BACKEND_PROVIDER=wordpress
WORDPRESS_URL=https://agency.com/wp-json
WORDPRESS_USERNAME=admin
WORDPRESS_APP_PASSWORD=xxxx xxxx xxxx xxxx
```
**Benefits:**
- Use existing content
- No data migration
- Familiar admin UI
- SEO-optimized URLs
### Use Case 3: Consultancy (Notion)
**Profile:**
- Team uses Notion daily
- Collaborative documents
- Client proposals/tracking
**Solution:**
```bash
BACKEND_PROVIDER=notion
NOTION_API_TOKEN=secret_abc123...
NOTION_DATABASE_ID=def456...
```
**Benefits:**
- Sync with existing workflow
- Beautiful editor built-in
- Permission system
- Export flexibility
### Use Case 4: Enterprise (Multi-Backend)
**Profile:**
- Multiple departments
- Different data sources
- Compliance requirements
**Solution:**
- Marketing dept → WordPress (public content)
- Engineering dept → Convex (real-time data)
- Sales dept → Notion (CRM docs)
**Benefits:**
- Each dept uses best tool
- ONE unified interface
- Cross-department connections
- Centralized reporting
## Testing
### Mock Provider Pattern
For testing, create a mock provider:
```typescript
// test/mocks/MockProvider.ts
export function createMockProvider(): DataProvider {
const mockData = new Map<string, Thing>();
return {
things: {
get: (id) =>
mockData.has(id)
? Effect.succeed(mockData.get(id)!)
: Effect.fail(new ThingNotFoundError(id)),
create: (input) =>
Effect.sync(() => {
const id = `mock_${Date.now()}`;
mockData.set(id, { _id: id, ...input });
return id;
}),
// ... more mocked operations
},
// ... other dimensions
};
}
```
### Component Testing
```typescript
import { render } from '@testing-library/react';
import { DataProviderProvider } from '@/hooks/useDataProvider';
import { createMockProvider } from '@/test/mocks/MockProvider';
import { CourseList } from './CourseList';
test('renders courses', async () => {
const mockProvider = createMockProvider();
// Seed mock data
await Effect.runPromise(
mockProvider.things.create({
type: 'course',
name: 'Test Course',
properties: { price: 99 }
})
);
const { getByText } = render(
<DataProviderProvider provider={mockProvider}>
<CourseList />
</DataProviderProvider>
);
expect(getByText('Test Course')).toBeInTheDocument();
});
```
## Security Considerations
### 1. Credential Encryption
All backend credentials are encrypted at rest:
```typescript
// Encrypt credentials
const encrypted = encryptCredentials(
{ username: 'admin', password: 'secret123' },
process.env.PROVIDER_ENCRYPTION_KEY
);
// Store encrypted
await db.insert('external_connections', {
type: 'wordpress',
config: { credentials: encrypted }
});
// Decrypt on use
const decrypted = decryptCredentials(
config.credentials,
process.env.PROVIDER_ENCRYPTION_KEY
);
```
### 2. Role-Based Access
Only specific roles can switch providers:
```typescript
// Check authorization (4 roles in the system)
// platform_owner, org_owner, org_user, customer
if (!hasRole(user, ['platform_owner', 'org_owner'])) {
throw new UnauthorizedError('Only group owners can switch providers');
}
```
### 3. Connection Testing
Test connectivity before saving config:
```typescript
// Validate provider works
const testResult = await testProviderConnection(newConfig);
if (!testResult.success) {
throw new ConnectionTestError(`Failed: ${testResult.error}`);
}
```
### 4. Audit Trail
Log all provider switches:
```typescript
await db.insert('events', {
type: 'settings_updated',
actorId: userId, // Person who made the change
targetId: groupId, // Group whose provider was switched
groupId: groupId, // Events scoped to group
metadata: {
setting: 'backendProvider',
oldValue: 'convex',
newValue: 'wordpress',
switchDuration: 28000 // ms
},
timestamp: Date.now()
});
```
## Monitoring & Debugging
### Provider Health Checks
Monitor backend connectivity:
```typescript
export async function checkProviderHealth(
provider: DataProvider
): Promise<HealthStatus> {
const start = Date.now();
try {
// Test basic operation
await Effect.runPromise(
provider.things.list({ limit: 1 })
);
return {
status: 'healthy',
responseTime: Date.now() - start,
timestamp: Date.now()
};
} catch (error) {
return {
status: 'unhealthy',
error: String(error),
timestamp: Date.now()
};
}
}
```
### Performance Monitoring
Track provider overhead:
```typescript
// Add instrumentation
const instrumentedProvider = {
things: {
get: (id) => {
const start = performance.now();
return provider.things.get(id).pipe(
Effect.tap(() => {
const duration = performance.now() - start;
analytics.track('provider.operation', {
provider: 'convex',
operation: 'things.get',
duration
});
})
);
}
}
};
```
## Future Enhancements
### Phase 2: Additional Providers
1. **Supabase** - PostgreSQL with real-time
2. **Firebase** - Google's backend-as-a-service
3. **Airtable** - Spreadsheet-like interface
4. **Contentful** - Headless CMS
5. **Strapi** - Open-source CMS
### Phase 3: Provider Features
1. **Cross-provider connections** - Connect Convex thing to WordPress post
2. **Provider federation** - Use multiple backends simultaneously
3. **Data synchronization** - Sync between providers
4. **Provider marketplace** - Community-built providers
5. **Hybrid storage** - Hot data in Convex, cold data in S3
### Phase 4: Advanced Features
1. **GraphQL federation** - Unified GraphQL API across providers
2. **Event streaming** - Real-time events across all providers
3. **Conflict resolution** - Handle concurrent updates
4. **Schema evolution** - Migrate data between providers
5. **Provider analytics** - Cost optimization recommendations
## Comparison: Traditional vs Backend-Agnostic
### Traditional Architecture
```
Frontend → Convex SDK → Convex Database
↓ (tightly coupled)
Changes to switch backends:
- Rewrite ALL components (100+ files)
- Update ALL queries/mutations
- Test EVERYTHING again
- High risk, high cost
```
### Backend-Agnostic Architecture
```
Frontend → DataProvider Interface → [Convex | WordPress | Notion]
↓ (loosely coupled)
Changes to switch backends:
- Update 1 env variable
- Zero code changes
- Same tests work
- Low risk, zero cost
```
## Key Takeaways
### ✅ What We Achieved
1. **True Backend Independence:** Components don't know what backend they're using
2. **Zero Migration Cost:** Switch backends with one config change
3. **Performance:** <10ms overhead per operation
4. **Type Safety:** Compiler catches errors at build time
5. **Developer Experience:** Clean, intuitive API
6. **Multi-Tenant:** Different orgs can use different backends
7. **Production Ready:** 110+ tests passing, TypeScript strict mode
### 🎯 Why This Matters
**For Developers:**
- Write once, run anywhere
- Test without backend (use mocks)
- Clear separation of concerns
- Future-proof architecture
**For Organizations:**
- Choose best backend for needs
- No vendor lock-in
- Migrate when ready
- Use existing infrastructure
**For Platform:**
- Market expansion (WordPress users, Notion users, etc.)
- Enterprise sales (use their database)
- Competitive advantage (unique feature)
- Proves ontology flexibility
## Getting Started
### 1. Install Dependencies
```bash
cd frontend
bun install
```
### 2. Configure Backend
```bash
# .env.local
BACKEND_PROVIDER=convex
PUBLIC_CONVEX_URL=https://your-deployment.convex.cloud
```
### 3. Wrap App with Provider
```tsx
// src/components/auth/YourPage.tsx
import { AppProviders } from '@/components/providers/AppProviders';
import { YourComponent } from './YourComponent';
export function YourPage() {
return (
<AppProviders>
<YourComponent />
</AppProviders>
);
}
```
### 4. Use Hooks
```tsx
// src/components/YourComponent.tsx
import { useThings } from '@/hooks/useThings';
export function YourComponent() {
const { data: courses } = useThings({ type: 'course' });
return (
<div>
{courses?.map(course => (
<div key={course._id}>{course.name}</div>
))}
</div>
);
}
```
### 5. Switch Backend (Optional)
```bash
# Change to WordPress
BACKEND_PROVIDER=wordpress
WORDPRESS_URL=https://mysite.com/wp-json
WORDPRESS_USERNAME=admin
WORDPRESS_APP_PASSWORD=xxxx xxxx xxxx xxxx
```
**No code changes needed!** 🎉
## Conclusion
The backend-agnostic architecture proves that the 6-dimension ontology is truly universal. Whether your data lives in Convex, WordPress, Notion, or any future database, the same components, hooks, and services work identically.
**This is not just an abstraction layer—it's a fundamental reimagining of how applications should be built.**
Organizations now have true freedom to choose their backend based on their needs, not the limitations of the frontend framework.
## Summary: Key Alignment Points
This document is now aligned with the main architecture specification:
### ✅ Aligned Concepts
1. **Groups (not Organizations)** - Dimension 1 is "groups" with 6 types (friend_circle, business, community, dao, government, organization)
2. **5-Table Implementation** - Groups table + 4 other tables (things, connections, events, knowledge) with groupId scoping
3. **Hierarchical Nesting** - Groups can contain groups via `parentGroupId` field
4. **Multi-Tenancy via groupId** - ALL dimensions (except groups) are scoped to groupId for data isolation
5. **People Representation** - People can be things with type: 'creator' OR a separate table (provider-dependent)
6. **4 Roles** - platform_owner, org_owner, org_user, customer (as defined in main architecture)
7. **Universal Ontology** - The 6 dimensions work with ANY backend (Convex, WordPress, Notion, Supabase, etc.)
### 🎯 Critical Differences from Legacy
This document **replaces** old 4-dimension terminology:
- OLD: "organizations" table → NEW: "groups" table (with type field including "organization")
- OLD: 4 dimensions → NEW: 6 dimensions (added groups and knowledge)
- OLD: organizationId field → NEW: groupId field (consistent across all tables)
- OLD: Flat hierarchy → NEW: Hierarchical groups (infinite nesting)
### 📚 Related Documentation
- `one/knowledge/architecture.md` - Main architecture reference (this document aligns with it)
- `one/knowledge/ontology.md` - Complete 6-dimension ontology specification
- `one/knowledge/provider-creation-guide.md` - How to build new providers
- `one/things/features/2-1-dataprovider-interface.md` - DataProvider spec
- `one/things/features/2-3-effectts-services.md` - Service layer patterns
- `frontend/AGENTS.md` - Quick reference for development
**Implementation Status:** ✅ **Production Ready** (Plan 2 Complete - 7/7 features)
**Alignment Status:** ✅ **Fully Aligned** with main architecture.md (Version 3.0)