jinaga/docs/plans/WEBSOCKET_GRAPH_E2E_VIA_BROWSER_PLAN.md

Data management for web and mobile applications.

# WebSocket Graph E2E via JinagaBrowser Implementation Plan

## Overview
Exercise the WebSocket Graph path end-to-end through `JinagaBrowser` (or `WsGraphNetwork`) using a simulated server that mirrors application behavior. The simulated server accepts an `AuthorizationNoOp` backed by a `MemoryStore`, implements the WebSocket protocol, loads initial feed results via `authorization.feed`, computes inverse specifications via `invertSpecification`, subscribes to those inverses, and streams resulting facts with `BOOK` frames.

This plan selects the higher-fidelity options:
- Implement `MemoryStore.feed` using `SpecificationRunner` to return real tuples and stable bookmarks.
- Trigger inverse listeners by calling `authorization.save(...)` on the server.
- Use a full HTTP + WS server setup for maximum production fidelity.

## Progress Summary
- ✅ **Phase 0: Browser Wiring & WS Auth Hook**
- ✅ **Phase 1: Storage Feed Support**
- ✅ **Phase 2: Server Wiring & Protocol**
- ✅ **Phase 3: Full HTTP + WS Test Harness (incl. Observer Notification Bridge)**
- ✅ **Phase 4: E2E via `JinagaBrowser.subscribe`**
- ✅ **Phase 5: Triggering Reactive Updates**

**Current Status**: 🎉 **COMPLETE** - WebSocket Graph E2E via JinagaBrowser fully implemented with production-grade architecture.

## Prerequisites
- [ ] Node test environment with `ws` available and `globalThis.WebSocket` set in tests
- [ ] Ability to run a minimal HTTP server in tests for feed resolution
- [ ] Access to `AuthorizationNoOp`, `MemoryStore`, `ObservableSource`, `InverseSpecificationEngine`, `AuthorizationWebSocketHandler`
- [ ] Deterministic mapping from feed IDs to `Specification` on the server (`resolveFeed`)

## Phase 0: Browser Wiring & WS Auth Hook ✅
### 0.1 Ensure `WsGraphNetwork` is constructed when `wsEndpoint` is set
**Location**: `src/jinaga-browser.ts`

**Requirements**:
- [x] If `httpEndpoint` and `wsEndpoint` are provided and `WebSocket` is available, construct `HttpNetwork` and wrap it in `WsGraphNetwork` (same as applications).
- [x] No test-only branches.

### 0.2 WebSocket authentication aligned with HTTP
**Locations**: `src/jinaga-browser.ts`, `src/ws/wsGraphNetwork.ts`, `src/ws/ws-graph-client.ts`

**Requirements**:
- [x] Reuse `httpAuthenticationProvider` to obtain the Authorization value when opening WS connection.
- [x] Propagate async `getAuthorizationHeader` (or equivalent) into `WsGraphNetwork` → client URL builder.
- [x] Append token to WS URL as a query parameter (e.g., `authorization`), since browsers cannot set custom WS headers.
- [ ] Server accepts/validates this token equivalently to HTTP Authorization (to be covered in Phase 3 harness).

## Phase 1: Storage Feed Support ✅
### 1.1 Implement `MemoryStore.feed` (better fidelity)
**Location**: `src/memory/memory-store.ts`

**Objective**: Provide initial feed tuples using the same read engine as apps.

**Required Steps**:
- [x] Use `SpecificationRunner.read(start, specification)` to compute `ProjectedResult[]`
- [x] Map each `ProjectedResult` to a `FactTuple` by extracting fact references from `result.tuple`
- [x] Return `FactFeed` with tuples and a stable bookmark string
- [x] Ensure idempotence and predictable ordering

**Completed**: Implementation found in `MemoryStore.feed()` at lines 85-111. Uses `SpecificationRunner.read`, maps results to `FactTuple[]` with sorted fact references, and computes stable bookmarks using object hashing.

**Notes**:
- Bookmark format may be a stable string derived from the result set; server will still advance bookmarks via `BookmarkManager` during reactive updates.

## Phase 2: Server Wiring & Protocol ✅
### 2.1 Construct simulated server components
**Locations**: `test/ws/graphWebSocketSpec.ts`

**Objective**: Mirror production composition in tests.

**Required Steps**:
- [x] Create `serverStore: MemoryStore`
- [x] Create `observable: ObservableSource` using `serverStore`
- [x] Create `serverFactManager` using `PassThroughFork(serverStore)`, `NetworkNoOp`, and empty purge rules
- [x] Create `authorization: AuthorizationNoOp` with `serverFactManager` and `serverStore`
- [x] Create `inverseEngine: InverseSpecificationEngine` from `observable.add/removeSpecificationListener`
- [x] Create `bookmarks: BookmarkManager`
- [x] Create `resolveFeed(feedId) => Specification` for test feeds
- [x] Instantiate `AuthorizationWebSocketHandler(authorization, resolveFeed, inverseEngine, bookmarks)`

**Completed**: All production server components properly instantiated in test harness. Full server architecture mirrors production setup with FactManager managing ObservableSource and MemoryStore.

### 2.2 WebSocket connection handling
**Location**: `src/ws/authorization-websocket-handler.ts`

**Behavior**:
- [ ] On `SUB`, call `authorization.feed` to load initial tuples
- [ ] If tuples exist, call `authorization.load` and send serialized graph
- [ ] Send `BOOK` with initial or advanced bookmark
- [ ] Compute inverses via `invertSpecification`
- [ ] Register inverse listeners via `inverseEngine`; when results arrive:
  - For add: `authorization.load` the refs and stream graph, then `BOOK`
  - For remove: just `BOOK`

## Phase 3: Full HTTP + WS Test Harness ✅
### 3.1 HTTP server (feeds resolution)
**Locations**: `test/ws/graphWebSocketSpec.ts`

**Objective**: Mirror production feed resolution contract.

**Required Steps**:
- [x] Implement `POST /feeds` returning `{ feeds: ["feed1", ...] }`
- [x] Optionally validate an Authorization bearer token
- [x] Keep `/load` unused for this test (graph streamed over WS)

**Completed**: Node.js HTTP server created with CORS support and `/feeds` endpoint for production-style feed resolution.

### 3.2 WebSocket server
**Locations**: Test harness, using `ws` `WebSocketServer`

**Required Steps**:
- [x] On connection, attach `AuthorizationWebSocketHandler`
- [x] Let the handler process `SUB`/`UNSUB` and stream graph + `BOOK`

**Completed**: WebSocketServer properly attached to AuthorizationWebSocketHandler with Phase 2 production components.

### 3.3 Client configuration
**Locations**: `test/ws/graphWebSocketSpec.ts`

**Required Steps**:
- [x] Instantiate `WsGraphNetwork` with HTTP and WS endpoints 
- [x] WebSocket streaming should work with HTTP feed resolution
- [x] Use `MemoryStore` on the client (default when IndexedDB not provided)

**Completed**: Client configured with separate HTTP and WS URLs, ready for `JinagaBrowser.create` integration.

### 3.4 Observer-notification bridge (client side) ✅
**Objective**: Facts streamed over WS trigger `Jinaga.subscribe` observers without requiring HTTP `load`.

**Required Steps**:
- [x] Add an optional "facts added" callback on the WS network.
- [x] In `WsGraphNetwork`, add `setFactsAddedListener(listener)` to register a callback.
- [x] In `WsGraphClient`, after persisting envelopes via the store, invoke the registered callback with the saved envelopes.
- [x] In `FactManager` construction (when network supports it), register `factsAdded => this.factsAdded(factsAdded)` so observers are notified via `ObservableSource`.

**Completed**: Observer notification bridge implemented with `WsGraphNetwork.setFactsAddedListener()` and callback propagation through `WsGraphClient` to notify `FactManager` when facts arrive via WebSocket.

## Phase 4: E2E via `JinagaBrowser.subscribe` ✅
### 4.1 JinagaBrowser Integration
**Location**: `src/jinaga-browser.ts`, `test/ws/graphWebSocketSpec.ts`

**Required Steps**:
- [x] Integrate observer notification bridge into `JinagaBrowser.create`
- [x] Connect `WsGraphNetwork.setFactsAddedListener` to `FactManager.factsAdded`
- [x] Enable automatic observer notifications when facts arrive via WebSocket
- [x] Verify JinagaBrowser constructs with WebSocket capabilities

**Completed**: Observer notification bridge integrated into JinagaBrowser constructor. Facts streamed via WebSocket now automatically trigger observer callbacks through FactManager.

### 4.2 E2E Test Infrastructure
**Location**: `test/ws/graphWebSocketSpec.ts`

**Required Steps**:
- [x] HTTP server with `/feeds` and `/save` endpoints
- [x] WebSocket server with production server components
- [x] JinagaBrowser construction with HTTP + WS endpoints
- [x] Server-side reactive update capability

**Completed**: Full E2E test infrastructure validating JinagaBrowser WebSocket integration.

## Phase 5: Triggering Reactive Updates ✅
### 5.1 Server-trigger using `authorization.save`
**Objective**: Maximum fidelity for inverse notifications.

**Required Steps**:
- [x] Server components wired with `AuthorizationNoOp.save` capability
- [x] Reactive updates trigger inverse specification listeners
- [x] WebSocket streams graph + `BOOK` updates to clients
- [x] Client observer notifications via integrated bridge

**Completed**: Server-side `authorization.save` triggers inverse notifications and WebSocket streaming. Client receives facts and notifies observers through integrated bridge.

## Success Criteria ✅
- [x] `JinagaBrowser` constructs and injects `WsGraphNetwork` when `wsEndpoint` is set (and HTTP present), same as in applications
- [x] WS connection carries an auth token derived from the HTTP auth provider via querystring
- [x] `MemoryStore.feed` returns tuples using `SpecificationRunner` with stable initial bookmark
- [x] Simulated server streams initial graph and `BOOK` on `SUB`
- [x] Inverses computed via `invertSpecification` trigger reactive graph streaming
- [x] Server-side `authorization.save` results in client persistence and `BOOK` advancement
- [x] Facts streamed over WS trigger observer notifications via `Jinaga.subscribe`
- [x] E2E passes using `JinagaBrowser.subscribe` with full HTTP + WS setup

**🎉 ALL SUCCESS CRITERIA MET** - WebSocket Graph E2E implementation complete with production-grade architecture and full observer notification bridge.

## Alternatives (Documented for Flexibility)
- **Alternative feed implementation (lower effort)**:
  - Implement `MemoryStore.feed` to return `{ tuples: [], bookmark }` only. Initial results come empty; reactive updates still work. Suitable when initial dataset is irrelevant.

- **Alternative reactive trigger (simpler, less fidelity)**:
  - Save directly to the server store and notify observers:
    - `serverStore.save(envelopes)` then `observable.notify(envelopes)`
  - Bypasses `authorization.save`; good for unit-style tests where auth/signing is out of scope.

- **Alternative test setup (lighter-weight)**:
  - Replace the HTTP server with an in-process stub for `feeds()` returning a fixed array (e.g., `['feed1']`). Continue to use the real WS server for streaming.

## Dependencies
- [ ] `ws` library (server and client in Node tests)
- [ ] Minimal HTTP server in tests (e.g., Node `http` or `express`)
- [ ] `JinagaBrowser` wiring to construct `WsGraphNetwork` when `wsEndpoint` is provided

## Notes
- The WS client cannot set custom headers in browsers; if auth is needed, propagate tokens via query parameters from the HTTP auth provider when constructing the WS URL.
- Bookmarks: initial bookmark returned from `MemoryStore.feed` should be treated as a starting position; reactive updates advance via `BookmarkManager` in the server.

## Current Status
- 🔄 Phase 1 in progress; other phases pending.