evelodb/AGENTS.md

A high-performance native B-tree database for Node.js. Made by Evelocore.

# EveloDB — Agent Reference Guide

> This document is written for AI agents and automated tools. It provides everything needed to correctly read, write, and maintain code that uses **EveloDB** (`evelodb` npm package).

---

## 1. What is EveloDB?

EveloDB is a **local, file-backed, BSON B-Tree database for Node.js**. It stores data in binary `.db` files on disk using B-Tree indexes (`.bidx` files) for O(log n) lookups. It requires **no external database server** and is designed for high-throughput local storage.

**Key facts:**
- Data is stored on the local filesystem in a configurable directory (default: `./evelodbprime`).
- All records are serialized as BSON documents.
- Primary key is always `_id` internally (a 24-character hex ObjectID). A custom `objectIdKey` can alias it in the API.
- System-managed fields: `_id` (or `objectIdKey` alias), `_createdAt`, `_modifiedAt`. **Never set these manually.**
- Supports schema validation, B-Tree secondary indexes, unique constraints, atomic operators, and transactions.

---

## 2. Installation & Import

```bash
npm i evelodb
```

```typescript
// TypeScript / ES Modules
import eveloDB from 'evelodb';

// CommonJS
const eveloDB = require('evelodb');
```

---

## 3. CRITICAL RULES — Read Before Writing Any Code

> **Violating these rules will cause bugs that are hard to trace.**

### Rule 1: Single Instance Only
Create **one** `eveloDB` instance per application and export/share it. **Never** call `new eveloDB()` in multiple files or modules. Doing so creates desynchronized caches and causes `EPERM` lock errors and data loss.

```typescript
// db.ts — do this ONCE
import eveloDB from 'evelodb';
export const db = new eveloDB({ directory: './data' });

// other files — import the shared instance
import { db } from './db';
```

### Rule 2: Never Manually Set System Fields
The fields `_id`, `_createdAt`, `_modifiedAt`, and any custom `objectIdKey` are auto-managed. Passing them to `create()` or `edit()` will throw an error. The system strips them before writing.

### Rule 3: Empty Conditions `{}` Targets ALL Records
Calling `db.edit('users', {}, ...)`, `db.delete('users', {})`, or `db.find('users', {})` matches every record in the collection.

### Rule 4: `readImage()` is async, everything else is sync
All methods are **synchronous** except `readImage()`, `atomic()`, and `transaction()`. Do not `await` synchronous methods.

### Rule 5: Schema Strict Mode
If a `schema` is defined in the config with at least one collection, **only collections listed in the schema are accessible**. Calling any method on an unlisted collection returns `{ err: '...not defined in schema', code: 'COLLECTION_NOT_DEFINED' }`.

---

## 4. Configuration

```typescript
const db = new eveloDB({
  directory: './evelodbprime',  // Storage path (default: './evelodbprime')
  maxHandles: 64,               // Max open file handles in LRU cache (default: 64)
  compactThreshold: 0.3,        // Auto-compact when tombstones/total >= this ratio (default: 0.3)
  schema: {                     // Optional schema, indexes, and constraints per collection
    users: {
      fields: {
        username: { type: String, required: true, min: 5, max: 30 },
        email:    { type: String, required: true },
        age:      { type: Number, required: false, max: 120 },
        address: {
          type: {
            city:    { type: String, required: true },
            country: { type: String, required: true },
          },
          required: false,
        },
      },
      indexes:    ['email', 'username'],  // B-Tree secondary indexes (O(log n) lookups)
      uniqueKeys: ['email', 'username'],  // Fields that must be globally unique
      objectIdKey: 'userId',             // Alias for _id exposed in the API
      noRepeat: true,                    // Reject exact duplicate records (default: true)
    },
  },
});
```

### Schema Field Definition
| Property   | Type              | Description                                                     |
|------------|-------------------|-----------------------------------------------------------------|
| `type`     | `String` \| `Number` \| `Boolean` \| `Array` \| `Object` \| nested obj | The data type. Use constructors, not strings. |
| `required` | `boolean`         | If `true`, field must be present in `create()` / `edit()`.     |
| `min`      | `number`          | Min value (Number) or min length (String/Array).                |
| `max`      | `number`          | Max value (Number) or max length (String/Array).                |

### Nested Object Types
Nest a plain object (not `Object` constructor) as the `type` to define nested schema validation:
```typescript
vehicle: {
  type: {
    color: { type: String, required: true },
    model: { type: String, required: true },
  },
  required: false,
}
```

---

## 5. Core CRUD Operations

### `db.create(collection, data)` → `WriteResult`
Inserts a new record. Returns the generated ID under the `objectIdKey` (or `_id` if not configured).

```typescript
const result = db.create('users', { username: 'alice', email: 'alice@example.com' });
// result: { success: true, userId: '662e5a4e3d5a4e3d5a4e3d5a' }
// On error: { err: '...', code: 'SCHEMA_VALIDATION_FAILED' | 'DUPLICATE_KEY' | 'DUPLICATE_DATA' | ... }
```

### `db.edit(collection, conditions, newData)` → `EditResult`
Updates all records matching `conditions`. Also aliased as `db.update()`.

```typescript
db.edit('users', { username: 'alice' }, { email: 'new@example.com' });
// result: { success: true, modifiedCount: 1, skippedDuplicates: 0 }
// On no match: { err: 'No match', code: 'NO_MATCH' }
```

Supports **atomic update operators** in `newData`:
```typescript
db.edit('products', { productId: 'p1' }, { $inc: { stock: -1 } });
db.edit('users',    { userId: 'u1'    }, { $set: { status: 'active' } });
db.edit('posts',    { postId: 'p2'    }, { $push: { tags: 'nodejs' } });
db.edit('posts',    { postId: 'p2'    }, { $pull: { tags: 'experimental' } });
db.edit('records',  { userId: 'u2'    }, { $unset: { tempFlag: true } });
```

| Operator | Effect |
|----------|--------|
| `$inc`   | Increment/decrement a numeric field |
| `$set`   | Set a field to a value |
| `$pull`  | Remove a value or matching items from an array |
| `$unset` | Remove a field from the document |
| `$push`  | Append a value to an array field |

> If both operators and plain fields are mixed, operators take full precedence. Use only operators OR only plain merge, not both.

### `db.delete(collection, conditions)` → `DeleteResult`
Deletes all matching records.

```typescript
db.delete('users', { username: 'alice' });
// result: { success: true, deletedCount: 1 }
```

### `db.find(collection, conditions)` → `QueryResult<T>`
Returns a `QueryResult` for all matching records. Conditions support comparison operators.

```typescript
const result = db.find('users', { age: { $gte: 18 } });
const page = result.getList(0, 10);  // first 10
const total = result.count();
const all   = result.all();
```

### `db.findOne(collection, conditions)` → `T | null`
Returns the first matching record, or `null`.

```typescript
const user = db.findOne('users', { userId: 'abc123' });
```

### `db.get(collection)` → `QueryResult<T>`
Returns all records in a collection.

```typescript
const allUsers = db.get('users').all();
```

### `db.search(collection, conditions)` → `QueryResult<T>`
Case-insensitive substring search across specified fields.

```typescript
db.search('users', { username: 'ali' }); // matches "alice", "Ali", etc.
```

---

## 6. Comparison Operators (for `find()` and `delete()`)

```typescript
{ age:    { $eq: 25 } }                           // Equal
{ age:    { $ne: 25 } }                           // Not equal
{ age:    { $gt: 25 } }                           // Greater than
{ age:    { $gte: 25 } }                          // Greater than or equal
{ age:    { $lt: 25 } }                           // Less than
{ age:    { $lte: 25 } }                          // Less than or equal
{ status: { $in: ['active', 'pending'] } }        // In array
{ status: { $nin: ['inactive'] } }                // Not in array
{ name:   { $regex: '^Al', $options: 'i' } }      // Regex match
```

---

## 7. QueryResult Methods

`find()`, `search()`, and `get()` all return a `QueryResult<T>` object with:

| Method                         | Returns                      | Description                                |
|--------------------------------|------------------------------|--------------------------------------------|
| `.all()`                       | `T[] \| { err: string }`     | All matched records                        |
| `.getList(offset, limit)`      | `T[] \| { err: string }`     | Paginated slice                            |
| `.count()`                     | `number \| { err: string }`  | Total matched count                        |
| `.sort(compareFn)`             | `QueryResult<T>`             | Returns a new sorted QueryResult (chainable) |

```typescript
const result = db.find('users', { role: 'admin' })
  .sort((a, b) => a.username.localeCompare(b.username))
  .getList(0, 20);
```

---

## 8. Utility Operations

### `db.count(collection)` → `CountResult`
```typescript
const { count } = db.count('users');
```

### `db.check(collection, conditions)` → `boolean`
```typescript
const exists = db.check('users', { email: 'alice@example.com' });
```

### `db.inject(collection, data[], options?)` → `{ success, count }`
High-performance bulk import. Each record **must** include system fields: `_createdAt`, `_modifiedAt`, and the ID field.

```typescript
db.inject('users', records, { method: 'overwrite' }); // clears then fills
db.inject('users', records, { method: 'merge' });      // appends to existing
```

### `db.compact(collection)` → `{ success }`
Reclaims space used by tombstoned (deleted/updated) records. Normally triggered automatically.

### `db.rebuildIndexes(collection)` → `void`
Rebuilds all B-Tree index files from the raw data. Use for recovery from corruption.

### `db.drop(collection)` / `db.reset(collection)` → `DropResult`
Permanently deletes all data and index files for a collection.

### `db.closeAll()` → `void`
Flushes all indexes and closes all file handles. Call on graceful shutdown.

---

## 9. Object Store

For storing small key-value application state (not collections). Stored as `.objdb` BSON files.

```typescript
db.object('appConfig').write({ theme: 'dark', version: '1.0' });
db.object('appConfig').update({ theme: 'light' });       // deep merge
const cfg = db.object('appConfig').read();               // returns object or null
db.object('appConfig').rename('userSettings');
db.object('userSettings').delete();
const list = db.object().list();                         // ['appConfig', ...]
```

---

## 10. File Store

Stores raw binary files in `{directory}/files/`.

```typescript
db.writeFile('photo.jpg', buffer);          // save
const { data } = db.readFile('photo.jpg'); // read as Buffer
db.deleteFile('photo.jpg');                 // delete
const files = db.allFiles();               // ['photo.jpg', ...]
```

---

## 11. Image Processing (`readImage`)

> **This method is `async`.** Always `await` it.

```typescript
const result = await db.readImage('photo.jpg', {
  returnBase64: true,   // default true — returns data URL; false returns Buffer
  quality: 0.8,         // 0.1–1.0, compression quality
  pixels: 500000,       // max total pixel count (0 = no limit)
  maxWidth: 1280,       // max width in pixels
  maxHeight: 720,       // max height in pixels
  blackAndWhite: false, // greyscale
  mirror: false,        // flip horizontal
  upToDown: false,      // flip vertical
  invert: false,        // invert colors
  brightness: 1,        // 0.1–5, 1 = original
  contrast: 1,          // 0.1–5, 1 = original
});
// result: { success: true, data: 'data:image/jpeg;base64,...', metadata: { ... } }
```

Supported formats: `.jpg`, `.jpeg`, `.jfif`, `.png`, `.webp`, `.avif`, `.tiff`, `.gif`, `.bmp`, `.svg`, `.ico`, `.heic`.

**Performance notes:**
- Results are cached in a 200MB LRU cache (keyed by image hash + config). Repeated calls with the same input are near-instant.
- AVIF encodes are limited to 2 parallel workers by default. Set `AVIF_CONCURRENCY` env var to tune.

---

## 12. Backup & Restore

```typescript
// Create backup
db.createBackup('users', {
  type: 'binary',           // 'binary' (default, XOR-encoded) or 'json'
  path: './backups',
  password: 'secret',       // optional, for binary type
  title: 'April 2026',      // optional metadata
});

// Read backup info without restoring
const info = db.readBackupFile('./backups/users_backup.backup', 'secret');

// Restore (OVERWRITES current data)
db.restoreBackup('users', {
  type: 'binary',
  file: './backups/users_backup.backup',
  password: 'secret',
});
```

> Backups always preserve the raw `_id` field, regardless of `objectIdKey` settings.

---

## 13. Transactions & Atomic Operations

Use these when there is an `await` between a read and a subsequent write, to prevent race conditions.

### `db.transaction(collection, callback)` — Collection-level lock
```typescript
await db.transaction('products', async () => {
  const item = db.findOne('products', { productId: 'p1' });
  await someAsyncLogic();
  db.edit('products', { productId: 'p1' }, { stock: item.stock - 1 });
});
```

### `db.atomic(collection, callback)` — Collection-level lock with `tx` object
```typescript
await db.atomic('products', async (tx) => {
  const item = tx.findOne('products', { productId: 'p1' });
  await someAsyncLogic();
  tx.edit('products', { productId: 'p1' }, { $inc: { stock: -1 } });
});
```

### `db.atomic(callback)` — Global lock (all collections)
```typescript
await db.atomic(async (tx) => {
  const user = tx.findOne('users', { userId: 'u1' });
  tx.create('logs', { message: `${user.username} updated` });
});
```

> For simple counter updates without any `await` gaps, prefer `$inc` directly — it's faster and doesn't need a lock.

---

## 14. `objectIdKey` Mapping — How the ID Alias Works

When you set `objectIdKey: 'userId'` in the schema:
- **Reading**: The internal `_id` field is renamed to `userId` in all returned documents.
- **Writing/Querying**: Use `userId` in conditions and returned results; `_id` is internal only.
- **Inject**: The injection payload must include the ID as `userId` (or `_id` — both are accepted during inject).

```typescript
// Creating
db.create('users', { username: 'bob', email: 'bob@b.com' });
// → { success: true, userId: 'abc...' }  ← NOT _id

// Finding
db.findOne('users', { userId: 'abc...' });
// → { userId: 'abc...', username: 'bob', ... }  ← _id is hidden
```

---

## 15. Error Codes Reference

| Code                       | Meaning                                          |
|----------------------------|--------------------------------------------------|
| `COLLECTION_NOT_DEFINED`   | Collection not in schema (when schema is set)    |
| `SCHEMA_VALIDATION_FAILED` | Field type/required/min/max validation failed    |
| `DUPLICATE_KEY`            | `_id` already exists                             |
| `DUPLICATE_UNIQUE_KEY`     | A `uniqueKeys` field value is already taken      |
| `DUPLICATE_DATA`           | Exact record already exists (`noRepeat: true`)   |
| `NO_MATCH`                 | `edit()` found no matching records               |
| `MISSING_ID`               | `inject()` record missing ID field               |
| `MISSING_CREATED_AT`       | `inject()` record missing `_createdAt`           |
| `MISSING_MODIFIED_AT`      | `inject()` record missing `_modifiedAt`          |
| `ID_CONFLICT`              | `inject({ method: 'merge' })` — ID already exists |
| `404`                      | File not found (file store operations)           |

---

## 16. TypeScript Types Quick Reference

```typescript
import eveloDB, {
  EveloDBConfig,
  WriteResult,
  EditResult,
  DeleteResult,
  CountResult,
  DropResult,
  FileResult,
  ReadImageResult,
  ReadImageConfig,
  BackupResult,
  BackupFileInfo,
  Conditions,
  Condition,
  QueryResult,
  ObjectResult,
  ObjectStore,
} from 'evelodb';
```

---

## 17. Common Patterns for Agents

### Pattern: Paginated list endpoint
```typescript
const page = 0, limit = 20;
const result = db.find('products', { category: 'electronics' });
const items = result.getList(page * limit, limit);
const total = result.count();
```

### Pattern: Upsert (check-then-create-or-edit)
```typescript
const existing = db.findOne('users', { email: 'x@x.com' });
if (existing) {
  db.edit('users', { email: 'x@x.com' }, { lastSeen: new Date().toISOString() });
} else {
  db.create('users', { email: 'x@x.com', username: 'newuser' });
}
```

### Pattern: Atomic stock decrement
```typescript
// Simple (no async gap) — no lock needed
db.edit('products', { productId: 'p1' }, { $inc: { stock: -1 } });

// With async gap — use atomic
await db.atomic('products', async (tx) => {
  const p = tx.findOne('products', { productId: 'p1' });
  if (!p || p.stock <= 0) throw new Error('Out of stock');
  await chargePayment();
  tx.edit('products', { productId: 'p1' }, { $inc: { stock: -1 } });
});
```

### Pattern: Sorted search with pagination
```typescript
const users = db.search('users', { username: 'ali' })
  .sort((a, b) => a.username.localeCompare(b.username))
  .getList(0, 10);
```

### Pattern: Safe shutdown
```typescript
process.on('SIGINT', () => {
  db.closeAll();
  process.exit(0);
});
```