@trap_stevo/star-vault
Version:
Unleash the future of data management with the ultimate platform for secure, scalable, and dynamic data operations. Power the next generation of applications by combining advanced encryption, revolutionary real-time querying, and seamless synchronization
620 lines (461 loc) β’ 29.8 kB
Markdown
# π Star-Vault
**Unleash the future of data management** with the ultimate platform for secure, scalable, and dynamic data operations. Power the next generation of applications by combining advanced encryption, revolutionary real-time querying, and seamless synchronization to create an ecosystem where data transforms into action.
## π Features
- π **Optional Encryption** β Secure sensitive data at rest
- βοΈ **Sharded Storage Engine** β Efficiently scales writes across shards
- π§ **In-Memory Caching** β High-speed read layer with `StarCache`
- π **Write-Ahead Logging** β Resilient logs with rotation and retention policies
- π **Advanced Query Engine** β Chainable and expressive queries with filtering, search, sorting, and spatial support
- π **Real-Time Event Emission** β Listen to data changes with fine-grained control
- π‘οΈ **Authentication Layer** β Optional handler to authorize every operation
- π **Collection Wildcards** β Seamlessly operate across multiple collections
## βοΈ System Requirements
| Requirement | Version |
|----------------|--------------------|
| **Node.js** | β₯ 19.x |
| **npm** | β₯ 9.x (recommended)|
| **OS** | Windows, macOS, Linux |
# π API Specifications
## π§ Constructor
Use these parameters when initializing `StarVault`.
```js
new StarVault(dbPath, logPath, shardCount, maxLogSize, logRetention, options)
```
### Core Parameters
| Key | Description | Default |
|---------------|------------------------------------------------------------------------------|--------------|
| `dbPath` | Root directory for all collection data; each collection is stored in sharded files. | **Required** |
| `logPath` | Directory where write-ahead logs (WAL) are stored for durability and replay. | **Required** |
| `shardCount` | Number of shards used per collection. More shards improve concurrency. | `4` |
| `maxLogSize` | Max size of each WAL file before rotation. Accepts values like `"100MB"`. | `"869MB"` |
| `logRetention`| Duration to retain old WAL files. Accepts values like `"1w"` or `"30d"`. | `"1w"` |
### `options` Object
| Key | Description | Default |
|-------------------|-----------------------------------------------------------------------------|------------------------|
| `enableEncryption`| Enables encryption for data at rest. | `false` |
| `vaultPath` | Path to the vault metadata file used for encryption. | `"./star-vault.json"` |
| `masterKey` | Master encryption key for vault encryption. Must securely generate/store.| `null` |
| `authHandler` | Custom function that receives `clientAuth` and returns `true`/`false`. | `null` |
| `authOptions` | Configuration object passed to internal `StarAuth` for authentication. | `{}` |
| `dirMode` | UNIX permission mode for directories (e.g., `0o700`). | `0o700` |
| `fileMode` | UNIX permission mode for files (e.g., `0o600`). | `0o600` |
## π authOptions (StarAuth Configuration)
Pass this object under the `options.authOptions` key when creating your `StarVault` instance.
### Collection Settings
| Key | Description | Default |
|----------------------|--------------------------------------------|---------------------|
| `collection` | User collection name | `"auth-users"` |
| `sessionCollection` | Session tracking collection | `"auth-sessions"` |
| `resetCollection` | Password reset token collection | `"auth-resets"` |
| `stellarCollection` | Magic link / code token collection | `"stellar-auths"` |
### Session Behavior
| Key | Description | Default |
|---------------------------|---------------------------------------------------------|--------------------------|
| `tokenExpiry` | Token lifetime in seconds | `3600` (1 hour) |
| `lockoutDuration` | Lock duration after failed login attempts | `900000` (15 minutes) |
| `maxLoginAttempts` | Max allowed failures before lockout | `5` |
| `sessionValidationFields` | Fields to check for session hijack | `["ip", "fingerprint"]` |
| `strictSessionValidation` | Reject session if mismatch in validation fields | `true` |
| `enableSuspiciousCheck` | Compare geo/IP with previous sessions | `true` |
| `sessionPolicy` | "default", "strict", or "replace" session strategies | `"default"` |
### Password Management
| Key | Description | Default |
|----------------------|----------------------------------------------------------------------------------------------|--------------------------------------------|
| `passwordRequirements` | Object defining the password policy | `{ minLength: 8, requireLetter: true, requireNumber: true, requireSymbol: false }` |
| β `minLength` | Minimum number of characters required | `8` |
| β `requireLetter` | Requires at least one letter (a-z or A-Z) | `true` |
| β `requireNumber` | Requires at least one numeric digit (0-9) | `true` |
| β `requireSymbol` | Requires at least one symbol (e.g., `!@#$%^&*`) | `false` |
| β `customValidator` | Optional custom function `(password) => boolean` for advanced password checks | `null` |
| `lockingCombinations` | Password hash complexity | `10` |
### π Geo Lookup Configuration
| Key | Description | Default |
|----------------------|-----------------------------------------------------------------------------|----------|
| `enableGeo` | Enable geo lookups on login/session events | `false` |
| `enableGeoDebug` | If `true`, logs which geo service succeeded or failed during lookup | `false` |
| `geoServices` | Array of additional geo lookup functions `(ip) => geoData | null` | `[]` |
| `ipgeolocationKey` | API key for ipgeolocation.io (enables provider if set) | `null` |
| `ipinfoToken` | API token for ipinfo.io (enables provider if set) | `null` |
### Stellar Login
| Key | Description | Default |
|---------------------------|-------------------------------------------------|-----------------|
| `stellarRequestCooldown`| Minimum delay between stellar requests | `60000` (1 min) |
| `generateStellarCode` | Function to generate a stellar numeric code | 6-digit default |
### Hooks & Extensions
| Key | Description |
|----------------------|------------------------------------------------------|
| `onSuspiciousSession`| `(currentSession, pastSession) => void` |
| `handleHijack` | `(session, field, expected, actual) => void` |
| `onCleanup` | `({ result, timestamp, vaultID }) => void` |
| `tagSession` | `(session, userData) => string[]` |
### Guest Session Configurations
| Key | Description | Default |
|-----------------------------|-----------------------------------------------------------------------------|----------------------------------|
| `allowGuestSessions` | Enables or disables guest session support. | `true` |
| `guestInactivityThreshold` | Duration of inactivity (in ms) before considering a guest account stale. | `7 * 24 * 60 * 60 * 1000` (1w) |
| `cleanupGuestInterval` | Interval (in ms) to check for inactive guest accounts. | `5 * 60 * 1000` (5m) |
| `guestActivityTrackers` | Array of custom functions to execute on guest activity updates. | `[]` |
| `generateGuestID` | Function to generate guest user IDs. | `() => "guest-" + UUID` |
### Cleanup & Locking
| Key | Description | Default |
|--------------------------------|------------------------------------------------------|-------------|
| `vaultID` | ID for this vault instance | `null` |
| `autoCleanupInterval` | How often to auto-clean expired tokens | `null` |
| `expiredSessionCleanupInterval` | Interval for cleaning expired sessions | `null` |
| `cleanupExpiredTokensActionInfo` | Metadata for cleanup activity | `{}` |
| `cleanupExpiredTokensClientAuth` | Auth context used for cleanup calls | `null` |
| `cleanupExpiredSessionsActionInfo` | Metadata for session cleanup | `{}` |
| `cleanupExpiredSessionsClientAuth` | Auth context used during session cleanup | `null` |
## π¦ Core Methods
The primary database interaction methods.
| Method | Description | Sync/Async |
|--------|-------------|------------|
| `create(collection, data, actionInfo = {}, clientAuth = null)` | Creates a new record in the specified collection. Returns the created record object. | β
Sync |
| `update(collection, id, updates, actionInfo = {}, clientAuth = null)` | Updates an existing record by ID. Setting record data properties to undefined removes them respectively. Returns the updated record. Throws error if not found. | β
Sync |
| `unset(collection, id, dotPaths = [], actionInfo = {}, clientAuth = null)` | Removes specific nested fields using dot notation (e.g., "profile.stats.xp"). Returns updated record. | β
Sync |
| `deleteCollection(collection, actionInfo = {}, clientAuth = null)` | Deletes an entire collection. Returns `{ wholeCollection: true, deleted: true/false }`. | β
Sync |
| `deleteRecord(collection, id, actionInfo = {}, clientAuth = null)` | Deletes a specific record by ID. Returns `{ id, deleted: true/false }`. | β
Sync |
| `delete(collection, id, actionInfo = {}, clientAuth = null)` | Soft-deletes a record by filtering it out and overwriting the collection. Returns `{ id, deleted: true }`. | β
Sync |
## π Query Engine
Chainable query builder for filtering and searching collection data.
```js
vault.query("collection")
.whereRecord("id", "value") // Filters root record data by key-value pairs
.where({ key: "value" }) // Filters record data by key-value pairs
.search("field", "text") // Text search within a specific field
.recent("timestamp", "7d") // Gets records from the last 7 days
.near("location", { lat: 0, lng: 0 }, 50) // Geospatial filter within 50 units
.sort({ name: 1 }) // Sorts by a field (1 = asc, -1 = desc)
.select(["id", "name"]) // Selects specific fields
.limit(10) // Limits results
.offset(0) // Skips first N records
.filterBy(record => record.active) // Filters with custom function
.execute(exactCollection = false) // Run the query with optional strict collection matching
```
| Method | Description | Sync/Async |
|--------|-------------|------------|
| `query(collection)` | Begins a query on the specified collection. Returns a chainable builder. | β
Sync |
| `whereRecord(criteria)` | Filters records by matching key-value pairs. Accepts both object and parameter inputs. | β
Sync |
| `where(criteria)` | Filters records' data by matching key-value pairs. Accepts both object and parameter inputs. | β
Sync |
| `search(field, text)` | Performs a text search within a specified field. | β
Sync |
| `recent(field, duration)` | Filters records based on recency (e.g., `"7d"` = 7 days). | β
Sync |
| `near(field, center, radius)` | Filters based on proximity to a location object `{ lat, lng }`. | β
Sync |
| `sort(criteria)` | Sorts the results by one or more fields. | β
Sync |
| `select(fields)` | Selects only the specified fields for output. | β
Sync |
| `limit(number)` | Restricts the number of records returned. | β
Sync |
| `offset(number)` | Skips a number of records (used for pagination). | β
Sync |
| `filterBy(fn)` | Filters using a custom function on each record. | β
Sync |
| `callback(fn)` | Adds a side effect or transformation hook before execution. | β
Sync |
| `execute(exactCollection = false)` | Run the query. If `true`, match only the exact collection name; otherwise, include related subcollections. | β
Sync |
| `getByID(collection, id)` | Shortcut to retrieve a single record by ID. | β
Sync |
| `range(min, max)` | Static helper to return an inclusive array of numbers in a range. | β
Sync |
## π‘οΈ Authentication
All methods below require `authOptions` configuration and are used for session/user management.
### π€ User Registration & Authentication
| Method | Description | Sync/Async |
|--------|-------------|------------|
| `registerUser(email, password, actionInfo, clientAuth)` | Registers a new user. `. | βοΈ Async |
| `registerGuestUser(req, metadata, actionInfo, clientAuth)` | Registers a temporary guest user. `. | βοΈ Async |
| `loginGuestUser(accountID, req, actionInfo, clientAuth)` | Logs in a guest user and creates a session. `. | βοΈ Async |
| `loginUser(email, password, req, actionInfo, clientAuth)` | Logs in a user and creates a session. `. | βοΈ Async |
| `resumeUserSession(accountID, req, actionInfo, clientAuth)` | Resumes session from account ID (used for guest linking or session carryover). | βοΈ Async |
| `upgradeGuestUser(accountID, email, password, actionInfo, clientAuth)` | Converts a guest account to a full registered user. | βοΈ Async |
### π°οΈ Session Management
| Method | Description | Sync/Async |
|--------|-------------|------------|
| `logoutUser(token, actionInfo, clientAuth)` | Logs out a user by invalidating their session token. | β
Sync |
| `extendUserSession(token, ms, actionInfo, clientAuth)` | Extends an active session's duration. | β
Sync |
| `cleanupExpiredAuthTokens(actionInfo, clientAuth)` | Cleans up expired tokens and invalid sessions. | β
Sync |
| `cleanupInactiveGuests(actionInfo, clientAuth)` | Removes or deactivates inactive guest accounts. | βοΈ Async |
### π Password & Magic Link Recovery
| Method | Description | Sync/Async |
|--------|-------------|------------|
| `updateUserPassword(userID, newPassword, actionInfo, clientAuth)` | Updates a user's password. | βοΈ Async |
| `requestUserPasswordReset(email, actionInfo, clientAuth)` | Sends a password reset token. | βοΈ Async |
| `resetUserPassword(token, newPassword, actionInfo, clientAuth)` | Resets password using a valid token. | βοΈ Async |
| `requestUserStellarLink(email, type, actionInfo, clientAuth)` | Sends a Stellar link (magic link or code) for login or verification. | βοΈ Async |
| `consumeUserStellarToken(token, type, req, actionInfo, clientAuth)` | Consumes a Stellar link/token to verify user and return session. | βοΈ Async |
### π₯ Guest Account Management
| Method | Description | Sync/Async |
|--------|-------------|------------|
| `registerGuestUser(req, metadata, actionInfo, clientAuth)` | Registers a guest session with optional request/metadata. | βοΈ Async |
| `trackGuestActivity(userID, timestamp, actionInfo, clientAuth)` | Updates guest user's `lastActiveAt` timestamp. | βοΈ Async |
| `upgradeGuestUser(accountID, email, password, actionInfo, clientAuth)` | Promotes a guest to a full registered account. | βοΈ Async |
| `cleanupInactiveGuests(actionInfo, clientAuth)` | Cleans up inactive or stale guest users based on threshold. | βοΈ Async |
### π User Lifecycle Management
| Method | Description | Sync/Async |
|--------|-------------|------------|
| `updateUser(accountID, updatedMetadata, accountType, actionInfo, clientAuth)` | Updates a user by Account ID. Does not allow for password updates. | βοΈ Async |
| `deleteUser(userID, reason, actionInfo, clientAuth)` | Soft-deletes a user by ID. | βοΈ Async |
| `deleteUserByAccountID(accountID, reason, actionInfo, clientAuth)` | Soft-deletes a user using their account ID. | βοΈ Async |
| `reactivateUser(userID, actionInfo, clientAuth)` | Reactivates a previously deleted user. | βοΈ Async |
| `reactivateUserByAccountID(accountID, actionInfo, clientAuth)` | Reactivates a deleted user by account ID. | βοΈ Async |
### π User & Session Querying
| Method | Description | Sync/Async |
|--------|-------------|------------|
| `getUserByAccountID(accountID, filter, accountType, actionInfo)` | Retrieves a user by account id. If a certain account type (default ~ guest). | βοΈ Async |
| `getUser(userID, filter, accountType, actionInfo)` | Retrieves a user by id. If a certain account type (default ~ guest). | βοΈ Async |
| `getUserSession(sessionID, actionInfo)` | Retrieves a session by id. | βοΈ Async |
| `listSessions(userID, filters, includeInactive, actionInfo)` | Lists sessions for a specific user. | βοΈ Async |
| `listAllSessions(filters, includeInactive, actionInfo)` | Lists all sessions across users. | βοΈ Async |
| `listUsers(filters, options, actionInfo)` | Lists users based on filters and options (pagination, includes). | βοΈ Async |
| `listUserActivity(options, actionInfo)` | Lists recent activity, including filtering by active, inactive, or recently seen users. | βοΈ Async |
### π§© Internal Validation & Utility Methods
The following utility methods get utilized internally across authentication-related flows. You may call them directly for validation, diagnostics, or extension purposes.
| Method | Description | Sync/Async |
|--------------------------------|-----------------------------------------------------------------------------------------------|-------------------------------------------------------------------------|
| `validEmail(email)` | Determines if an input email follows proper email address format. | β
Sync |
| `satisfiesPasswordStrength(password)` | Validates whether the password meets the defined `passwordRequirements`. | β
Sync |
| `extractSessionMetadata(req)` | Extracts metadata (IP, fingerprint, user agent, etc.) from the request object . | β
Sync |
| `lookupUserGeo(ip)` | Performs geo-IP lookup using built-in and custom geo services . | βοΈ Async |
## π§ Event Listening
Used for reactive, event-driven architecture patterns.
| Method | Description | Sync/Async |
|--------|-------------|------------|
| `listen(event, nodePath, callback)` | Registers a callback for a specific event (`create`, `update`, `delete`) at a given path. | β
Sync |
| `emit(event, nodePath, data)` | Triggers callbacks for listeners whose paths match the emitted path. | β
Sync |
## π Collection & Path Matching
Utilities for dynamic collection operations and wildcard paths.
| Method | Description | Sync/Async |
|--------|-------------|------------|
| `getMatchingCollections(basePath)` | Resolves wildcard-based paths like `logs/*/2025` to real collection names. | β
Sync |
| `matchesPath(listenerPath, nodePath)` | Checks if an emitted node path matches a listener's wildcard path. | β
Sync |
## π StarVault Events
StarVault emits real-time events that let you react to data and authentication lifecycle changes across your app. These include both **record-level events** and **auth lifecycle events**.
You can listen to them at multiple levels of granularity:
- π **Global listeners** (across all collections and records)
- π **Collection-level listeners** (any record in a collection)
- π§Ύ **Record-specific listeners** (a specific record path)
### π Emitted Record Events (StarVault Core)
| Event | Description | Emitted Path |
|----------|-----------------------------------------|--------------------------|
| `create` | Fires on record creation. | `collectionName/recordID` |
| `update` | An existing record updated | `collectionName/recordID` |
| `delete` | A record got deleted (soft or hard) | `collectionName/recordID` |
### π Emitted Auth Lifecycle Events (StarAuth)
| Event | Description |
|---------------------|-----------------------------------------------------------|
| `guest:upgraded` | A guest account successfully upgraded to a full user |
| `guest:updated` | A guest account successfully updated |
| `user:updated` | A user account successfully updated |
| `session:resumed` | A user session resumed from a valid token |
| `user:guestExpired` | A guest account expired due to inactivity |
| `user:deleted` | A user account got marked for deletion |
| `user:reactivated` | A previously deleted user got reactivated |
### πͺ Global Listener (All Changes Across Vault)
```js
vault.listen("create", "*", (path, data) => {
console.log("Created:", path, data);
});
vault.listen("update", "*", (path, data) => {
console.log("Updated:", path, data);
});
vault.listen("delete", "*", (path, data) => {
console.log("Deleted:", path, data);
});
```
### π Collection-Level Listener
```js
vault.listen("create", "users/*", (path, data) => {
console.log("User created:", path, data);
});
vault.listen("update", "inventory/*", (path, data) => {
console.log("Inventory updated:", path, data);
});
```
### π§ Record-Specific Listener
```js
vault.listen("update", "orders/12345", (path, data) => {
console.log("Order updated:", data);
});
vault.listen("delete", "accounts/user-abc", (path, data) => {
console.log("User deleted:", data);
});
```
### π― Advanced Wildcard Usage
```js
// Match nested logs by year
vault.listen("update", "logs/*/2025/*", (path, data) => {
console.log("Log updated:", path);
});
```
### π§ Listening to StarAuth Lifecycle Events
Use `.on()` to subscribe to StarAuth-specific events:
```js
vault.on("guest:upgraded", (info) => {
console.log("Guest upgraded:", info);
});
vault.on("user:deleted", (info) => {
console.log("User deleted:", info);
});
vault.on("user:reactivated", (info) => {
console.log("User reactivated:", info);
});
```
These event hooks power real-time dashboards, activity tracking, notifications, and custom automation β all deeply integrated with StarVault's reactive system.
## β¨ Getting Started
### π¦ Installation
```bash
npm install @trap_stevo/star-vault
```
### Basic Usage
```js
const StarVault = require("@trap_stevo/star-vault");
const vault = new StarVault(
"./data",
"./logs",
4,
"869MB",
"1w",
{
enableEncryption : true,
masterKey : "supersecretkey",
authHandler : (auth) => auth.token === "valid-token"
}
);
vault.create("users", { id : "001", name : "Nova" }, { source : "init" }, { token : "valid-token" });
```
## π Querying
```js
const results = vault.query("users")
.where({ role : "admin" })
.sort({ name : 1 })
.select(["id", "name"])
.limit(10)
.execute();
```
## π§ Listening to Changes
```js
vault.listen("create", "*", (path, data) => {
console.log("[CREATE - ANY] Path:", path, data);
});
vault.listen("create", "users/*", (path, data) => {
console.log("[CREATE - users] Path:", path, data);
});
vault.listen("update", "users/*", (path, data) => {
console.log("[UPDATE - users] Path:", path, data);
});
// Listen for updates to a specific user record (e.g., user ID '001')
vault.listen("update", "users/user-001", (path, data) => {
console.log("[UPDATE - users/user-001] Path:", path, data);
});
vault.listen("delete", "users/*", (path, data) => {
console.log("[DELETE - users] Path:", path, data);
});
vault.listen("delete", "logs/*/2025", (path, data) => {
console.log("[DELETE - logs/*/2025] Path:", path, data);
});
// Listen for a specific log record inside a structured path (e.g., logs/server1/2025/log-99)
vault.listen("create", "logs/server1/2025/log-99", (path, data) => {
console.log("[CREATE - logs/server1/2025/log-99] Path:", path, data);
});
// Listen to everything happening in 'auth-sessions'
vault.listen("create", "auth-sessions", (path, data) => {
console.log("[CREATE - auth-sessions] Path:", path, data);
});
vault.listen("update", "auth-sessions", (path, data) => {
console.log("[UPDATE - auth-sessions] Path:", path, data);
});
vault.listen("delete", "auth-sessions", (path, data) => {
console.log("[DELETE - auth-sessions] Path:", path, data);
});
```
## π Wildcard Collection Queries
```js
vault.query("logs/*/2025")
.recent("timestamp", "7d")
.execute();
```
## π₯ Registering Users (Guest and Normal)
```js
const StarVault = require("@trap_stevo/star-vault");
const vault = new StarVault(
"./data",
"./logs",
4,
"869MB",
"1w",
{
enableEncryption : true,
masterKey : "supersecretkey",
authHandler : (auth) => auth.token === "star-vault-demo",
authOptions : {
allowGuestSessions : true,
generateGuestID : () => "guest-" + Math.floor(Math.random() * 1000000)
}
}
);
// Register a full user
const userResult = await vault.registerUser("nova@example.com", "strongpassword123", { source : "site" }, { token : "star-vault-demo" });
console.log("Registered user:", userResult);
// Register a guest user
const guestResult = await vault.registerGuestUser({ ip : "127.0.0.1" }, { referrer : "home" }, { source : "guest-form" }, { token : "star-vault-demo" });
console.log("Registered guest:", guestResult);
```
## π Listing Users and Sessions
```js
const StarVault = require("@trap_stevo/star-vault");
const vault = new StarVault(
"./data",
"./logs",
4,
"869MB",
"1w",
{
enableEncryption : true,
masterKey : "supersecretkey",
authHandler : (auth) => auth.token === "star-vault-demo"
}
);
// List all users
const users = await vault.listUsers({}, { limit : 25 }, { token : "star-vault-demo" });
console.log("All users:", users);
// List all sessions (including inactive and deleted)
const sessions = await vault.listAllSessions({ includeDeleted : "only" }, true, { token : "star-vault-demo" });
console.log("All sessions:", sessions);
// List sessions for a specific user
const userSessions = await vault.listSessions("user-001", { device : "mobile" }, false, { token : "star-vault-demo" });
console.log("Sessions for user-001:", userSessions);
// List recent user activity (active and inactive)
const activity = await vault.listUserActivity({
includeInactive : true,
onlyMostRecent : true
}, { token : "star-vault-demo" });
console.log("User activity:", activity);
```
## β¨ License
See License in [LICENSE.md](./LICENSE.md)
## π Transform Data into Action
Star-Vault transcends traditional data systemsβoffering a full-fledged database engine and cosmic foundation that propels secure, reactive, and intelligent data-driven architectures.