@n2flowjs/nbase
Version:
Neural Vector Database for efficient similarity search
146 lines (145 loc) • 8.13 kB
Plain Text
[Database] initialization started...
[Database] Initializing PartitionedVectorDB...
[PartitionedVectorDB] Initializing with options: {
"partitionsDir": "D:\\git\\nbase\\database\\partitions",
"partitionCapacity": 100000,
"maxActivePartitions": 3,
"autoCreatePartitions": true,
"autoLoadPartitions": true,
"vectorSize": 1536,
"useCompression": false,
"clusterOptions": {
"clusterSize": 100,
"newClusterThresholdFactor": 1.5,
"newClusterDistanceThreshold": 0.5,
"maxClusters": 1000,
"distanceMetric": "euclidean",
"useCompression": false
}
}
[PartitionedVectorDB] Configuration:
- partitionsDir: D:\git\nbase\database\partitions
- partitionCapacity: 100000
- maxActivePartitions: 3
- autoCreatePartitions: true
- vectorSize: 1536
- useCompression: false
- autoLoadHNSW: true
- runKMeansOnLoad: true
[PartitionedVectorDB] Starting initialization (autoLoad: true)
[PartitionedVectorDB] Loading partition configurations from D:\git\nbase\database\partitions
[Database] Waiting for PartitionedVectorDB to become ready...
API Server running on http://localhost:1307
[PartitionedVectorDB] Found 1 potential partition directories.
[PartitionedVectorDB] Attempting to load config: D:\git\nbase\database\partitions\p-1749282892289\p-1749282892289.config.json
[PartitionedVectorDB] Loaded config for partition: p-1749282892289 (Dir: p-1749282892289, Active: true, Vectors: 182)
[PartitionedVectorDB] Active partition ID after scan: p-1749282892289
[PartitionedVectorDB] Loaded 1 partition configurations.
[PartitionedVectorDB] Auto-loading initial partitions: [p-1749282892289]
[PartitionedVectorDB] Loading partition p-1749282892289 DB from base path: D:\git\nbase\database\partitions\p-1749282892289\data
[VectorDB] Constructor Loading database from D:\git\nbase\database\partitions\p-1749282892289\data...
[ClusteredVectorDB] Loading state from D:\git\nbase\database\partitions\p-1749282892289\data
[ClusteredVectorDB] Loading state from D:\git\nbase\database\partitions\p-1749282892289\data
[VectorDB] Loaded 182 vectors from D:\git\nbase\database\partitions\p-1749282892289\data
[ClusteredVectorDB] Base VectorDB data loaded.
[ClusteredVectorDB] Cluster state file path: D:\git\nbase\database\partitions\p-1749282892289\data
[ClusteredVectorDB] Loading cluster state from: D:\git\nbase\database\partitions\p-1749282892289\data\cluster.json
[VectorDB] Loaded 182 vectors from D:\git\nbase\database\partitions\p-1749282892289\data
[ClusteredVectorDB] Base VectorDB data loaded.
[ClusteredVectorDB] Cluster state file path: D:\git\nbase\database\partitions\p-1749282892289\data
[ClusteredVectorDB] Loading cluster state from: D:\git\nbase\database\partitions\p-1749282892289\data\cluster.json
[ClusteredVectorDB] Cluster state loaded successfully (165 clusters).
[PartitionedVectorDB] Partition DB p-1749282892289 loaded. Vector count: 182
[PartitionedVectorDB] Loading HNSW index for partition p-1749282892289
[PartitionedVectorDB] Loading HNSW index for partition p-1749282892289
[PartitionedVectorDB] Loading HNSW index for partition p-1749282892289 from D:\git\nbase\database\partitions\p-1749282892289\hnsw\hnsw_index.json
[ClusteredVectorDB] Cluster state loaded successfully (165 clusters).
[PartitionedVectorDB] Successfully loaded HNSW index for p-1749282892289. Nodes: 182
[PartitionedVectorDB] HNSW index loaded for partition p-1749282892289
[PartitionedVectorDB] Syncing vector count for partition p-1749282892289
[PartitionedVectorDB] Loaded vector count: 182
[PartitionedVectorDB] Initial partitions loaded (1 in memory, 1 HNSW indices loaded).
[PartitionedVectorDB] Initialization complete. Active: p-1749282892289
[Database] PartitionedVectorDB is ready.
[Database] Initializing UnifiedSearch...
[Database] Loading/Building indices based on buildOnStart option...
[Database] Checking if initial index build is required...
[Database] Building HNSW index for partition all loaded...
[Database] Requesting index build for all loaded partitions...
[PartitionedVectorDB] Building HNSW indices for 1 loaded partitions in parallel...
[PartitionedVectorDB] Building HNSW index for partition p-1749282892289...
[PartitionedVectorDB] Getting partition p-1749282892289...
[PartitionedVectorDB] Building HNSW index for partition p-1749282892289...
[Database] Progress: 0%
Building HNSW index with 1 different dimensions
Dimension 768: 182 vectors
768 in ids
[Database] Progress: 0.5945054945054946%
[Database] Progress: 1%
[PartitionedVectorDB] HNSW index built successfully for partition p-1749282892289.
[PartitionedVectorDB] Finished building HNSW indices for loaded partitions.
Index build process finished in 251.83ms for relevant partitions.
[Database] Initial index build process completed (if applicable).
Starting monitoring (interval: 60000ms)
Setting up auto-save via vectorDB.save() every 60000ms
[Database] initialization successful. Ready for operations.
[Database] Initial State: Partitions Configured: 1, Loaded: 1
[no-ip] GET /health started
[no-ip] GET /health 200 - 9ms
Auto-save triggered...
[PartitionedVectorDB] Starting comprehensive save...
[PartitionedVectorDB] Debounced saving of partition configurations...
[PartitionedVectorDB] Saved 1 partition configurations.
[PartitionedVectorDB] Partition configurations saved. Active partition: p-1749282892289
[PartitionedVectorDB] Saving data for 1 loaded partitions...
[ClusteredVectorDB] Saving state to D:\git\nbase\database\partitions\p-1749282892289\data
[VectorDB] Saving database...
[VectorDB] Saving to D:\git\nbase\database\partitions\p-1749282892289\data
[VectorDB] Meta file path: D:\git\nbase\database\partitions\p-1749282892289\data\meta.json
[VectorDB] Vector file path: D:\git\nbase\database\partitions\p-1749282892289\data\vec.bin
[VectorDB] Save promise created.
[PartitionedVectorDB] Saving 1 loaded HNSW indices...
[PartitionedVectorDB] Saving HNSW index for partition p-1749282892289
[PartitionedVectorDB] Saving HNSW index for partition p-1749282892289 to D:\git\nbase\database\partitions\p-1749282892289\hnsw\hnsw_index.json
[VectorDB] Meta file path: D:\git\nbase\database\partitions\p-1749282892289\data\meta.json
[VectorDB] Vector file path: D:\git\nbase\database\partitions\p-1749282892289\data\vec.bin
[VectorDB] Preparing vector data for saving with 182 vectors...
[VectorDB] Vector data prepared for saving: 182 vectors
[VectorDB] Writing metadata file to: D:\git\nbase\database\partitions\p-1749282892289\data\meta.json with 182 vectors
[VectorDB] Writing meta file to: D:\git\nbase\database\partitions\p-1749282892289\data\meta.json
[PartitionedVectorDB] Successfully saved HNSW index for p-1749282892289.
[VectorDB] Meta file written successfully.
[VectorDB] Writing vector file to: D:\git\nbase\database\partitions\p-1749282892289\data\vec.bin (182 vectors, 559104 bytes)
[VectorDB] Vector file written successfully.
[VectorDB] Save event emitted successfully.
[ClusteredVectorDB] Base VectorDB data saved.
[ClusteredVectorDB] Cluster state file path: D:\git\nbase\database\partitions\p-1749282892289\data
[ClusteredVectorDB] Saving cluster state to: D:\git\nbase\database\partitions\p-1749282892289\data\cluster.json
[ClusteredVectorDB] Cluster state saved successfully.
[PartitionedVectorDB] Saved data for partition p-1749282892289
[PartitionedVectorDB] Comprehensive save complete. Partitions saved: 1/1. HNSW indices saved: 1/1.
Auto-save completed successfully.
[no-ip] GET /stats started
[no-ip] GET /stats 200 - 5ms
[no-ip] GET /stats started
[no-ip] GET /stats 200 - 4ms
[no-ip] GET /stats started
[no-ip] GET /stats 200 - 5ms
[no-ip] GET /stats started
[no-ip] GET /stats 200 - 4ms
[no-ip] GET /stats started
[no-ip] GET /stats 200 - 4ms
[no-ip] GET /stats started
[no-ip] GET /stats 200 - 4ms
[no-ip] GET /stats started
[no-ip] GET /stats 200 - 3ms
[no-ip] GET /stats started
[no-ip] GET /stats 200 - 5ms
[no-ip] GET /stats started
[no-ip] GET /stats 200 - 4ms
[no-ip] GET /stats started
[no-ip] GET /stats 200 - 4ms
[no-ip] GET /stats started
[no-ip] GET /stats 200 - 2ms
[no-ip] GET /stats started
[no-ip] GET /stats 200 - 3ms