@propickler/localvectordb/README.md

A lightweight local vector database implementation in Node.js similar to ChromaDB

# LocalVectorDB

A lightweight local vector database implementation in Node.js, similar to ChromaDB. This project provides a simple and efficient way to store, index, and query high-dimensional vectors with similarity search capabilities.

## Features

- Vector storage and indexing using HNSW algorithm
- Similarity search (cosine similarity)
- Collection-based organization
- REST API interface
- Metadata support for each vector
- Automatic ID generation

## Prerequisites

- Node.js (v14 or higher)
- npm (Node Package Manager)

## Installation

1. Install the package from npm:
```bash
npm install localvectordb
```

2. Or clone the repository and install dependencies:
```bash
git clone https://github.com/yourusername/localvectordb.git
cd localvectordb
npm install
```

## Quick Start

1. Start the server:
```bash
node src/index.js
```

The server will start on port 3000 by default. You can change this by setting the PORT environment variable.

## API Usage

### 1. Create a Collection

Collections are used to organize your vectors. Each collection requires a name and dimension size for the vectors it will store.

```bash
curl -X POST http://localhost:3000/collections \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my_collection",
    "dimension": 3
  }'
```

### 2. Add Vectors

Add vectors to your collection along with optional metadata:

```bash
curl -X POST http://localhost:3000/collections/my_collection/add \
  -H "Content-Type: application/json" \
  -d '{
    "ids": ["vec1", "vec2"],
    "embeddings": [
      [1.1, 2.2, 3.3],
      [4.4, 5.5, 6.6]
    ],
    "metadatas": [
      {"description": "first vector"},
      {"description": "second vector"}
    ]
  }'
```

### 3. Query Similar Vectors

Search for similar vectors using cosine similarity:

```bash
curl -X POST http://localhost:3000/collections/my_collection/query \
  -H "Content-Type: application/json" \
  -d '{
    "queryEmbeddings": [[1.0, 2.0, 3.0]],
    "nResults": 2
  }'
```

Response format:
```json
{
  "status": "success",
  "results": [
    [
      {
        "id": "vec1",
        "distance": 0.0023,
        "metadata": {
          "description": "first vector"
        }
      },
      {
        "id": "vec2",
        "distance": 0.0156,
        "metadata": {
          "description": "second vector"
        }
      }
    ]
  ]
}
```

## Node.js Client Usage

You can also use LocalVectorDB directly in your Node.js applications:

```javascript
const VectorDB = require('localvectordb');

// Create a new VectorDB instance
const db = new VectorDB();

// Create a collection
const collection = await db.createCollection('my_collection', 3);

// Add vectors
await collection.add({
  ids: ['vec1', 'vec2'],
  embeddings: [
    [1.1, 2.2, 3.3],
    [4.4, 5.5, 6.6]
  ],
  metadatas: [
    { description: 'first vector' },
    { description: 'second vector' }
  ]
});

// Query similar vectors
const results = await collection.query({
  queryEmbeddings: [[1.0, 2.0, 3.0]],
  nResults: 2
});
```

## API Reference

### Collections

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/collections` | POST | Create a new collection |
| `/collections/:name/add` | POST | Add vectors to a collection |
| `/collections/:name/query` | POST | Query similar vectors |

### Parameters

#### Create Collection
- `name` (string): Name of the collection
- `dimension` (number): Dimension of vectors to be stored

#### Add Vectors
- `ids` (string[]): Optional array of IDs for the vectors
- `embeddings` (number[][]): Array of vectors to add
- `metadatas` (object[]): Optional array of metadata objects

#### Query Vectors
- `queryEmbeddings` (number[][]): Array of query vectors
- `nResults` (number): Number of similar vectors to return

## Error Handling

The API returns appropriate HTTP status codes:
- 200: Success
- 400: Bad Request (invalid parameters)
- 404: Collection not found
- 500: Server error

Error responses include a message explaining the error:
```json
{
  "status": "error",
  "message": "Collection not found"
}
```

## Limitations

- Maximum vectors per collection: 100,000 (configurable)
- Maximum dimension size: No hard limit, but performance may degrade with very high dimensions
- Distance metric: Currently only supports cosine similarity

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

## License

MIT License - feel free to use this in your own projects!