@needle-ai/needlectl/README.md

CLI tool for syncing local files to Needle

# Needle Control CLI Tool

`needlectl` is local client for MacOS, Windows, Linux to sync files to Needle.

## Features

- Real-time file monitoring and synchronization
- Automatic file state management
- Support for multiple file types
- Recursive and non-recursive folder monitoring
- Efficient handling of file changes, additions, and deletions

## Quick Start

Install `needlectl` command line tool globally using your favourite package manager, we recommend using `bun`:

```bash
bun install -g @needle-ai/needlectl
```

or using NPM:

```bash
npm install -g @needle-ai/needlectl
```

1. Verify installation:

```bash
needlectl --version
```

2. Set your API key:

```bash
needlectl api-key set <your-needle-api-key>
```

3. List your collections:

```bash
needlectl collections list
```

## Create a local connector to sync data

```bash
needlectl connectors local create --name "My Connector" --collection-id <collection-id> --folder <folder-to-sync>
```

Start the Needle daemon

```bash
needlectl daemon start
```

Your local connector will now watch the files in the folder and sync them to Needle. Initially, it will perform a full sync of the folder. In case you want to trigger a full sync manually, you can do so by restarting the daemon:

```bash
needlectl daemon restart
```

To stop the daemon, run:

```bash
needlectl daemon stop
```

To track the updates from the daemon, you can check the logs:

```bash
needlectl daemon logs
```

## File Synchronization

The daemon automatically:

- Monitors specified folders for changes
- Syncs new files to Needle platform
- Handles file updates
- Processes file deletions
- Maintains file state tracking

## Supported File Types

- PDF (application/pdf)
- Text (text/plain)
- Word (application/msword, application/vnd.openxmlformats-officedocument.wordprocessingml.document)
- Excel (application/vnd.ms-excel, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
- PowerPoint (application/vnd.ms-powerpoint, application/vnd.openxmlformats-officedocument.presentationml.presentation)
- CSV (text/csv)
- HTML (text/html)
- Calendar (text/calendar)

This list grows every day, for more up-to-date information please check [Needle](https://needle-ai.com/).

## Common Issues

1. "API key not found"

   - Solution: Set API key using `needlectl api-key set <key>`

2. "Collection not found"

   - Solution: Verify collection ID using `needlectl collections list`

3. "Upload failed"
   - Check API key validity
   - Verify file size limits
   - Check network connection

I'll create a comprehensive command reference based on the CLI commands shown in the code. Here's the markdown documentation:

## Command Reference

`needlectl` is a command-line tool to sync local files to Needle.

### API Key Management

- `needlectl api-key set <key>` - Set the Needle API key
- `needlectl api-key delete` - Delete the current API key

### Device Management

- `needlectl device info` - Show details of the current device

### Daemon Control

- `needlectl daemon start` - Start the Needle daemon
- `needlectl daemon status` - Check the Needle daemon status
- `needlectl daemon logs [--tail <n>]` - Show daemon logs
  - `--tail <n>`: Show last n lines
- `needlectl daemon stop` - Stop the Needle daemon
- `needlectl daemon restart` - Restart the Needle daemon
- `needlectl daemon sync` - Manually run a full sync

### Files Management

- `needlectl files upload -f <file...>` - Upload files to Needle
  - `-f, --files <file...>`: Path to file(s) to upload (can specify multiple)

### Collections Management

- `needlectl collections create --name <name> [--model <model>]` - Create a new collection
  - `--name <name>`: Collection name
  - `--model <model>`: Model to use (basilikum-minima, mate-meta) [default: basilikum-minima]
- `needlectl collections list` - List all collections
- `needlectl collections get <collection-id>` - Get collection details
- `needlectl collections stats <collection-id>` - Get collection statistics
- `needlectl collections search <collection-id> --text <text> [options]` - Search within a collection
  - `--text <text>`: Text to search for
  - `--max-distance <n>`: Maximum distance for search
  - `--top-k <n>`: Number of results to return

### Collection Files

- `needlectl collections files list <collection-id>` - List files in a collection
- `needlectl collections files add <collection-id> --name <names...> --url <urls...>` - Add files to a collection
  - `--name <names...>`: File names
  - `--url <urls...>`: File URLs

### Connectors Management

- `needlectl connectors list` - List all connectors

### Local Connectors

- `needlectl connectors local list` - List local connectors
- `needlectl connectors local get <connector-id>` - Show details of a local connector
- `needlectl connectors local create [options]` - Create a new local connector
  - `--name <name>`: Connector name
  - `--collection-id <ids...>`: Target collection IDs
  - `-f, --folder <path...>`: Folder paths to monitor (non-recursive)
  - `-r, --recursive <path...>`: Folder paths to monitor (recursive)
- `needlectl connectors local delete <connector-id>` - Delete a local connector