@cere/rob-cli/README.md

CLI tool for deploying and managing rafts and data sources

# ROB CLI

Command-line interface for deploying and managing rafts and data sources for the Cere Network's Real-time Orchestration Backend (ROB).

## System Requirements

* NodeJS >= 18.0.0

## Installation

```bash
npm install -g @cere/rob-cli
```

### As a Project Dependency

```bash
npm install --save-dev @cere/rob-cli
```

### Running with NPX

```bash
npx @cere/rob-cli [command] [options]
```

## Usage

All interactions with ROB CLI are of the form:

```bash
rob [command] [options]
```

or if using NPX:

```bash
npx @cere/rob-cli [command] [options]
```

### Help

To display basic commands and arguments:

```bash
rob --help
```

### Basic Commands

```bash
# Deploy data sources
rob deploy data-source ./data-sources.yaml

# Deploy rafts
rob deploy raft ./rafts.yaml

# Specify a data service ID
rob deploy raft ./rafts.yaml --dataServiceId 123456
rob deploy data-source ./data-sources.yaml --dataServiceId 123456

# Dry run (validate without making changes)
rob deploy raft ./rafts.yaml --dryRun

# Specify custom API base URL
rob deploy raft ./rafts.yaml --baseUrl https://rob.dev.cere.io
```

### Authentication

The CLI supports authentication using Basic Auth:

Use `--username` and `--password` flags

```bash
# Using basic auth
rob deploy data-source ./data-sources.yaml --username admin --password securepass

# Short form
rob deploy raft ./rafts.yaml -u admin -p securepass
```

### Data Service Scope

When working with rafts and data sources, it's important to understand that these resources are always scoped to a specific data service via the `dataServiceId` parameter. This helps in organizing and isolating resources for different applications or environments.

#### Using dataServiceId

The `dataServiceId` is used to:

1. **Namespace resources**: Each data service has its own namespace of rafts and data sources, allowing you to reuse IDs across different data services if needed.

2. **Filter resources during deployment**: When deploying or updating resources, the CLI uses the dataServiceId to check if a resource already exists within that specific data service, not globally.

3. **Prevent conflicts**: Resources with the same ID in different data services don't conflict with each other.

You can specify the `dataServiceId` in:
- The YAML configuration file (recommended)
- As a command-line parameter with `--dataServiceId`

```yaml
# Example with dataServiceId in YAML (recommended)
dataServiceId: "2106"

dataSources:
  - id: "postgres_main"
    # other properties...
```

```bash
# Example with dataServiceId as command line parameter
rob deploy data-source ./data-sources.yaml --dataServiceId 2106
```

The command-line parameter takes precedence over the value in the YAML file.

## Configuration Files

### Data Sources

```yaml
# data-sources.yaml
dataServiceId: "2106"  # Optional, can be provided via command line

dataSources:
  - id: "test_postgresql"  # Custom ID (optional)
    name: "Main PostgreSQL Database"
    type: "postgresql"
    config:
      host: "postgres.example.com"
      port: 5432
      database: "app_data"
      username: "${POSTGRES_USERNAME}"  # Uses environment variable
      password: "${POSTGRES_PASSWORD}"  # Uses environment variable
```

### Rafts

```yaml
# rafts.yaml
dataServiceId: "2106"  # Optional, can be provided via command line

rafts:
  - id: "quest_raft"  # Custom ID (optional)
    name: "Quest Raft"
    description: "Indexes and encapsulates quests."
    dataSourcesIds:
      - "test_postgresql"
    triggers:
      - id: "quest_trigger"  # Custom ID (optional)
        eventPattern: "campaign.quests.*"
        enabled: true
        description: "Trigger for quest progress events"
        conditions:
          - field: "campaignId"
            operator: "equals"
            value: "120"
    indexingScript:
      id: "quest_index_script"  # Custom ID (optional)
      scriptFile: "./scripts/quest_indexing.ts"
    queryOperations:
      - id: "get_quests"  # Custom ID (optional)
        name: "Get Quests"
        alias: "getQuests"
        description: "Retrieve quests by campaign and user IDs."
        scriptFile: "./scripts/get_quests_query.ts"
```

## Environment Variables

The CLI supports loading environment variables from a `.env` file and using them in your YAML configurations and for API configuration:

```
# .env file
POSTGRES_USERNAME=db_user
POSTGRES_PASSWORD=secure_password
ROB_API_URL=https://rob.stage.cere.io
```

By default, the CLI uses `https://rob.cere.io` as the base URL. This can be overridden with:
1. The `--baseUrl` command line parameter
2. The `ROB_API_URL` environment variable

## Writing Scripts

The CLI provides TypeScript type definitions for scripts (`Event`, `Context`, etc.) that are accessible globally in your script files without needing to import them.

Example:
```typescript
// scripts/quest_indexing.ts - No import needed for types!
export default async function(event: Event, context: Context) {
  const { postgres, redis, elasticsearch } = context;
  
  // Your script logic here...
}
```

## Dry Run Mode

The `--dryRun` flag allows you to validate your configuration files and see what changes would be made without actually applying them:

```bash
rob deploy raft ./rafts.yaml --dryRun
```

In dry run mode, the CLI will:
1. Parse and validate all YAML files
2. Load and process script files (if any)
3. Display detailed information about what would be created or updated
4. Show a summary of planned changes
5. Skip making any API calls that would modify resources

This is useful for:
- Validating your configuration syntax
- Checking script compilation
- Reviewing changes before applying them
- Testing authentication and connectivity

## Commands

### Deploy Data Sources

Deploy data sources defined in a YAML file:

```bash
rob deploy data-source ./data-sources.yaml
```

Options:
- `--dataServiceId <id>` - ID of the data service to associate with the data sources
- `--dryRun` - Validate and process the file without making any changes
- `--baseUrl <url>` - Custom API base URL

### Deploy Rafts

Deploy rafts defined in a YAML file:

```bash
rob deploy raft ./rafts.yaml
```

Options:
- `--dataServiceId <id>` - ID of the data service to associate with the rafts
- `--dryRun` - Validate and process the file without making any changes
- `--baseUrl <url>` - Custom API base URL

## YAML File Formats

### Data Sources YAML

```yaml
dataServiceId: "123456789"  # Optional, can be provided via command line

dataSources:
  - id: "postgres_main"
    name: "Main PostgreSQL Database"
    type: "postgresql"
    host: "postgres.example.com"
    port: 5432
    database: "app_data"
    username: "${POSTGRES_USERNAME}"  # Uses environment variable
    password: "${POSTGRES_PASSWORD}"  # Uses environment variable
```

### Rafts YAML

```yaml
dataServiceId: "123456789"  # Optional, can be provided via command line

rafts:
  - id: "quest_raft"
    name: "Quest Raft"
    description: "Indexes and encapsulates quests."
    dataSourcesIds:
      - "postgres_main"
      - "redis_cache"
    triggers:
      - id: "quest_progress_trigger"
        eventPattern: "campaign.quests.*"
        enabled: true
        conditions:
          - field: "campaignId"
            operator: "equals"
            value: "120"
    indexing:
      scriptFile: "./scripts/quest_indexing.ts"
    queries:
      - id: "get_quests"
        description: "Retrieve quests by campaign and user IDs."
        scriptFile: "./scripts/get_quests_query.ts"
```

## Project Structure for Scripts

When using script files in your raft configurations, it's recommended to organize your project like this:

```
my-rob-project/
├── .env                     # Environment variables
├── data-sources.yaml        # Data sources configuration
├── rafts.yaml               # Rafts configuration
└── scripts/
    ├── quest_indexing.ts    # Indexing script
    └── get_quests_query.ts  # Query script
```

## Publishing the Package

For maintainers of this package, to publish a new version:

1. The package is configured to be published as a public npm package (`--access public`)
2. Publishing is done via GitHub Actions workflow triggered manually
3. To publish a new version:
   - Go to the GitHub repository Actions tab
   - Select the "Publish CLI Package" workflow
   - Click "Run workflow"
   - Enter the new version number (e.g., "1.0.1")
   - Click "Run workflow"

The workflow will:
1. Check out the code
2. Install dependencies
3. Run tests
4. Update the package version
5. Build the package
6. Publish to npm

## License

Apache-2.0