vmsnap/README.md

A Node based backup and backup rotation tool for KVM domains.

# VMSnap

[![Node.js Package](https://github.com/slackdaystudio/vmsnap/workflows/Node.js%20Package/badge.svg)](https://github.com/slackdaystudio/vmsnap/actions)
[![Test Coverage](https://img.shields.io/badge/coverage-92%25-brightgreen.svg)](https://github.com/slackdaystudio/vmsnap)
[![Security Audit](https://img.shields.io/badge/audit-0%20vulnerabilities-brightgreen.svg)](https://github.com/slackdaystudio/vmsnap)

VMSnap is a production-ready Node.js tool designed to simplify the process of creating and managing
snapshots of KVM domains. Backups generated by VMSnap are incremental, if 
possible. VMSnap is also capable of performing intelligent backup rotation 
with configurable retention policies.

This README provides an overview of the project, installation instructions, 
usage guidelines, and contribution information.

## Features

- 🔍 **Query KVM domains** for comprehensive backup status with integrity checks
- 💾 **Create incremental snapshots** of virtual machines with checkpoint management
- 🧹 **Delete unnecessary bitmaps and checkpoints** with selective or bulk cleanup
- 📅 **Intelligent backup rotation** with configurable periods (monthly, quarterly, bi-annual, yearly)
- 🛡️ **Enterprise-grade reliability** with 92% test coverage and comprehensive error handling
- 📊 **Multiple output formats** including JSON, YAML, and human-readable reports
- ⚡ **Performance optimized** with concurrent operations and efficient resource usage

## Requirements

You must have the following on your host OS:
 - [virsh](https://libvirt.org/manpages/virsh.html)
 - [qemu-img](https://qemu-project.gitlab.io/qemu/tools/qemu-img.html)
 - [virtnbdbackup](https://github.com/abbbi/virtnbdbackup)

Getting these installed is out of scope for this doc.

The app will let you know if you are missing any required programs when you 
start running commands with it.

## Installation

To install VMSnap, follow these steps:

```
npm install -g vmsnap
```

### Local

You may also choose to install VMSnap by checking the code out and running it
locally. To run localy, do the following:

1. Clone the repository:
   ```sh
   git clone git@github.com:slackdaystudio/vmsnap.git
   ```
2. Navigate to the project directory:
   ```sh
   cd vmsnap
   ```
3. Install the required dependencies:
   ```sh
   npm install
   ```

## Usage

This usage guide assumes you have installed VMSnap via the `npm install -g vmsnap`
command. Doing so will install VMSnap which includes a vmsnap bin.

> **Tip:** You may execute the same commands from a local checkout by swapping
> out the name of the bin for `npm run vmsnap --`. For example, to run a status
> check from a local version you first go to your code checkout and then run
> `npm run vmsnap -- --domains=vm1,vm2 --status`

## Command Line Switches

The following CLI switches are available when invoking VMSnap.

| Switch         | Status | Backup | Scrub  |  Type   |                     Examples/Notes                                           |
|----------------|--------|--------|--------|---------|------------------------------------------------------------------------------|
| domains        | ✅     | ✅     | ✅     | string  | "vm1" or "vm1,vm2,etc" or "*"                                                |
| status         | ✅     | -      | -      | boolean | Querys the domain(s)                                                         |
| backup         | -      | ✅     | -      | boolean | Does an incremental backup (if possible)                                     |
| scrub          | -      | -      | ✅     | boolean | Cleans checkpoints and bitmaps off of the domain                             |
| output         | ✅     | ✅     | -      | string  | A full path to a directory where backups are placed                          |
| verbose        | ✅     | -      | -      | boolean | Prints out extra information when running a status check                     |
| machine        | ✅     | -      | -      | boolean | Removes some output from the status command                                  |
| json           | ✅     | -      | -      | boolean | Outputs the status command is JSON                                           |
| yaml           | ✅     | -      | -      | boolean | Output YAML from the status command (aliased to `--yml`)                     |
| raw            | -      | ✅     | -      | boolean | Enables raw disk handling                                                    |
| groupBy        | ✅     | ✅     | -      | string  | Defines how backups are grouped on disk (month, quarter, bi-annual or year)  | 
| prune          | -      | ✅     | -      | boolean | Rotates backups by **deleting** last periods backup*                         |
| pretty         | ✅     | -      | -      | boolean | Pretty prints disk sizes (42.6 GB, 120 GB, etc)                              |
| checkpointName | -      | -      | ✅     | string  | The name of the checkpoint to delete (no effect when scrubType=*)            |
| scrubType      | -      | -      | ✅     | string  | The type of item to scrub (checkpoint, bitmap, both, or * for ALL)           |

*\*This happens on or after the the middle of the current period (15 days monthly, 45 days quarterly, 90 days bi-annually or 180 yearly)*

## Status

The default action for VMSnap is to display a status report for VMs supplied.

```sh
vmsnap --domains=vm1 --status
```

> **Tip:** The `--domains` flag also accepts a comma seperated list of domains. 
> You may also pass in "\*" to select all found VMs. This is applicable to 
> backing up, scrubbing, or querying VMs.
>
> **Tip:** The `--status` flag may be omited.  Leaving it in is useful when 
> constructing backup and scrub commands because you may test the command by 
> querying the status of the domain.  If that query works you then swap the 
> `--status` flag for the `--backup` or `--scrub` flag, as appropriate.

This could return the following information if ran, as an example.

```
Status for vm1:
  Overall status: OK
  Checkpoints found for vm1:
    virtnbdbackup.0
    virtnbdbackup.1
    virtnbdbackup.2
  Eligible disks found for vm1:
    vda
      Virtual size: 107374182400
      Actual size: 14286573568
      Bitmaps found for vda:
          virtnbdbackup.0
          virtnbdbackup.1
          virtnbdbackup.2
```

>**Tip:** Pass in an `output=/PATH/TO/BACKUP_ROOT` flag to see statistics about 
> the backups already saved to disk.  VMSnap will perform additional integrity
> checks using the information it collects.

Machine parsable output is possible with the `--json` and `--yaml` flags in
combination with the `--machine` flag.

For example, running the following command...

```sh
vmsnap --domains=vm1 --machine --json
```

..will produce something like the following.

```json
{"vm1":{"checkpoints":["virtnbdbackup.0","virtnbdbackup.1","virtnbdbackup.2"],"disks":[{"disk":"vda","virtualSize":107374182400,"actualSize":14293934080,"bitmaps":["virtnbdbackup.0","virtnbdbackup.1","virtnbdbackup.2"]}],"overallStatus":0}}
```

## Backup

Backups are always incremental unless VMSnap is cutting a new periods first 
backup.  Subsequent backups will be incremental meaning only the changes from
the VM will be captured.

Create a snapshot for `vm1` and output it to the `tmp` direcory:

```sh
vmsnap --domains=vm1 --output=/tmp --backup
```

The above command will create a the backup for the domain. This creates a
checkpoint and dirty bitmap on the VM file and deposits the backup to the `/tmp`
directory. 

> **Tip:** Make sure you can read and write to the target directory in `--output`

You may also specify the `--groupBy` flag to tell VMSnap how to group your files
on disk. Look at the table below for more information.

| groupBy Flag | Middle Mark | Sample Folder Name                |
|--------------|-------------|-----------------------------------|
| month        | 15d         | vmsnap-backup-monthly-2024-11     |
| quarter      | 45d         | vmsnap-backup-quarterly-2024-Q4   |
| bi-annual    | 90d         | vmsnap-backup-bi-annually-2024-p2 |
| year         | 180d        | vmsnap-backup-yearly-2024         | 

>**Tip:** If you **do not** set the `groupBy` flag the default period is assumed
> to be "month."

### Backup Pruning (Caution)

> **Note:** Pruning is destructive.  Be careful when using it and check your 
 backups frequently!

Pruning backups may be done by setting `--prune` on the backup command.
This flag will automatically delete last periods backup once the middle of the
current backup period comes up.

Pruning provides a sliding window for the given period of +/-50% depending upon
where you are in the backup cycle.  For example, setting the `groupBy` flag to
"month" would mean you would have 2-6 weeks of backups on hand at any given 
time.

### Raw Disk Handling

You can turn on raw disk handling by setting the `--raw` flag.

## Scrubbing

> **Note:** These commands are inherently destructive, be careful!

It is occasionally useful to be able to scrub one or more checkpoints or bitmaps
from your domain.  Doing so is fairly straight forward with VMSnap but please do
be cautious.  

Use this command to scrub a single bitmap from your backup disks.  Keep in mind 
that bitmaps are stored on a per disk basis.  VMSnap will scrub each disk of the 
bitmap if it find it.

```sh
vmsnap --domains=vm1 --scrub --scrubType=bitmap --checkpointName=virtnbdbackup.17
```

To scrub a domain of **ALL** checkpoints and bitmaps

```sh
vmsnap --domains=vm1 --scrub --scrubType=*
```

## Testing & Quality Assurance

VMSnap maintains enterprise-grade quality standards with comprehensive testing:

### Test Coverage
- **188 unit tests** across all modules
- **50 integration tests** against real KVM virtual machines
- **92.07% line coverage** (target: 90%+)
- **85.71% function coverage** (target: 95%+)
- **Complete CI/CD integration** with GitHub Actions

### Running Tests
```sh
# Run all tests
npm test

# Run tests with coverage report
npm run test:coverage

# Run tests in watch mode during development
npm run test:watch

# Run only unit tests
npm run test:unit
```

### Integration Tests

VMSnap includes a comprehensive integration test suite that tests real backup operations against actual KVM virtual machines. These tests require a KVM-enabled environment.

#### Prerequisites for Integration Tests

- **KVM/QEMU** with hardware virtualization support
- **libvirt** daemon running (`libvirtd`)
- **virtnbdbackup** installed and configured
- **qemu-img** and **virsh** command-line tools
- User must have permissions to create/manage VMs (typically `libvirt` and `kvm` groups)

#### Running Integration Tests Locally

```sh
# Verify KVM is available
ls -la /dev/kvm

# Verify libvirt connection
virsh version

# Run integration tests
npm run test:integration

# Optional: Setup test VMs manually
npm run test:integration:setup

# Optional: Cleanup test environment
npm run test:integration:cleanup
```

#### Integration Test Categories

| Test Suite | Tests | Description |
|------------|-------|-------------|
| `backup-operations.test.js` | 6 | Single/multiple VM backups, wildcards, selective domains |
| `incremental-backup.test.js` | 7 | Full and incremental backup chains, disk change handling |
| `rotation-pruning.test.js` | 7 | Monthly/quarterly/bi-annual/yearly grouping and pruning |
| `scrubbing-operations.test.js` | 6 | Checkpoint and bitmap cleanup (handles offline VMs) |
| `status-commands.test.js` | 11 | Status output in text/JSON/YAML formats, multiple VMs |
| `error-scenarios.test.js` | 13 | Error handling, invalid inputs, concurrent execution |

#### Running on a Self-Hosted GitHub Actions Runner

Integration tests can run in CI using a self-hosted runner with KVM support:

1. **Set up a self-hosted runner** with nested virtualization enabled:
   ```sh
   # On the runner host, verify KVM support
   egrep -c '(vmx|svm)' /proc/cpuinfo  # Should return > 0

   # Install required packages (Ubuntu/Debian)
   sudo apt-get install -y qemu-kvm libvirt-daemon-system libvirt-clients virtinst qemu-utils

   # Install virtnbdbackup
   pip3 install virtnbdbackup

   # Add runner user to required groups
   sudo usermod -aG kvm,libvirt $USER

   # Start libvirt
   sudo systemctl enable --now libvirtd
   ```

2. **Configure the runner** following [GitHub's self-hosted runner documentation](https://docs.github.com/en/actions/hosting-your-own-runners)

3. **Update the workflow** to use your self-hosted runner:
   ```yaml
   # In .github/workflows/integration-tests.yml
   jobs:
     integration-tests:
       runs-on: self-hosted  # Change from ubuntu-latest
   ```

#### Why Integration Tests Skip on GitHub-Hosted Runners

Standard GitHub-hosted runners (`ubuntu-latest`) don't expose `/dev/kvm` because nested virtualization is disabled. The integration test workflow automatically detects this and:
- Skips integration tests with a warning
- Still runs all unit tests successfully

This ensures CI doesn't fail while allowing full integration testing on capable environments.

### Code Quality
```sh
# Lint code
npm run lint

# Format code
npm run format

# Check formatting
npm run check-format

# Build project
npm run build
```

## Security

VMSnap prioritizes security and maintains:
- **0 known vulnerabilities** (regularly audited)
- **Latest secure dependencies** with automated updates
- **Comprehensive error handling** to prevent data corruption
- **Input validation** on all user-provided parameters
- **Safe backup operations** with integrity checks

Run security audit:
```sh
npm audit
```

## Development

### Prerequisites
- Node.js 18+ (specified in package.json engines)
- npm or yarn package manager

### Setup Development Environment
1. Clone and setup:
   ```sh
   git clone git@github.com:slackdaystudio/vmsnap.git
   cd vmsnap
   npm install
   ```

2. Run in development mode:
   ```sh
   # Watch mode for automatic rebuilds
   npm run watch
   
   # Run locally without building
   npm run vmsnap -- --domains=vm1 --status
   ```

3. Before committing:
   ```sh
   npm run lint          # Check code style
   npm run test          # Run all tests
   npm run build         # Verify build works
   ```

### Project Structure
```
vmsnap/
├── libs/                 # Core modules
│   ├── general.js       # Utility functions, dependency checking, error handling
│   ├── libnbdbackup.js  # Main backup orchestration with virtnbdbackup
│   ├── print.js         # Output formatting (text, JSON, YAML)
│   ├── qemu-img.js      # QEMU image operations & bitmap management
│   ├── serialization.js # Status collection & integrity analysis
│   └── virsh.js         # KVM domain & checkpoint management
├── test/
│   ├── unit/            # 188 unit tests across all modules
│   │   └── libs/        # Module-specific unit tests
│   └── integration/     # 50 integration tests with real KVM VMs
│       ├── helpers/     # VM lifecycle management & test assertions
│       │   ├── vm-manager.js      # Create/destroy test VMs
│       │   ├── test-assertions.js # Backup verification helpers
│       │   └── cleanup-helpers.js # Environment cleanup
│       ├── setup/       # Shell scripts for test environment
│       └── tests/       # 6 integration test suites
├── dist/                # Built output (generated)
└── vmsnap.js           # Main CLI entry point with argument parsing
```

### Architecture
VMSnap follows a modular architecture with clear separation of concerns:
- **CLI Layer**: Argument parsing and user interaction (vmsnap.js)
- **Service Layer**: Core backup logic (libnbdbackup.js)
- **Utility Layer**: System integration (virsh.js, qemu-img.js)
- **Data Layer**: Status collection and serialization

## Performance & Reliability

VMSnap is designed for production environments with emphasis on reliability:

### Reliability Features
- **Comprehensive error handling** with specific exit codes
- **Input validation** prevents invalid operations
- **Dependency checking** ensures required tools are available  
- **Atomic operations** with proper rollback on failures
- **Lock file management** prevents concurrent execution conflicts

### Performance Optimizations
- **Concurrent operations** where safe (status checks, validation)
- **Efficient resource usage** with streaming and memory management
- **Incremental backups** minimize storage and time requirements
- **Intelligent pruning** maintains optimal backup retention
- **Fast status reporting** with optimized disk scanning

### Monitoring & Logging
- **Structured logging** with configurable verbosity levels
- **Progress reporting** for long-running operations  
- **Exit codes** for integration with automation tools
- **JSON/YAML output** for monitoring system integration

# Contributing

We welcome contributions! VMSnap maintains high standards for code quality, testing, and security.

## Contribution Guidelines

1. **Fork and setup**:
   ```sh
   git clone git@github.com:YOUR_USERNAME/vmsnap.git
   cd vmsnap
   npm install
   ```

2. **Create a feature branch**:
   ```sh
   git checkout -b feature/your-feature-name
   ```

3. **Follow development standards**:
   - Write tests for new functionality (maintain >90% coverage)
   - Follow existing code style (ESLint + Prettier)
   - Update documentation as needed
   - Ensure all checks pass:
     ```sh
     npm run lint          # Code style
     npm run test          # All tests pass
     npm run build         # Successful build
     npm audit            # No security issues
     ```

4. **Commit your changes**:
   ```sh
   git commit -m "feat: add new backup validation feature"
   ```
   Follow [Conventional Commits](https://conventionalcommits.org/) format.

5. **Push and create PR**:
   ```sh
   git push origin feature/your-feature-name
   ```
   Then create a pull request with:
   - Clear description of changes
   - Test results and coverage impact
   - Screenshots/examples if applicable

## Testing Requirements

All contributions must maintain our testing standards:
- **Unit tests** for new functions/modules
- **Integration tests** for complex workflows  
- **Edge case coverage** for error scenarios
- **Documentation updates** for new features

Run the full test suite before submitting:
```sh
npm run test:coverage
```

## License

This project is licensed under the MIT License. See the [LICENSE](LICENSE) file
for details.

## Release Notes

### v1.1.0-beta (Latest)

🎉 **Major Quality & Security Release**

**New Features:**
- ✨ Comprehensive test suite with 188 unit tests and 50 integration tests
- 📊 92% test coverage across all modules
- 🛡️ Zero security vulnerabilities (fixed 7 issues)
- 🚀 Enhanced performance with edge case handling
- 📈 Improved error reporting and validation with proper exit codes

**Integration Test Suite:**
- 🧪 Full end-to-end testing against real KVM virtual machines
- 💾 Tests for backup operations (single VM, multiple VMs, wildcards)
- 🔄 Incremental backup chain validation
- 📅 Backup rotation testing (monthly, quarterly, bi-annual, yearly)
- 🧹 Checkpoint and bitmap scrubbing verification
- 📊 Status command output validation (text, JSON, YAML)
- ⚠️ Comprehensive error scenario coverage
- 🔌 Automatic libvirt connection handling (system/session)
- ⏸️ Graceful handling of offline VMs (copy mode vs checkpoints)

**Security & Dependencies:**
- 🔒 Updated all dependencies to latest secure versions
- 🛠️ Removed unnecessary React/Babel dependencies
- ⚡ Updated build tools (esbuild 0.24→0.27)
- 🔍 Added automated security auditing

**Developer Experience:**
- 🧪 Full test infrastructure with Vitest
- 📋 Enhanced documentation and architecture guides
- 🔧 Improved development workflow and standards
- 📝 Updated contribution guidelines with testing requirements

**Bug Fixes:**
- 🐛 Fixed error code propagation (errors now return proper exit codes)
- 🐛 Fixed virtnbdbackup command arguments
- 🐛 Improved empty domain list handling with clear error messages

**Infrastructure:**
- ✅ GitHub Actions CI/CD integration
- 📦 Automated testing and security checks
- 🏗️ Optimized build process and distribution
- 🔄 Sequential test execution for VM operations

## Contact

For any questions or feedback, please open an issue on GitHub.

---

**VMSnap v1.1.0-beta** - Production-ready KVM backup solution with 238 tests (188 unit + 50 integration) and enterprise-grade reliability.