dep-copy-file/README.md

Blazing-fast file and directory copy for Node.js

# dep-copy-file

> Blazing-fast file and directory copy for Node.js — constant memory, maximum throughput.

## Features

- ⚡ **Constant memory** — sequential DFS tree walk keeps heap bounded regardless of tree size (tested: 182k files, 34k dirs → 73 MB peak)
- 🔒 **No OOM risk** — works with millions of files without exploding memory
- ✅ **Zero dependencies** — pure Node.js, no native modules
- 🔗 **Symlink aware** — detects and re-creates symlinks correctly
- 📁 **Empty directory safe** — empty directories are preserved
- 🛡️ **Error resilient** — one bad file won't cancel the whole operation
- 🎯 **Filter support** — skip files/directories with a callback
- 🔄 **Dual CJS/ESM** — works with both `require()` and `import`
- TypeScript types included

## Install

```bash
npm install dep-copy-file --save
```

## Quick start

```javascript
const copy = require('dep-copy-file');
// or: import copy from 'dep-copy-file';

// Copy a single file
await copy('./src/config.json', './dist/config.json');

// Copy an entire directory tree
await copy('./source', './backup');

// With options
const result = await copy('./source', './dest', {
  concurrency: 64,                  // max parallel copies (default: cpu × 2)
  overwrite: false,                 // skip existing files
  filter: (srcPath) => !srcPath.includes('node_modules'),
  onProgress: (stats) => console.log(`${stats.completed}/${stats.total}`),
});

console.log(result);
// {
//   files: 1500,
//   dirs: 42,
//   symlinks: 3,
//   succeeded: 1503,
//   failed: 0,
//   errors: []
// }
```

## API

### `copy(source, dest [, options])` → `Promise<CopyResult>`

| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `concurrency` | `number \| 'auto'` | `cpu × 2` (min 4) | Max parallel file/symlink copies. Pass `'auto'` for `cpu × 4` (max 128). Lower values reduce memory; higher values may increase throughput on fast SSDs. |
| `overwrite` | `boolean` | `true` | Allow overwriting existing files |
| `filter` | `(srcPath: string, entry: Dirent) => boolean` | — | Skip entries when returning `false` |
| `excludeSymlinks` | `boolean` | `false` | Skip symlinks entirely |
| `onProgress` | `(stats: CopyProgress) => void` | — | Called after each file copy completes |

### `CopyResult`

```typescript
interface CopyResult {
  files: number;        // total files discovered
  dirs: number;         // total directories created (including root)
  symlinks: number;     // total symlinks discovered
  succeeded: number;    // items copied successfully (files + symlinks)
  failed: number;       // items that failed
  errors: CopyError[];  // detailed failure info
}
```

## Architecture

1. **Sequential DFS tree walk** — directories are processed one at a time, depth-first. Only a single `readdir` result is in memory at any moment, keeping heap usage constant regardless of tree size.

2. **Parallel file/symlink I/O** — copies run through a concurrency pool (default `cpu × 2` in-flight). The pool is shared by both file copies and symlink creation, so the system is never overwhelmed.

3. **No deferred accumulation** — file copies start immediately as entries are discovered. No arrays of `[src, dst]` tuples are ever accumulated (those are the #1 cause of OOM on large trees).

4. **macOS APFS** systems get `COPYFILE_FICLONE` — copy-on-write clones that are near-instant regardless of file size.

5. **`mkdir` deduplication** — dest directories are created with `{ recursive: true }`, eliminating redundant syscalls and race conditions.

## Why sequential directory walking?

Parallel directory walking sounds faster, but `readdir` is microsecond-fast on modern file systems. The overhead of coordinating thousands of concurrent `readdir` calls — and holding all their results in memory — far outweighs any perceived speed gain. A sequential DFS walk keeps memory flat while letting the real bottleneck (file copy I/O) run at full parallelism.

## Migration from v1.x

The default export (`copyDep`) still works identically — just `await` the result:

```diff
- copy(source, dest).then(() => { ... }).catch(...)
+ const result = await copy(source, dest)
```

The return value is now a `CopyResult` object instead of `void`, but this is backward-compatible.