codecanon/README.md

CLI tool that downloads documentation from 3rd party libraries, converts them to Markdown, and optimizes for LLMs

# CodeCanon

**Stop copy-pasting docs. Start getting answers.**

CodeCanon transforms your AI coding assistant from a general-purpose tool into a project-aware expert by automatically building a local, optimized knowledge base from your dependencies' documentation.

## The Problem: The AI "Context Gap"

Modern AI assistants like GitHub Copilot, ChatGPT, and Claude are incredibly powerful, but they have a critical weakness: they work best with rich, relevant context. Without proper context, they're prone to "hallucinations"—generating plausible but incorrect code.

Currently, developers bridge this gap manually by:

- 📋 Copy-pasting documentation into chat interfaces
- ✍️ Writing detailed prompts with library-specific context
- 📁 Creating ad-hoc instruction files like `copilot-instructions.md`

This manual process is time-consuming, error-prone, and doesn't scale across teams.

## The Solution: Context-as-Code

CodeCanon automates the entire Retrieval-Augmented Generation (RAG) pipeline for your project dependencies. With a single command, it transforms scattered web documentation into a local, queryable knowledge base that makes your AI assistants dramatically more effective.

## Key Features

- 🤖 **AI-First Design**: Specifically optimized for AI coding assistants (GitHub Copilot, ChatGPT, Claude)
- � **Multi-Source Intelligence**: Automatically discovers and fetches documentation from npm, GitHub, and official docs
- 🎯 **Smart Processing**: Converts and chunks documentation for optimal LLM comprehension
- 💾 **Local-First**: All processed documentation stays in your project (`.canon/` directory)
- 🔍 **Semantic Search**: Query your dependencies using natural language
- 👥 **Team-Ready**: Share context configurations across your development team
- ⚡ **CLI-Native**: Integrate seamlessly into your existing development workflow

## The "Git" Model for Context

CodeCanon follows the proven "Git model" for managing project state. Just like `.git/` manages your version control, `.canon/` manages your AI context:

```bash
.canon/
├── settings.json          # Project configuration
├── cache/                 # Raw and processed documentation
│   └── <package>/
│       └── <version>/
│           ├── raw/       # Original scraped content
│           └── chunks/    # Semantic chunks for AI
├── context/
│   ├── llms.txt          # Context index (llms.txt standard)
│   └── llms-full.txt     # Full concatenated context
└── db/                   # Vector database
```

## Quick Start

### Installation

```bash
npm install -g codecanon
```

### Initialize Your Project

```bash
# Create .canon workspace in your project
canon init

# Add documentation for your dependencies
canon add react
canon add express
canon add zod

# Generate context files for AI assistants
canon context
```

### Use with Your AI Assistant

Once you've run `canon context`, you can use the generated context with any AI assistant:

**GitHub Copilot Chat:**

```plaintext
Based on the context in #file:.canon/context/llms-full.txt, show me how to create a form with validation using react-hook-form and zod.
```

**ChatGPT/Claude:**

Upload or reference `.canon/context/llms-full.txt` in your conversation for accurate, project-specific assistance.

## Commands

### Core Commands (Free)

- `canon init [--model <model>]` - Initialize workspace with optional default LLM model
- `canon add <package>[@version]` - Fetch and process package documentation
- `canon sync` - Auto-sync all dependencies from package.json
- `canon context` - Generate final context files for AI assistants
- `canon query "<question>"` - Interactive Q&A with your documentation
- `canon login` - Authenticate with codecanon.ai

## Why CodeCanon?

### For Individual Developers

"Stop copy-pasting docs and start getting answers. CodeCanon makes your AI assistant an expert on the libraries you actually use, directly in your project."

### For Development Teams

"Ensure your entire team's AI assistants are working from the same, up-to-date set of documentation and best practices. Improve code consistency and accelerate onboarding."

## Development & Contributing

### Setup

```bash
# Install dependencies
pnpm install

# Run in development mode
pnpm dev

# Build for production
pnpm build
```

### Project Structure

- `cli/` - Command implementations (`init`, `add`, `query`, etc.)
- `lib/` - Core library functions (workspace, package handling, query engine)
- `.canon/` - Generated workspace directory (created by `canon init`)

### Contributing

We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.

1. Fork the repository
2. Create your feature branch (`git checkout -b feat/amazing-feature`)
3. Commit your changes (`git commit -m 'feat: add some amazing feature'`)
4. Push to the branch (`git push origin feat/amazing-feature`)
5. Open a Pull Request

## Roadmap

### Phase 1: Core Foundation ✅

- [x] CLI framework with Commander.js
- [x] Workspace initialization (`.canon/` directory)
- [x] Package metadata fetching from npm
- [x] Basic documentation scraping and processing
- [x] Context file generation (`llms.txt` standard)

### Phase 2: Enhanced Processing 🚧

- [ ] Intelligent documentation discovery (homepage, GitHub, docs sites)
- [ ] Advanced semantic chunking algorithms
- [ ] Support for multiple documentation formats
- [ ] Caching and incremental updates

### Phase 3: Vector Database 📋

- [ ] Vector embeddings and similarity search
- [ ] `canon query` interactive Q&A
- [ ] Private npm registry support
- [ ] codecanon.ai authentication and billing

### Phase 4: Team & Enterprise 🎯

- [ ] Shared workspace configurations
- [ ] Private repository documentation scraping
- [ ] VS Code extension
- [ ] Additional language ecosystems (PyPI, Cargo, etc.)

## Community & Support

- � [Discord Community](https://discord.gg/codecanon) - Get help and share ideas
- � [GitHub Issues](https://github.com/kriasoft/codecanon/issues) - Report bugs and request features
- � [Documentation](https://docs.codecanon.ai) - Comprehensive guides and API reference
- � [Twitter](https://twitter.com/codecanonai) - Latest updates and tips

## License

This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.

---

Built with ❤️ for the developer community

_Making AI assistants smarter, one dependency at a time._