@puberty-labs/clits/docs/AI_INTEGRATION.md

CLiTS (Chrome Logging and Inspection Tool Suite) is a powerful Node.js library for AI-controlled Chrome browser automation, testing, and inspection. Features enhanced CSS selector support (:contains(), XPath), dry-run mode, element discovery tools, and co

# 🤖 CLITS AI Integration Guide

## Overview

CLITS (Chrome Log Inspector & Troubleshooting System) is designed to work seamlessly with AI assistants for automated browser testing and debugging. This guide provides essential setup and usage instructions specifically for AI agents.

## 🚨 **CRITICAL: Human-in-the-Loop Protocol**

### **MANDATORY USER CONFIRMATION REQUIRED**

**⚠️ BEFORE ANY BROWSER AUTOMATION: AI assistants MUST follow this protocol**

1. **PAUSE AUTOMATION** - Do not attempt browser control without user permission
2. **PROMPT USER** - Explicitly ask for permission to control their browser
3. **WAIT FOR CONFIRMATION** - Wait for explicit user approval (Y/N response)
4. **RESUME AUTOMATION** - Only proceed after receiving user consent

### **Authentication & Session Management**

**🔐 LOGIN REQUIREMENTS:**
- **User login sessions cannot be automated** - Require human intervention
- **AI cannot handle authentication** - User must be logged in manually
- **Session cookies/profiles** - Use existing browser sessions only

**Protocol for Authentication Needed:**
```
AI: "I need to interact with [website]. Please ensure you are logged in to your account first."
User: [Logs in manually]  
AI: "May I proceed with browser automation? (Y/N)"
User: "Y"
AI: [Proceeds with CLITS commands]
```

## 🚀 Getting Started for AI Assistants

### **Step 1: Environment Setup**

**Chrome with Remote Debugging (REQUIRED):**
```bash
# macOS
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" --remote-debugging-port=9222 --user-data-dir=/tmp/chrome-debug

# Windows  
"C:\Program Files\Google\Chrome\Application\chrome.exe" --remote-debugging-port=9222 --user-data-dir=c:\temp\chrome-debug

# Linux
google-chrome --remote-debugging-port=9222 --user-data-dir=/tmp/chrome-debug
```

**Verify Chrome Connection:**
```bash
# Check if Chrome debugging is running
curl http://localhost:9222/json/version
```

Expected response: WebSocket URL like `ws://127.0.0.1:9222/devtools/browser/[ID]`

### **Step 2: CLITS Installation & Verification**

**Global Installation (Recommended for AI):**
```bash
npm install -g @puberty-labs/clits
```

**Verify Installation:**
```bash
clits --version
# Should show current version (v1.2.0+)
```

**Test Connection:**
```bash
clits inspect --list-tabs
# Should show open browser tabs
```

### **Step 3: Standard AI Workflows**

**Basic Navigation & Interaction:**
```bash
# 1. Navigate to URL
clits navigate --url "https://example.com"

# 2. Wait for page load
clits wait --timeout 3000

# 3. Take screenshot for context
clits vision --screenshot

# 4. Click button (with user permission)
clits interact --click "[data-testid='submit-button']"

# 5. Extract text/data
clits extract --text ".result-message"
```

## 🎯 AI-Specific Command Usage

### **Element Targeting Strategies (Priority Order)**

1. **data-testid attributes** (Primary - Most Reliable)
   ```bash
   clits interact --click "[data-testid='save-button']"
   ```

2. **Accessibility attributes** (Secondary - Semantic)
   ```bash
   clits interact --click "[aria-label='Submit form']"
   ```

3. **data-clits attributes** (OnDeck specific)
   ```bash
   clits interact --click "[data-clits='submit-action']"
   ```

4. **Visible text** (Fallback - User-friendly)
   ```bash
   clits interact --click-text "Save Changes"
   ```

5. **CSS selectors** (Last resort - Fragile)
   ```bash
   clits interact --click ".btn-primary:nth-child(2)"
   ```

### **AI-Optimized Command Flags**

**JSON Output for Structured Data:**
```bash
# Get structured error responses for AI error handling
clits extract --chrome --chrome-port 9222 --json-errors
clits interact --click ".button" --json-errors
clits navigate --url "https://example.com" --json-errors

# Get tab information as JSON
clits inspect --tabs --json
```

**Non-Interactive Mode:**
```bash
# Skip confirmations where safe
clits vision --screenshot --silent
```

### **Wait Strategies for AI**

**Recommended Wait Patterns:**
```bash
# Wait for page load
clits wait --timeout 5000

# Wait for element to appear
clits wait --element "[data-testid='content']" --timeout 10000

# Wait for network requests to complete
clits wait --network-idle --timeout 8000
```

## 🛠 Common AI Workflows

### **Workflow 1: Form Submission**
```bash
# 1. Navigate and wait
clits navigate --url "https://app.example.com/form"
clits wait --timeout 3000

# 2. Fill form fields
clits interact --type "[data-testid='name-input']" --text "John Doe"
clits interact --type "[data-testid='email-input']" --text "john@example.com"

# 3. Submit with screenshot validation
clits vision --screenshot  # Before submission
clits interact --click "[data-testid='submit-button']"
clits wait --timeout 5000
clits vision --screenshot  # After submission
```

### **Workflow 2: Data Extraction**
```bash
# 1. Navigate to data page
clits navigate --url "https://dashboard.example.com"
clits wait --timeout 3000

# 2. Extract structured data
clits extract --text ".data-table tbody tr" --json
clits extract --attributes ".data-items" --attr "data-value" --json
```

### **Workflow 3: Multi-Tab Management**
```bash
# 1. List current tabs
clits inspect --tabs

# 2. Switch to specific tab
clits tab --switch-tab 1

# 3. Navigate in current tab
clits navigate --url "https://newpage.com"

# 4. Return to previous tab
clits tab --tab-prev
```

## ⚠️ Error Handling for AI

### **Common Error Scenarios**

**ELEMENT_NOT_FOUND:**
```bash
# Recovery strategy:
clits wait --timeout 5000  # Wait longer
clits vision --screenshot  # Visual diagnosis
clits inspect --find-clickable  # Find available elements
```

**AUTH_REQUIRED:**
```bash
# Requires human intervention
AI: "Authentication required. Please log in manually and confirm when ready."
[Wait for user confirmation]
```

**CHROME_NOT_CONNECTED:**
```bash
# Check Chrome debugging setup
curl http://localhost:9222/json/version
# Restart Chrome with debugging if needed
```

### **Debugging Commands for AI**

**Visual Diagnosis:**
```bash
clits vision --screenshot  # Take current state screenshot
clits inspect --find-clickable  # List all clickable elements
clits inspect --list-tabs  # Check browser state
```

**Network Diagnosis:**
```bash
clits extract --network --filter-errors  # Check for network issues
clits extract --console --log-levels error  # Check console errors
```

## 🔧 Advanced AI Configuration

### **Profile Management**
```bash
# Use specific browser profile
clits configure --chrome-profile "/path/to/profile"

# Use existing user session
clits configure --use-existing-session
```

### **Performance Optimization**
```bash
# Disable images for faster loading
clits configure --chrome-args "--disable-images"

# Headless mode (when visual verification not needed)
clits configure --headless
```

## 📋 AI Integration Checklist

**Before Starting Automation:**
- [ ] Chrome running with remote debugging (port 9222)
- [ ] User manually logged in to required websites
- [ ] CLITS installed and verified (`clits --version`)
- [ ] Connection test passed (`clits inspect --list-tabs`)
- [ ] User permission obtained for browser automation

**During Automation:**
- [ ] Use semantic element targeting (data-testid, aria-label)
- [ ] Take screenshots before major actions
- [ ] Implement proper wait strategies
- [ ] Handle errors gracefully with recovery attempts

**Error Recovery:**
- [ ] Visual diagnosis with screenshots
- [ ] Element availability checks
- [ ] Network/console error inspection
- [ ] Human intervention requests when needed

## 🚀 Next Steps

1. **Read AI_EXAMPLES.md** - Copy-pasteable command sequences
2. **Review ERROR_CODES.md** - Structured error handling (coming soon)
3. **Check API_REFERENCE.md** - Complete command documentation

---

**Need Help?** 
- Check existing GitHub issues for common problems
- Review TROUBLESHOOTING.md for debugging steps
- Use `clits --help` for command-specific guidance