ai-debug-local-mcp/AI-FRAMEWORK-DETECTION-GUIDE.md

🎯 ENHANCED AI GUIDANCE v4.1.2: Dramatically improved tool descriptions help AI users choose the right tools instead of 'close enough' options. Ultra-fast keyboard automation (10x speed), universal recording, multi-ecosystem debugging support, and compreh

# 🤖 AI Framework Detection & Tool Selection Guide

## **CRITICAL: How AI Models Should Choose Between Universal Workflows vs Specialized Tools**

This guide provides explicit decision-making rules for AI models working with the ai-debug-local-mcp toolset.

## 🎯 **Decision Matrix: Which Tools to Use**

### **Flutter Web Applications**
**Detection Signals:**
- URL contains "flutter"
- DOM contains `<flutter-view>` or `flt-*` elements
- User mentions "Flutter", "widgets", "Dart"
- Canvas-heavy rendering detected
- `window.flutterConfiguration` exists

**Tool Selection:**
```typescript
✅ USE THESE for Flutter Web:
- flutter_quantum_interact (primary interaction tool)
- flutter_quantum_find (widget discovery)  
- flutter_widget_tree (structure analysis)
- flutter_enable_accessibility (a11y setup)

❌ AVOID for Flutter Web:
- smart_form_interaction (use flutter_quantum_interact instead)
- handle_dynamic_elements (use flutter_quantum_find instead)
- Standard DOM selectors (won't work with Canvas rendering)
```

### **Phoenix LiveView Applications**
**Detection Signals:**
- DOM contains `phx-*` attributes (`phx-click`, `phx-submit`, etc.)
- User mentions "Phoenix", "LiveView", "Elixir"
- WebSocket connections to `/live/websocket`
- `window.liveSocket` exists

**Tool Selection:**
```typescript
✅ USE THESE for Phoenix LiveView:
- create_workflow_template (excellent compatibility)
- navigate_to_url (handles LiveView navigation)
- smart_form_interaction (works with LiveView forms)
- execute_user_journey (full workflow orchestration)

✅ SELECTOR PATTERNS:
- '[phx-click="update_data"]' 
- '[data-testid="liveview-content"]'
- '[phx-hook="Dashboard"]'
```

### **Standard Web Applications (React, Vue, Next.js, etc.)**
**Detection Signals:**
- Standard HTML/DOM structure
- Framework-specific attributes (`data-reactroot`, `v-*`, etc.)
- No Flutter or Phoenix-specific markers

**Tool Selection:**
```typescript
✅ USE THESE for Standard Web:
- create_workflow_template (primary orchestration)
- smart_form_interaction (complex forms)
- handle_dynamic_elements (modals, dropdowns)
- simulate_file_upload (file handling)
- All Universal Workflow tools work perfectly
```

## 🔧 **Framework Detection Implementation**

### **Automatic Detection Pattern**
```typescript
// AI models should use this decision logic:

async function selectOptimalTools(url: string, userContext: string): Promise<ToolStrategy> {
  // 1. Check user context for explicit framework mentions
  if (userContext.includes('Flutter') || userContext.includes('widget')) {
    return 'flutter-specialized';
  }
  
  if (userContext.includes('Phoenix') || userContext.includes('LiveView')) {
    return 'phoenix-universal';
  }
  
  // 2. Check URL patterns
  if (url.includes('flutter') || url.includes('dart')) {
    return 'flutter-specialized';
  }
  
  // 3. Default to universal (works for most web apps)
  return 'standard-universal';
}
```

### **Progressive Detection Strategy**
```typescript
// Recommended AI workflow:

1. Start with Universal Workflows (safest default)
2. If interaction fails → Try framework detection
3. Switch to specialized tools if needed
4. Document framework type for future sessions
```

## 📋 **AI Usage Patterns by Framework**

### **Flutter Web Workflow Example**
```typescript
// ✅ CORRECT: Flutter-specific approach
await flutter_quantum_interact({
  sessionId: 'session-123',
  command: 'click Submit Report button'
});

await flutter_quantum_find({
  sessionId: 'session-123', 
  findType: 'semantic',
  query: 'button with text Submit'
});

// ❌ INCORRECT: Universal approach for Flutter
await smart_form_interaction({
  selector: '.submit-button'  // Won't work with Flutter Canvas
});
```

### **Phoenix LiveView Workflow Example**
```typescript
// ✅ CORRECT: Universal Workflows with LiveView selectors
await create_workflow_template({
  workflow: {
    name: 'phoenix-user-registration',
    steps: [
      {
        name: 'fill-registration-form',
        action: 'fill',
        data: { 
          selector: '[phx-change="validate_user"]',
          value: 'user@example.com'
        }
      },
      {
        name: 'submit-form',
        action: 'click',
        data: { selector: '[phx-click="register_user"]' }
      },
      {
        name: 'wait-for-liveview-update',
        action: 'waitForSelector',
        data: { selector: '[data-testid="registration-success"]' }
      }
    ]
  }
});
```

### **Standard Web Workflow Example**
```typescript
// ✅ CORRECT: Full Universal Workflows capability
await create_workflow_template({
  workflow: {
    name: 'react-ecommerce-checkout',
    steps: [
      {
        name: 'navigate-to-checkout',
        action: 'navigate',
        data: { url: '/checkout' }
      },
      {
        name: 'fill-shipping-form',
        action: 'custom',
        data: {
          script: `
            document.querySelector('[data-testid="shipping-name"]').value = 'John Doe';
            document.querySelector('[data-testid="shipping-address"]').value = '123 Main St';
          `
        }
      }
    ]
  }
});
```

## 🚨 **Common AI Model Mistakes to Avoid**

### **❌ MISTAKE 1: Using Universal Workflows for Flutter Canvas Apps**
```typescript
// This WILL FAIL for Flutter Canvas rendering:
await smart_form_interaction({
  selector: '.flutter-button'  // No DOM elements in Canvas
});

// ✅ CORRECT:
await flutter_quantum_interact({
  command: 'click button with text Submit'
});
```

### **❌ MISTAKE 2: Using Flutter Tools for Standard Web Apps**
```typescript
// This is overkill and may not work:
await flutter_quantum_interact({
  command: 'click React button'  // React isn't Flutter
});

// ✅ CORRECT:
await smart_form_interaction({
  selector: '[data-testid="react-submit-button"]'
});
```

### **❌ MISTAKE 3: Ignoring Phoenix LiveView Patterns**
```typescript
// This misses LiveView's real-time nature:
await navigate_to_url({ url: '/dashboard' });
await smart_form_interaction({ selector: '.update-button' });

// ✅ CORRECT: Account for LiveView updates
await navigate_to_url({ url: '/dashboard' });
await wait_for_navigation({ 
  transitionType: 'spa',  // LiveView uses SPA-like updates
  expectedUrl: '/dashboard'
});
await smart_form_interaction({ 
  selector: '[phx-click="update_dashboard"]'  // Use phx-* attributes
});
```

## 🎯 **AI Decision Tree**

```
User Request for Web App Testing
         ↓
Does user mention Flutter/widgets?
   ↓ YES → Use flutter_* tools
   ↓ NO
Does user mention Phoenix/LiveView?  
   ↓ YES → Use Universal Workflows + phx-* selectors
   ↓ NO
Does URL contain 'flutter'?
   ↓ YES → Use flutter_* tools  
   ↓ NO  
Standard web app → Use Universal Workflows
```

## 📊 **Tool Compatibility Matrix**

| Framework | Universal Workflows | Flutter Tools | Recommended Primary |
|-----------|-------------------|---------------|-------------------|
| Flutter Web (Canvas) | ❌ Limited | ✅ Excellent | Flutter Tools |
| Flutter Web (HTML) | ⚠️ Partial | ✅ Excellent | Flutter Tools |
| Phoenix LiveView | ✅ Excellent | ❌ N/A | Universal Workflows |
| React/Next.js | ✅ Excellent | ❌ N/A | Universal Workflows |
| Vue/Nuxt | ✅ Excellent | ❌ N/A | Universal Workflows |
| Angular | ✅ Excellent | ❌ N/A | Universal Workflows |
| Svelte/SvelteKit | ✅ Excellent | ❌ N/A | Universal Workflows |

## 🚀 **Quick Reference for AI Models**

### **When User Says...**
- **"Test my Flutter app"** → Use `flutter_quantum_interact`
- **"Test my Phoenix LiveView"** → Use `create_workflow_template` 
- **"Test my React app"** → Use `create_workflow_template`
- **"Click a widget"** → Flutter detected, use `flutter_quantum_interact`
- **"Submit this form"** → If Phoenix use Universal + phx selectors, if standard web use Universal, if Flutter use flutter tools

### **When You See URL Patterns...**
- **`/flutter`** → Flutter tools
- **`localhost:4000`** → Likely Phoenix, use Universal
- **`localhost:3000`** → Likely React/Next.js, use Universal
- **`.flutter.dev`** → Flutter tools

### **When DOM Contains...**
- **`<flutter-view>`** → Flutter tools
- **`phx-click=`** → Universal + Phoenix patterns
- **`data-reactroot`** → Universal workflows
- **`<canvas>` heavy** → Possibly Flutter, try Flutter tools

## 🎉 **Success Metrics**

**AI models should achieve:**
- ✅ 95%+ success rate with correct tool selection
- ✅ Automatic framework detection without user guidance
- ✅ Graceful fallback when primary strategy fails
- ✅ Clear error messages when framework mismatch occurs

**This guide eliminates AI guesswork and provides systematic decision-making for optimal tool usage across all web framework types.**