@salama/image-finder

# Template Matcher Wrapper - Comprehensive Usage Guide A powerful, production-ready template matching tool built on OpenCV.js with advanced features for computer vision applications, automated testing, and image recognition tasks. ## Table of Contents 1. [Overview](#overview) 2. [Quick Start](#quick-start) 3. [Installation](#installation) 4. [Basic Usage](#basic-usage) 5. [Advanced Features](#advanced-features) 6. [CLI Reference](#cli-reference) 7. [Programming API](#programming-api) 8. [Performance Optimization](#performance-optimization) 9. [Troubleshooting](#troubleshooting) 10. [Best Practices](#best-practices) ## Overview This tool provides enterprise-grade template matching capabilities with: - **🚀 High Performance**: Input scaling for up to 89% faster processing - **🎨 Color Sensitivity**: Advanced HSV color space matching with dual RGB+HSV scoring - **📱 Multi-Template**: Batch processing with cross-template non-maximum suppression - **🔄 Variant Detection**: Automatic detection of numbered template variants - **⚡ Parallel Safe**: Unique annotation filenames for concurrent execution - **🎯 Precision Control**: Configurable thresholds, preprocessing, and matching methods ## Quick Start ```bash # Basic template matching node cli.js screenshot.png template.png # With lower threshold for harder-to-find templates node cli.js screenshot.png button.png --threshold 0.6 # Auto-detect template variants (button1.png, button2.png, etc.) node cli.js screenshot.png button.png --detect-variants # Batch process multiple templates node cli.js screenshot.png template1.png template2.png template3.png # High-performance mode with input scaling node cli.js large_screenshot.png template.png --input-scaling 0.33 ``` ## Installation ### Prerequisites - Node.js 14 or higher - npm or yarn package manager ### Setup ```bash # Clone or download the project cd imatcher_wrapper2 # Install dependencies npm install # Verify installation node cli.js --help ``` ## Basic Usage ### Single Template Matching Find all instances of a template in a screenshot: ```bash node cli.js screenshot.png template.png ``` **Output:** ``` Processing screenshot: screenshot.png Against template: template.png === SUMMARY === Found 2 match(es) in 1247ms Matches: #1: [150, 200] 85.3% #2: [450, 300] 82.1% Annotated image saved to: ./tmp/annotations/single_screenshot_20250717_143022_a1b2c3.png ``` ### Multiple Templates (Batch Mode) Process multiple templates against one screenshot: ```bash node cli.js screenshot.png button1.png button2.png menu.png ``` This automatically enables: - **Parallel processing** of all templates - **Cross-template NMS** to remove overlapping matches - **Batch annotation** with color-coded results ### Template Variant Detection Automatically find numbered variants of a template: ```bash node cli.js screenshot.png button.png --detect-variants ``` This will automatically detect and process: - `button.png` (base template) - `button1.png`, `button2.png`, `button3.png`, etc. Stops after 3 consecutive missing files for efficiency. ## Advanced Features ### 1. Color-Sensitive Matching #### HSV Color Enhancement ```bash # Enable HSV color space for better color distinction node cli.js screenshot.png red_button.png --hsv # Custom HSV channel weights [Hue, Saturation, Value] node cli.js screenshot.png button.png --hsv --hsv-weights "[3.0,2.0,0.5]" ``` #### Dual RGB+HSV Scoring ```bash # Maximum color sensitivity with dual scoring node cli.js screenshot.png colored_element.png --dual-scoring # Custom weights for RGB and HSV components node cli.js screenshot.png button.png --dual-scoring --rgb-weight 1.0 --hsv-weight 0.8 ``` ### 2. Performance Optimization #### Input Scaling Dramatically improve performance by scaling down input images: ```bash # ~89% faster processing (33% of original size) node cli.js large_screenshot.png template.png --input-scaling 0.33 # ~75% faster processing (50% of original size) node cli.js screenshot.png template.png --input-scaling 0.5 ``` **Note:** Coordinates are automatically scaled back to original image size. #### Multi-Scale Matching ```bash # Search at multiple scales for size-invariant matching node cli.js screenshot.png template.png --multiscale # Custom scale factors node cli.js screenshot.png template.png --multiscale --scales "[0.8,1.0,1.2,1.5]" ``` ### 3. Preprocessing Options ```bash # Disable edge detection (enabled by default) node cli.js screenshot.png template.png --no-edges # Convert to grayscale for better structural matching node cli.js screenshot.png template.png --grayscale # Apply Gaussian blur to reduce noise node cli.js screenshot.png template.png --blur ``` ### 4. Annotation Control #### Custom Annotation Directory ```bash # Save annotations to custom directory node cli.js screenshot.png template.png --annotation-dir ./my_results # Disable annotations completely node cli.js screenshot.png template.png --no-annotations # Specific output file (overrides annotation directory) node cli.js screenshot.png template.png --output custom_result.png ``` Annotation filenames are automatically generated as: `{prefix}_{screenshot}_{YYYYMMDD_HHMMSS}_{randomhex}.png` Example: `single_screenshot_20250717_143022_a1b2c3.png` ### 5. Cross-Template NMS Control overlap removal between different templates: ```bash # Disable cross-template overlap removal node cli.js screenshot.png template1.png template2.png --no-cross-template-nms # Custom overlap threshold (0.0 = no overlap, 1.0 = complete overlap) node cli.js screenshot.png button1.png button2.png --overlap-threshold 0.5 ``` ## CLI Reference ### Core Options | Option | Description | Default | |--------|-------------|---------| | `--threshold <value>` | Matching confidence threshold (0.0-1.0) | 0.8 | | `--method <name>` | OpenCV matching method | Auto | | `--max-matches <num>` | Maximum matches to return | 10 | | `--debug` | Enable detailed logging | false | ### Preprocessing Options | Option | Description | Default | |--------|-------------|---------| | `--edges` / `--no-edges` | Edge detection preprocessing | enabled | | `--blur` / `--no-blur` | Gaussian blur preprocessing | disabled | | `--grayscale` / `--no-grayscale` | Convert to grayscale | disabled | | `--multiscale` | Multi-scale matching | disabled | | `--scales <json>` | Custom scale factors | [0.33,0.5,0.8...1.2] | ### Color Sensitivity Options | Option | Description | Default | |--------|-------------|---------| | `--hsv` / `--no-hsv` | HSV color enhancement | enabled | | `--hsv-weights <json>` | HSV channel weights [H,S,V] | [0.3,13,1.1] | | `--dual-scoring` / `--no-dual-scoring` | RGB+HSV dual scoring | enabled | | `--rgb-weight <value>` | RGB component weight | 0.4 | | `--hsv-weight <value>` | HSV component weight | 0.6 | ### Performance Options | Option | Description | Default | |--------|-------------|---------| | `--input-scaling <value>` | Scale input images (0.1-1.0) | 1.0 | | `--spatial-tolerance <px>` | Pixel tolerance for dual scoring | 10 | ### Batch Processing Options | Option | Description | Default | |--------|-------------|---------| | `--batch` | Force batch mode | auto | | `--cross-template-nms` / `--no-cross-template-nms` | Cross-template overlap removal | enabled | | `--overlap-threshold <value>` | IoU threshold for NMS | 0.7 | ### Variant Detection Options | Option | Description | Default | |--------|-------------|---------| | `--detect-variants` / `--no-detect-variants` | Auto-detect numbered variants | disabled | ### Output Options | Option | Description | Default | |--------|-------------|---------| | `--output <path>` | Specific output file path | auto-generated | | `--annotation-dir <dir>` | Annotation directory | ./tmp/annotations | | `--no-annotations` | Disable annotation generation | enabled | | `--json` | JSON output format | human-readable | ### Available Matching Methods - `CCOEFF` - Correlation coefficient - `CCOEFF_NORMED` - Normalized correlation coefficient - `CCORR` - Cross correlation - `CCORR_NORMED` - Normalized cross correlation - `SQDIFF` - Squared difference - `SQDIFF_NORMED` - Normalized squared difference ## Programming API ### MatcherWrapper Class ```javascript const MatcherWrapper = require('./MatcherWrapper'); // Single template matching const matches = await MatcherWrapper.findMatches( 'screenshot.png', 'template.png', { threshold: 0.8, useHSV: true, useDualScoring: true, annotationDir: './results' } ); // Batch template matching const batchMatches = await MatcherWrapper.findMatchesBatch( 'screenshot.png', ['template1.png', 'template2.png', 'template3.png'], { threshold: 0.7, crossTemplateNMS: true, inputScaling: 0.5 } ); // Variant detection const variantMatches = await MatcherWrapper.findMatches( 'screenshot.png', 'button.png', { enableVariantDetection: true, threshold: 0.8 } ); ``` ### Match Object Structure ```javascript { x: 150, // Top-left X coordinate y: 200, // Top-left Y coordinate width: 50, // Template width height: 30, // Template height confidence: 0.853, // Match confidence (0.0-1.0) centerX: 175, // Center X coordinate centerY: 215, // Center Y coordinate templatePath: 'template.png' // (batch mode only) } ``` ### Default Options ```javascript const defaultOptions = MatcherWrapper.getDefaultOptions(); console.log(defaultOptions); ``` ### Available Methods ```javascript const methods = MatcherWrapper.getAvailableMethods(); console.log(methods); // ['CCOEFF', 'CCOEFF_NORMED', 'CCORR', 'CCORR_NORMED', 'SQDIFF', 'SQDIFF_NORMED'] ``` ## Performance Optimization ### 1. Input Scaling Strategy | Scale Factor | Speed Improvement | Use Case | |--------------|-------------------|----------| | 0.33 | ~89% faster | Large screenshots, rough matching | | 0.5 | ~75% faster | High-resolution images | | 0.75 | ~44% faster | Moderate resolution, precision needed | | 1.0 | No improvement | Small images, maximum precision | ### 2. Threshold Optimization - **Start with 0.8** for most applications - **Lower to 0.6-0.7** for difficult templates - **Raise to 0.9+** for high-precision requirements ### 3. Color Sensitivity vs Performance | Configuration | Speed | Color Accuracy | Best For | |---------------|-------|----------------|----------| | Standard RGB | Fastest | Basic | Grayscale/simple images | | HSV Only | Medium | Good | Color-critical matching | | Dual Scoring | Slower | Excellent | Maximum accuracy needed | ### 4. Batch Processing Benefits - **Parallel execution** of multiple templates - **Shared OpenCV initialization** reduces overhead - **Cross-template NMS** prevents duplicate detections - **Automatic scaling** applied to all templates simultaneously ## Troubleshooting ### Common Issues #### 1. No Matches Found ```bash # Try lowering the threshold node cli.js screenshot.png template.png --threshold 0.5 # Disable edge detection node cli.js screenshot.png template.png --no-edges --threshold 0.6 # Try multiscale matching node cli.js screenshot.png template.png --multiscale --threshold 0.6 # For color-sensitive elements node cli.js screenshot.png template.png --dual-scoring --threshold 0.6 ``` #### 2. Too Many False Positives ```bash # Raise the threshold node cli.js screenshot.png template.png --threshold 0.9 # Enable edge detection node cli.js screenshot.png template.png --edges # Use specific matching method node cli.js screenshot.png template.png --method CCOEFF_NORMED ``` #### 3. Performance Issues ```bash # Use input scaling node cli.js screenshot.png template.png --input-scaling 0.5 # Disable dual scoring node cli.js screenshot.png template.png --no-dual-scoring # Convert to grayscale node cli.js screenshot.png template.png --grayscale ``` #### 4. Color Matching Problems ```bash # Enable HSV enhancement node cli.js screenshot.png template.png --hsv # Adjust HSV weights for your use case node cli.js screenshot.png template.png --hsv --hsv-weights "[2.0,3.0,0.5]" # Use dual scoring for maximum color sensitivity node cli.js screenshot.png template.png --dual-scoring ``` ### Debug Mode Enable detailed logging for troubleshooting: ```bash node cli.js screenshot.png template.png --debug ``` This provides: - Processing time breakdown - Confidence scores for all matches - Preprocessing details - Scaling information - Variant detection progress ## Best Practices ### 1. Template Preparation - **High quality templates**: Use clear, uncompressed images - **Appropriate size**: Templates should be 20-200 pixels in each dimension - **Consistent lighting**: Match screenshot lighting conditions when possible - **Multiple variants**: Create numbered variants for different states/styles ### 2. Screenshot Considerations - **Consistent resolution**: Higher resolution provides better accuracy - **Stable elements**: Avoid matching dynamic content - **Clean backgrounds**: Reduce noise around target elements ### 3. Threshold Selection - **Start conservative**: Begin with 0.8 and adjust based on results - **Test thoroughly**: Use representative screenshots during development - **Consider context**: UI testing can use higher thresholds than general image recognition ### 4. Performance Tuning - **Profile first**: Use `--debug` to identify bottlenecks - **Scale appropriately**: Use input scaling for large images - **Batch when possible**: Process multiple templates together - **Cache results**: Store match results when templates don't change ### 5. Production Deployment - **Parallel execution**: Tool is safe for concurrent use - **Annotation management**: Use custom annotation directories per process - **Error handling**: Implement proper error handling for missing files - **Monitoring**: Log processing times and match counts for performance monitoring ### 6. Template Organization ``` templates/ ├── buttons/ │ ├── submit.png │ ├── submit1.png # Variant for different states │ ├── submit2.png │ └── cancel.png ├── icons/ │ ├── settings.png │ └── help.png └── mobile/ ├── android/ └── ios/ ``` ### 7. Automation Integration ```bash #!/bin/bash # Example automation script SCREENSHOT="current_screen.png" RESULTS_DIR="./test_results/$(date +%Y%m%d_%H%M%S)" # Take screenshot (your screenshot tool here) take_screenshot "$SCREENSHOT" # Process with template matching node cli.js "$SCREENSHOT" templates/login_button.png \ --detect-variants \ --threshold 0.7 \ --annotation-dir "$RESULTS_DIR" \ --json > "$RESULTS_DIR/results.json" # Parse results for automation decisions MATCHES=$(cat "$RESULTS_DIR/results.json" | jq length) if [ "$MATCHES" -gt 0 ]; then echo "Login button found, proceeding with test..." else echo "Login button not found, test failed" exit 1 fi ``` ## Conclusion This template matching tool provides enterprise-grade capabilities for computer vision applications. Its combination of performance optimization, color sensitivity, and parallel processing support makes it suitable for everything from simple automation tasks to complex image recognition systems. For additional support or feature requests, refer to the codebase documentation and examples provided in `example.js`.