cleanifix

Cleanifix A CLI tool that automatically cleans your data files through natural language commands. Like having a data analyst in your terminal. 🚀 Quick Start bash# Install npm install -g cleanifix # Basic usage cleanifix @sales.csv "remove duplicates" cleanifix @users.csv "fill missing emails with 'unknown@example.com'" cleanifix @data.json "standardize all dates to ISO format" # Interactive mode cleanifix interactive @messy_data.csv 🎯 Features Core Capabilities (MVP) Missing Value Detection & Handling - Find and fix missing data automatically Data Standardization - Normalize dates, phone numbers, addresses, and more Deduplication - Remove duplicate rows with smart matching Natural Language Interface bash# Just describe what you want cleanifix @customers.csv "find missing phone numbers and fill with 'N/A'" cleanifix @inventory.csv "standardize product names to title case" cleanifix @transactions.csv "remove duplicate entries keeping the most recent" Smart Suggestions bash$ cleanifix @data.csv "analyze" 📊 Data Quality Report: ✗ 156 missing values in 'email' column ✗ 89 inconsistent date formats ✗ 34 potential duplicates Suggested fixes: 1. Fill missing emails with domain-based patterns 2. Standardize dates to YYYY-MM-DD 3. Remove exact duplicates keeping first occurrence Apply all fixes? [Y/n] 📦 Installation Prerequisites Node.js 18+ Python 3.8+ 4GB RAM recommended for large files Install from npm bashnpm install -g cleanifix Install from source bashgit clone https://github.com/rickyjs1955/cleanifix.git cd cleanifix ./scripts/setup-dev.sh 🛠️ Usage Examples Basic Cleaning bash# Find issues cleanifix @data.csv "show me data quality issues" # Fix missing values cleanifix @sales.csv "fill missing prices with median" # Standardize formats cleanifix @contacts.csv "standardize all phone numbers to international format" # Remove duplicates cleanifix @emails.csv "remove duplicate emails keeping the latest entry" Batch Processing bash# Create a config file cat > cleaning-rules.yaml << EOF rules: - type: missing_values columns: [price, quantity] strategy: median - type: standardize column: phone format: E164 - type: deduplicate keys: [email] keep: last EOF # Run batch cleaning cleanifix batch @data.csv --rules cleaning-rules.yaml Interactive Mode bashcleanifix interactive @messy_data.csv 🧹 Cleanifix Interactive Mode > analyze my data > fill missing ages with average by city > standardize all names to proper case > save as cleaned_data.csv > exit 🏗️ Architecture Cleanifix uses a hybrid architecture: CLI Interface (Node.js) - Fast, responsive user interaction Processing Engine (Python) - Powerful data manipulation with pandas Communication - JSON-based message passing between components 🤝 Contributing We welcome contributions! See CONTRIBUTING.md for guidelines. Development Setup bash# Clone the repo git clone https://github.com/rickyjs1955/cleanifix.git cd cleanifix # Setup development environment ./scripts/setup-dev.sh # Run tests npm test # CLI tests python -m pytest # Engine tests # Run in development mode npm run dev 📋 Roadmap Phase 1 (Current) - MVP Basic CLI interface Missing value handling Simple standardization Exact deduplication CSV support JSON support Phase 2 - Enhanced Rules Fuzzy deduplication Custom regex patterns Outlier detection Data type inference Excel support Phase 3 - ML Integration Smart imputation Anomaly detection Pattern learning Confidence scoring Auto-cleaning mode 📄 License MIT License - see LICENSE file for details 🙏 Acknowledgments Built with: Commander.js - CLI framework Pandas - Data manipulation Chalk - Terminal styling 💬 Support Documentation: docs.cleanifix.dev Issues: GitHub Issues Discussions: GitHub Discussions Made with ❤️ by data people, for data people