UNPKG

aiwg

Version:

Deployment tool and support utility for AI context. Copies agents, skills, commands, rules, and behaviors into the paths each AI platform reads (Claude Code, Codex, Copilot, Cursor, Warp, OpenClaw, and 6 more) so one source of truth works across 10 platfo

1,122 lines (805 loc) 49 kB
# Vision Document: AIWG Research Framework **Project**: AIWG Research Framework **Framework ID**: research-complete **Version**: 1.0.0 **Document ID**: VIS-RF-001 **Created**: 2026-01-25 **Status**: Draft for Approval **Document Type**: Vision Document ## References - @$AIWG_ROOT/agentic/code/frameworks/research-complete/inception/project-intake.md - Project foundation and scope - @$AIWG_ROOT/agentic/code/frameworks/research-complete/inception/solution-profile.md - Technical solution overview - @$AIWG_ROOT/agentic/code/frameworks/research-complete/inception/stakeholder-analysis.md - Stakeholder needs and engagement - @.aiwg/research/research-framework-findings.md - Comprehensive research foundation (15 topics, 50+ sources) - @$AIWG_ROOT/agentic/code/frameworks/sdlc-complete/README.md - SDLC framework patterns - @$AIWG_ROOT/docs/extensions/overview.md - AIWG extension system architecture --- ## 1. Executive Summary ### 1.1 The Opportunity The intersection of AI-assisted development and academic research presents a critical gap: developers and researchers lack systematic tools to manage the full research lifecycle from discovery through archival while maintaining professional standards. Current approaches are fragmented, manual, and fail to leverage AI capabilities for systematic literature review, quality assessment, and knowledge synthesis. ### 1.2 Our Response The AIWG Research Framework transforms research management from an ad-hoc, time-consuming burden into a structured, AI-powered, reproducible process. By integrating proven methodologies (PRISMA, GRADE, Zettelkasten, OAIS) with modern AI capabilities (LLM-powered summarization, automated gap detection, citation network analysis), we enable both developers and researchers to conduct professional-grade research at development velocity. ### 1.3 Strategic Positioning This framework completes AIWG's mission of improving AI-generated content quality by providing the evidence layer underlying all development and documentation decisions. Where sdlc-complete manages software lifecycles and media-marketing-kit handles content creation, research-complete ensures every claim, architecture decision, and strategic choice is backed by traceable, quality-assessed sources. ### 1.4 Success Vision Within 12 months of release, AIWG Research Framework will: - Enable 100+ developers to manage research at academic standards without academic overhead - Power the professionalization of 10+ informal projects into publication-ready documentation - Support at least 3 published research papers with full reproducibility - Demonstrate 60%+ time savings on research tasks while improving quality and rigor - Become the reference implementation for AI-assisted systematic review --- ## 2. Problem Statement ### 2.1 The Research Crisis Modern software development and research face converging crises: **Reproducibility Crisis**: 46% replication failure rate in cancer studies. Researchers lack tools to ensure their work is verifiable. **Professionalization Gap**: Projects mature from informal experiments to production systems, but documentation remains casual, lacking citations and professional terminology. **Manual Burden**: Researchers spend 60+ hours manually screening papers for systematic reviews, with 40% of that effort being mechanical work suitable for automation. **Fragmented Tools**: Research workflow spans disconnected tools (manual PDF downloads, Zotero for references, Notion for notes, custom scripts for analysis). No integrated solution exists. **Quality Uncertainty**: No systematic way to assess source quality, leading to citation of low-evidence sources and missed high-quality alternatives. **Lost Knowledge**: Research insights scattered across files, emails, and memory. Citation networks remain implicit, preventing discovery of connections. ### 2.2 Current State Pain Points From stakeholder analysis and project intake, users experience: **Discovery Stage**: - Manual database searches with no reproducibility guarantees - No automated gap detection to identify missing research areas - Citation chaining requires tedious manual work - Keyword bias misses semantically related papers **Acquisition Stage**: - Manual PDF downloads and renaming - Ad-hoc filing systems that don't scale - Inconsistent metadata quality - No FAIR compliance validation - Lost sources and broken links **Documentation Stage**: - Manual summarization takes 15-30 minutes per paper - No structured extraction of data - Notes scattered across tools without linking - Progressive summarization requires repeated manual passes - Knowledge synthesis depends on memory **Integration Stage**: - Manual citation formatting with error-prone transcription - Claims lack backing citations (citable-claims-index.md pattern exists but manual) - No systematic way to track what needs citations - Citation context (supported/contradicted) invisible - Knowledge graphs remain implicit mental models **Archival Stage**: - No version control for research artifacts - Provenance tracking is manual and incomplete - Reproducibility packages non-existent - Long-term preservation unplanned - Integrity verification absent ### 2.3 Impact on Stakeholders **Developers**: Spend hours finding and organizing research, delaying feature development. Architecture decisions lack research backing, reducing credibility. **Researchers**: Cannot meet PRISMA standards without manual heroics. FAIR compliance is aspiration, not reality. Reproducibility remains hope, not guarantee. **Documentation Writers**: Citation management consumes disproportionate time. Claims remain unbacked. Professional tone undermined by lack of authoritative sources. **Projects**: matric-memory professionalization blocked (Issues #154, #155). token-lifecycle validation incomplete. Research-papers repo underutilized. ### 2.4 The Core Problem **We lack a systematic, AI-powered framework that manages the complete research lifecycle while enforcing professional standards (PRISMA, FAIR, GRADE, OAIS) without requiring academic training or manual overhead.** --- ## 3. Vision Statement **Transform research from a manual, fragmented burden into a systematic, AI-powered capability that enables developers and researchers to discover, assess, synthesize, and preserve knowledge at academic rigor with development velocity.** --- ## 4. Mission Statement **The AIWG Research Framework provides specialized agents, structured workflows, and artifact management that automate research discovery, enforce quality standards, track provenance, and enable reproducibility, making professional-grade research accessible to every developer and researcher.** --- ## 5. Goals & Objectives ### 5.1 Primary Goals #### Goal 1: Automate Research Discovery **Objective**: Reduce manual search and screening time by 60%+ while increasing comprehensive coverage. **Key Results**: - Semantic Scholar API integration discovers 200M+ papers - Automated gap analysis identifies missing research areas with >80% accuracy - Citation network traversal surfaces relevant papers not found by keyword search - PRISMA-compliant search strategies ensure reproducibility **Timeline**: Phase 2 (Weeks 6-8) **Owner**: Discovery Agent team **Success Metric**: User-reported time savings >60%, expert validation of gap reports >80% --- #### Goal 2: Enforce Quality Standards **Objective**: Ensure 100% FAIR compliance and GRADE-inspired quality assessment for all research artifacts. **Key Results**: - FAIR compliance scoring identifies deficiencies automatically - GRADE-inspired multi-dimensional quality scores differentiate sources - Automated validation checks prevent low-quality sources - Quality dashboards guide source selection **Timeline**: Phase 3 (Weeks 9-11) **Owner**: Quality Agent team **Success Metric**: 100% of sources have quality scores, >80% score high/moderate --- #### Goal 3: Enable Reproducibility **Objective**: Achieve 100% W3C PROV-compliant provenance tracking and reproducible workflows. **Key Results**: - All research operations logged per W3C PROV standard - Workflow definitions enable exact replication - Checksums verify data integrity on every read - Reproducibility packages include containerized environments **Timeline**: Phase 5 (Weeks 15-17) **Owner**: Provenance Agent, Workflow Agent teams **Success Metric**: 100% operations logged, external researchers can replicate results --- #### Goal 4: Synthesize Knowledge **Objective**: Transform 100% of acquired papers into structured, linked knowledge artifacts. **Key Results**: - LLM-powered summarization achieves >90% factual accuracy - Zettelkasten-style notes enable knowledge synthesis - Citation networks reveal hidden connections - Knowledge graphs support semantic queries **Timeline**: Phase 3-4 (Weeks 9-14) **Owner**: Documentation Agent, Citation Agent teams **Success Metric**: 100% sources have summaries/notes, knowledge graph density ~8 links/note --- #### Goal 5: Integrate Seamlessly **Objective**: Zero-friction integration with existing AIWG workflows and external research tools. **Key Results**: - Research artifacts live in .aiwg/research/ alongside SDLC artifacts - Zotero, Obsidian, BibTeX export without data loss - Citation formatting follows any of 9,000+ styles - Claims index auto-links to SDLC documents **Timeline**: Phase 4 (Weeks 12-14) **Owner**: Citation Agent, Integration team **Success Metric**: <5% integration issues, 95%+ export accuracy --- ### 5.2 Secondary Objectives **Professionalize matric-memory** (Timeline: Phase 6, Week 18-20) - All claims backed with citations - Terminology mapped from informal to professional - Documentation meets publication standards **Enable AIWG Self-Improvement** (Timeline: Post-v1.0) - Framework development itself uses research-complete - Architecture informed by AI agents, LLM, SDLC research - Continuous evolution based on latest research **Build Community** (Timeline: Ongoing) - 10+ active contributors within 6 months - 3-5 published papers using framework - Academic partnerships for validation **Minimize Maintenance** (Timeline: All phases) - <5 hours/week maintainer effort - 90%+ code coverage, automated testing - Self-service documentation reduces support --- ## 6. Target Audience ### 6.1 Primary Personas #### Persona 1: Developer-Researcher **Profile**: Software developer working on AI/ML systems who needs research backing for architecture decisions and documentation. **Characteristics**: - Technical background, limited academic research training - Values velocity but needs professional credibility - Works on matric-* ecosystem or similar projects - Uses AIWG SDLC framework for development **Needs**: - Quick discovery of relevant papers (<5 minutes) - Automated citation integration with SDLC docs - Quality assessment to guide source selection - Gap analysis to identify missing research **Success Criteria**: - Completes first research task in <15 minutes - Reduces research time by 60%+ - All architecture decisions have research backing - Continues using framework after 30 days (75% retention) --- #### Persona 2: Academic Researcher **Profile**: Graduate student or faculty conducting systematic literature reviews for papers, dissertations, or grants. **Characteristics**: - Academic training in research methodology - Requires PRISMA, FAIR, GRADE compliance - Values reproducibility and transparency - Publishes in peer-reviewed venues **Needs**: - PRISMA-compliant workflow templates - FAIR metadata validation - W3C PROV provenance tracking - Citation network visualization - Reproducibility packages for publication **Success Criteria**: - 100% PRISMA compliance for systematic reviews - 90% FAIR compliance scores - Published at least 1 paper using framework - External researchers can replicate findings --- #### Persona 3: Documentation Specialist **Profile**: Technical writer professionalizing informal documentation for production systems. **Characteristics**: - Writing background, not necessarily technical - Needs to back claims with citations - Manages multi-audience documentation levels - Works on projects like matric-memory professionalization **Needs**: - Automated claims index (what needs citations) - Citation formatting and integration - Terminology mapping (informal to professional) - Quality scoring for source selection - Voice framework integration for tone consistency **Success Criteria**: - 100% claims backed with citations - 95% citation accuracy - 80% reduction in citation management time - Voice framework works seamlessly --- ### 6.2 Secondary Personas **Solo Practitioner**: Individual developer on personal projects, needs lightweight onboarding, progressive feature disclosure. **AI Coding Assistant**: Claude Code, GitHub Copilot requiring structured knowledge access for research-backed responses. **Framework Contributor**: Community developer extending agents, commands, or templates. **Agent Developer**: Specialist building research agents with API integrations and security compliance. --- ### 6.3 Audience Prioritization | Priority | Persona | Phase Focus | Key Validation | |----------|---------|-------------|----------------| | 1 | Developer-Researcher | Phases 1-3 | matric-memory professionalization | | 2 | Academic Researcher | Phases 3-5 | PRISMA/FAIR compliance validation | | 3 | Documentation Specialist | Phases 3-4 | Voice framework integration | | 4 | Solo Practitioner | Phases 2-6 | Quick-start guide effectiveness | --- ## 7. Key Features (MoSCoW Prioritization) ### 7.1 Must Have (MVP/v1.0) **Discovery Automation**: - Semantic Scholar API integration (200M+ papers) - PRISMA-inspired search strategy templates - Automated gap detection via citation analysis - Reproducible search protocols **Acquisition Management**: - PDF download and metadata extraction - FAIR compliance validation - REF-XXX persistent identifier assignment - Integrity verification (checksums) **Documentation Support**: - LLM-powered summarization (RAG-based, no hallucinations) - Structured data extraction - Literature notes (Zettelkasten pattern) - Maps of Content for topic organization **Quality Assessment**: - GRADE-inspired quality scoring - FAIR compliance reporting - Source quality dashboards - Automated validation checks **Citation Integration**: - Citation formatting (multiple styles via Zotero patterns) - Claims index automation - BibTeX/RIS export - Citation backing for SDLC documents **Provenance Tracking**: - W3C PROV-compatible operation logging - Git-based version control - Artifact lineage graphs - Integrity verification **Artifact Structure**: - .aiwg/research/ directory organization - Discovery, sources, knowledge, networks, analysis, provenance subdirectories - JSON schemas for metadata, quality scores, extraction - Markdown templates for notes, MoCs, claims --- ### 7.2 Should Have (Post-MVP) **Advanced Knowledge Synthesis**: - Knowledge graph construction (concepts, papers, authors) - Citation network visualization - Community detection in citation networks - Concept relationship discovery **Workflow Orchestration**: - Snakemake/Nextflow-style pipeline definitions - DAG visualization for workflows - Reproducibility packages (containerized) - Preregistration templates (open science) **Archival Compliance**: - OAIS-inspired information packages (SIP/AIP/DIP) - Preservation planning (format migration) - Lifecycle management automation - Export packages for repositories **Collaboration Features**: - research-papers repo synchronization - Shared corpus deduplication - Cross-project discovery - Team workflow templates **Enhanced Integrations**: - Zotero command-line tools - Obsidian bidirectional linking - Neo4j knowledge graph queries - CI/CD pipeline integration --- ### 7.3 Could Have (Future Enhancements) **Advanced Discovery**: - Research Rabbit integration - Connected Papers integration - Litmaps citation chains - Author network analysis **Quality Enhancements**: - Full GRADE framework implementation (beyond basics) - Scite citation context (supported/contradicted) - Altmetrics integration - Peer review simulation **Workflow Features**: - Real-time collaboration (async-only in v1) - Visual knowledge graph editors - Interactive dashboards - Mobile applications **Extended Capabilities**: - Meta-analysis statistical tools - Systematic review report generation - Grant writing assistance - Institutional repository integration --- ### 7.4 Won't Have (Out of Scope) - Machine learning-based recommendation engines (use external: Semantic Scholar) - Proprietary database access (beyond public APIs) - Plagiarism detection - Full-text paper generation - Automated peer review - Publishing workflow management - Statistical computation (meta-analysis calculations) --- ## 8. User Journeys ### 8.1 Journey 1: Developer Backs Architecture Decision **Persona**: Developer-Researcher **Context**: Designing authentication system, needs research on OAuth2 security patterns. **Steps**: 1. **Discovery**: Developer runs `aiwg research search "OAuth2 security best practices"` - Discovery Agent queries Semantic Scholar API - Returns ranked list of 20 relevant papers - Gap analysis suggests "token refresh security" is under-researched - Developer selects 5 high-quality papers for acquisition 2. **Acquisition**: Developer runs `aiwg research acquire REF-001 REF-002 REF-003 REF-004 REF-005` - Acquisition Agent downloads PDFs from open access sources - Extracts metadata (authors, DOI, abstract, year, venue) - Validates FAIR compliance (all sources have DOIs) - Assigns persistent identifiers (REF-025 through REF-029) 3. **Documentation**: Developer runs `aiwg research summarize REF-025-oauth2-security-patterns` - Documentation Agent generates summary using LLM with RAG (no hallucinations) - Extracts key findings: "Token rotation reduces CSRF risk by 80%" - Creates literature note with attribution - Adds to Map of Content: "Authentication Research" 4. **Integration**: Developer runs `aiwg research cite "Token rotation reduces CSRF risk" --source REF-025` - Citation Agent formats citation in Chicago style - Updates .aiwg/requirements/nfr-modules/security.md with citation - Adds to claims-index.md showing claim is now backed - Links to source metadata for provenance 5. **Outcome**: Architecture Decision Record (ADR-005-oauth2-token-rotation.md) cites 5 research papers, quality scores show evidence strength, decision is defensible and reproducible. **Time Savings**: 4 hours (manual) 30 minutes (framework) = 87.5% reduction **Quality Improvement**: Claims backed by high-quality sources (GRADE scores), FAIR-compliant metadata --- ### 8.2 Journey 2: Researcher Conducts Systematic Review **Persona**: Academic Researcher **Context**: Conducting systematic review of LLM evaluation methods for dissertation. **Steps**: 1. **Preregistration**: Researcher creates search strategy using PRISMA template - Documents inclusion/exclusion criteria - Defines search terms and databases - Sets quality thresholds - Registers protocol in .aiwg/research/discovery/preregistration/ 2. **Discovery**: Researcher executes search strategy - Discovery Agent searches Semantic Scholar with defined terms - Screens 500 papers using LLM-assisted relevance ranking (60% time reduction) - Automated gap analysis identifies "human-AI comparison" as missing topic - Final inclusion: 50 papers for full review 3. **Documentation**: Researcher processes included papers - Documentation Agent summarizes all 50 papers (5 minutes each vs. 20 minutes manual) - Extracts structured data: methods, datasets, metrics, findings - Creates literature notes following Zettelkasten method - Permanent notes synthesize cross-paper insights 4. **Analysis**: Researcher builds knowledge artifacts - Citation network reveals 3 distinct research communities - Knowledge graph shows "BLEU score" connected to 30+ evaluation approaches - Quality Agent scores sources: 40 high/moderate, 10 low (flag for sensitivity analysis) - FAIR compliance: 100% (all sources have DOIs and metadata) 5. **Reporting**: Researcher generates dissertation chapter - Citation Agent formats all references in APA style - PRISMA flow diagram auto-generated from screening logs - W3C PROV lineage shows full audit trail - Reproducibility package includes search strategy, screening decisions, analysis code 6. **Publication**: Dissertation passes committee review - External researcher validates findings using provenance logs - Reproducibility package enables replication study - Framework process documented in methodology section **Time Savings**: 120 hours (manual) 50 hours (framework) = 58% reduction **Quality Improvement**: 100% PRISMA compliance, full reproducibility, FAIR-compliant data --- ### 8.3 Journey 3: Documentation Writer Professionalizes Project **Persona**: Documentation Specialist **Context**: matric-memory project needs professionalization (Issues #154, #155). **Steps**: 1. **Assessment**: Writer analyzes current documentation - Claims-index agent scans all docs, identifies 200 unbacked claims - Terminology audit finds 50 informal terms needing professional equivalents - Quality assessment: current state is "blog-level," target is "publication-ready" 2. **Research Planning**: Writer creates acquisition list - For each unbacked claim, identifies research topic - Discovery Agent searches for authoritative sources - Prioritizes sources with high GRADE scores - Targets 100 sources to back 200 claims (some sources back multiple claims) 3. **Acquisition**: Writer acquires research corpus - Acquisition Agent downloads papers to research-papers repo (shared across projects) - Metadata extraction populates REF-XXX files - FAIR validation ensures all sources meet standards - Quality scores guide which sources to prioritize 4. **Integration**: Writer backs claims with citations - For each claim, Citation Agent formats appropriate citation - Claims-index.md updated to show coverage status - Terminology mapping agent suggests professional alternatives - Voice framework ensures tone remains consistent 5. **Validation**: Writer reviews quality - All 200 claims now backed with citations - Bibliography auto-generated with 100 sources - FAIR compliance: 100% - Professional terminology adopted across documentation 6. **Publication**: matric-memory documentation ready for academic venues - Research corpus shared in research-papers repo for reuse - Cross-project references enable knowledge sharing - Framework process documented for other projects **Time Savings**: 80 hours (manual) 25 hours (framework) = 69% reduction **Quality Improvement**: 100% citation coverage, FAIR-compliant, publication-ready --- ### 8.4 Journey 4: Solo Practitioner Builds Personal Knowledge Base **Persona**: Solo Practitioner **Context**: Exploring reinforcement learning for side project, needs to organize research. **Steps**: 1. **Quick Start**: Practitioner runs `aiwg research init` - Framework creates .aiwg/research/ structure - Provides quick-start guide and examples - No complex configuration required - Ready to use in <5 minutes 2. **Casual Discovery**: Practitioner searches as ideas emerge - Runs `aiwg research search "deep Q-learning"` - Acquires 3 foundational papers - Adds notes: "Good intro to RL basics" - No formal methodology needed for personal use 3. **Progressive Enhancement**: Over time, practitioner adds structure - Literature notes accumulate (20 papers after 2 months) - Maps of Content emerge organically - Citation network shows connections between papers - Knowledge graph reveals "policy gradient" relates to "Q-learning" 4. **Export to PKM**: Practitioner integrates with Obsidian - Runs `aiwg research export --obsidian` - All notes, MoCs, and citation links migrate to Obsidian vault - Bidirectional linking preserved - Continues using both tools interchangeably 5. **Scale to Formal**: Project matures, practitioner needs publication - Upgrades to PRISMA workflow for systematic review - Quality Agent retroactively scores existing sources - FAIR validation identifies 2 sources with missing DOIs, fixes them - Reproducibility package generated for publication supplement **Time Savings**: Incremental value from day 1, scales seamlessly to formal needs **Quality Improvement**: No upfront overhead, quality improves as needs evolve --- ### 8.5 Journey 5: AI Agent Provides Research-Backed Responses **Persona**: AI Coding Assistant (Claude Code) **Context**: Developer asks "What's the best approach for caching LLM responses?" **Steps**: 1. **Knowledge Access**: Agent queries research framework - Searches .aiwg/research/knowledge/ for "LLM caching" - Finds 3 relevant literature notes and 1 Map of Content - Quality scores show 2 high-quality sources, 1 moderate 2. **Synthesized Response**: Agent provides answer with citations - "Research suggests semantic caching reduces latency by 40% (REF-042, GRADE: High)" - "Alternative: prefix caching for repeated prompts (REF-043, GRADE: Moderate)" - "Gap analysis indicates vector database performance is under-researched" 3. **Provenance**: Developer verifies claims - Clicks REF-042 link, reads summary and metadata - FAIR compliance confirmed (DOI, license, persistent URL) - W3C PROV log shows how agent accessed and synthesized sources 4. **Citation Integration**: Developer documents decision - Agent runs `aiwg research cite "Semantic caching reduces latency" --source REF-042` - ADR-008-llm-caching-strategy.md updated with citation - Claims-index.md shows claim backed **Time Savings**: Instant research access vs. 30+ minutes of manual search **Quality Improvement**: Agent responses verifiable, traceable, quality-assessed --- ## 9. Success Metrics & KPIs ### 9.1 Efficiency Metrics | Metric | Baseline | Target | Measurement Method | |--------|----------|--------|-------------------| | **Discovery Time** | 60 min/search | <10 min/search | User surveys, time logging | | **Screening Time** | 100 hours/systematic review | <40 hours/review | Compare manual vs. LLM-assisted | | **Summarization Time** | 20 min/paper | <5 min/paper | Documentation Agent logs | | **Citation Formatting Time** | 5 min/citation | <30 sec/citation | Citation Agent logs | | **Overall Research Time Savings** | Baseline | 60%+ reduction | User-reported time tracking | ### 9.2 Quality Metrics | Metric | Baseline | Target | Measurement Method | |--------|----------|--------|-------------------| | **FAIR Compliance** | 20% | 100% | Automated F-UJI assessment | | **Citation Accuracy** | 85% (manual) | 99%+ (automated) | Validation against reference managers | | **GRADE Quality Scores** | N/A | >80% high/moderate | Score distribution across corpus | | **Provenance Completeness** | 30% | 100% | W3C PROV validation | | **Reproducibility Rate** | 46% (published baseline) | >90% | External replication studies | ### 9.3 Coverage Metrics | Metric | Baseline | Target | Measurement Method | |--------|----------|--------|-------------------| | **Sources with Metadata** | 60% | 100% | .aiwg/research/sources/ audit | | **Sources with Summaries** | 20% | 100% | Documentation Agent coverage | | **Claims Backed by Citations** | 10% | 100% | claims-index.md tracking | | **Gap Detection Recall** | 0% (manual) | >90% | Expert validation of gap reports | | **Knowledge Graph Density** | 0 | ~8 links/note | Zettelkasten best practice | ### 9.4 Adoption Metrics | Metric | Baseline | 6-Month Target | 12-Month Target | Measurement Method | |--------|----------|----------------|-----------------|-------------------| | **Active Users** | 0 | 50 | 200 | GitHub Insights, telemetry opt-in | | **Projects Using Framework** | 0 | 5 | 20 | Public repository analysis | | **Published Papers** | 0 | 1 | 3 | Community showcase, bibliography | | **Contributors** | 0 | 10 | 25 | GitHub contributor stats | | **Framework Retention** | N/A | 75% (30-day) | 60% (90-day) | User surveys | ### 9.5 User Satisfaction Metrics | Metric | Target | Measurement Method | |--------|--------|-------------------| | **Framework NPS Score** | >50 | Quarterly user surveys | | **Documentation Quality** | 4.5/5 | User ratings, feedback | | **Learning Curve** | <1 hour to proficiency | Time-to-first-success tracking | | **Feature Discoverability** | >80% find features without docs | Usability testing | | **Support Satisfaction** | 4.5/5 | Discord/GitHub issue surveys | ### 9.6 Technical Performance Metrics | Metric | Target | Measurement Method | |--------|--------|-------------------| | **API Response Time** | <2 sec (median) | Semantic Scholar API logs | | **PDF Extraction Success** | >95% | Acquisition Agent success rate | | **Metadata Accuracy** | >95% | Validation against manual entry | | **Test Coverage** | >90% | Automated coverage reports | | **Bug Escape Rate** | <5% | Post-release bug reports | ### 9.7 Strategic Success Metrics | Metric | 6-Month Target | 12-Month Target | Impact | |--------|----------------|-----------------|--------| | **matric-memory Professionalization** | 100% complete | Maintained | Proves professionalization use case | | **AIWG Self-Improvement** | Framework adopted | 50+ papers managed | Dogfooding validates framework | | **Academic Recognition** | 1 conference mention | 1 publication citing framework | Establishes credibility | | **Ecosystem Integration** | 3 matric-* projects | 10 AIWG projects | Validates cross-project value | --- ## 10. Positioning Statement **For** developers and researchers **who** need to manage research literature at professional standards without academic overhead, **the** AIWG Research Framework **is a** comprehensive lifecycle management system **that** automates discovery, enforces quality standards (PRISMA, FAIR, GRADE, OAIS), and enables reproducibility through AI-powered agents and structured workflows. **Unlike** Zotero (reference management only), Obsidian (note-taking only), or Elicit (summarization only), **the** AIWG Research Framework integrates the complete lifecycle from discovery through archival, embeds directly into AIWG development workflows, and enforces professional standards automatically rather than requiring manual compliance. **Our framework** is the only solution that combines academic rigor (PRISMA systematic reviews, W3C PROV provenance, OAIS preservation) with development velocity (60%+ time savings, automated gap detection, LLM-powered summarization) while maintaining 100% FAIR compliance and reproducibility. --- ## 11. Assumptions & Dependencies ### 11.1 Critical Assumptions **User Assumptions**: 1. Users have basic command-line proficiency (can run `aiwg` commands) 2. Users work primarily in English-language research (internationalization is future work) 3. Users value reproducibility and quality enough to adopt structured workflows 4. Users have access to papers via institutional access, open access, or personal decisions **Technical Assumptions**: 1. Node.js >=18.20.8 available on target systems 2. Semantic Scholar API remains free and available with reasonable rate limits 3. LLM APIs (Claude, OpenAI) remain accessible for AI-assisted operations 4. Markdown and JSON formats remain viable for long-term preservation 5. Git provides sufficient version control for research artifacts **Adoption Assumptions**: 1. matric-* ecosystem projects will serve as early validation cases 2. AIWG community will contribute to framework development 3. Open science movement continues to grow, increasing demand for reproducibility tools 4. Researchers value automation if quality standards are maintained **Validation Plan**: - Test assumption #1 (CLI proficiency) through quick-start user testing (Elaboration phase) - Monitor assumption #2 (Semantic Scholar API) through uptime tracking and fallback planning - Validate assumption #3 (user values) through adoption metrics and retention rates - Review all assumptions quarterly, update mitigation strategies --- ### 11.2 External Dependencies **API Dependencies**: - Semantic Scholar API (discovery, citation data) - **Critical** - DOI resolution services (CrossRef) - **High** - Scite API (citation context) - **Optional** **Mitigation**: - Implement caching to reduce API calls - Rate limiting to respect quotas - Fallback to manual search if APIs unavailable - Version-pin API client libraries **Tool Dependencies**: - Zotero (reference management integration) - **Medium** - Obsidian (PKM integration) - **Optional** - Neo4j (knowledge graph queries) - **Optional** **Mitigation**: - Framework works without optional integrations - Export formats (BibTeX, RIS, Markdown) enable portability - Document alternative tools (e.g., Mendeley instead of Zotero) **Standards Dependencies**: - PRISMA Statement (systematic review methodology) - FAIR Principles (data management) - W3C PROV (provenance ontology) - OAIS Reference Model (digital preservation) - GRADE Framework (evidence quality) **Mitigation**: - Standards are stable, published specifications - Implement core requirements, not full specifications - Document compliance levels explicitly - Monitor standards evolution, plan migrations --- ### 11.3 Internal Dependencies **AIWG Framework Dependencies**: - Core AIWG CLI (agent deployment, command registry) - Extension system (agent registry, framework deployment) - Artifact management (.aiwg/ directory conventions) - Voice framework (documentation quality for generated docs) **Mitigation**: - Research framework developed in lockstep with core framework - Version compatibility matrix published - Automated tests for cross-framework integration **Process Dependencies**: - Research findings document complete (DONE: @.aiwg/research/research-framework-findings.md) - matric-memory professionalization needs identified (DONE: Issues #154, #155) - research-papers repo established (DONE: /tmp/research-papers) **Mitigation**: - All prerequisite artifacts complete before development - Clear handoff points between phases - Dependency tracking in flow-status.yaml --- ### 11.4 Resource Dependencies **Personnel**: - Solo developer (Joseph Magly) for v1.0 development - **Critical** - 2-5 community contributors for post-v1 maintenance - **High** - 2-5 early adopter researchers for validation - **High** **Mitigation**: - Phased development plan accommodates solo velocity - Contributor onboarding materials prepared early - Early adopter recruitment starts in Elaboration phase **Infrastructure**: - GitHub repository (version control, CI/CD) - **Critical** - GitHub Actions (automated testing) - **High** - Local development environment - **Critical** **Mitigation**: - All infrastructure already in place - Backup plans for GitHub outages (Gitea fallback) - Local-first architecture (works offline) **Budget**: - $0 for v1.0 (open source, volunteer-driven) - **Constraint** - Optional: $20-50/month for commercial APIs - **Nice-to-have** **Mitigation**: - Free tier of all external services sufficient for v1.0 - Commercial API access deferred to post-v1 if needed - Community donations can fund future enhancements --- ### 11.5 Dependency Risk Matrix | Dependency | Type | Impact if Unavailable | Likelihood | Mitigation Priority | |------------|------|----------------------|------------|-------------------| | Semantic Scholar API | External | High | Low | High (implement caching, fallbacks) | | Node.js | Technical | High | Very Low | Low (standard environment) | | LLM APIs | External | Medium | Low | Medium (graceful degradation to manual) | | Zotero | External | Low | Low | Low (alternative export formats) | | Solo Developer | Resource | High | Medium | High (contributor onboarding, documentation) | | Early Adopters | Resource | Medium | Medium | Medium (community outreach, incentives) | --- ## 12. Glossary of Terms ### 12.1 Research Methodology Terms **PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses)**: Evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. 27-item checklist ensuring transparency and reproducibility. **GRADE (Grading of Recommendations, Assessment, Development and Evaluations)**: Framework for rating quality of evidence and strength of recommendations. Evaluates sources on multiple dimensions: risk of bias, inconsistency, indirectness, imprecision, publication bias. **FAIR Principles**: Data management standards ensuring data is Findable, Accessible, Interoperable, and Reusable. Core to open science and reproducibility. **W3C PROV**: World Wide Web Consortium standard for provenance tracking. Enables recording of entities, activities, and agents involved in producing data, with full audit trail. **OAIS (Open Archival Information System)**: Reference model for digital preservation (ISO 14721). Defines Submission Information Packages (SIP), Archival Information Packages (AIP), and Dissemination Information Packages (DIP) for long-term preservation. **Zettelkasten**: Note-taking method emphasizing atomic notes, bidirectional linking, and knowledge emergence. Originated by sociologist Niklas Luhmann (90,000 notes, 150 publications). **Systematic Review**: Rigorous, reproducible literature review following defined protocol (PRISMA). Includes explicit search strategy, eligibility criteria, quality assessment, and data synthesis. **Gap Analysis**: Identification of missing research areas or unanswered questions in existing literature. Guides future research priorities. **Citation Network**: Graph of papers connected by citations (who cites whom). Enables discovery of related work and research communities. **Knowledge Graph**: Semantic network of concepts, papers, authors, and relationships. Enables sophisticated queries and knowledge discovery. --- ### 12.2 Framework-Specific Terms **REF-XXX**: Persistent identifier for research sources within framework. Format: REF-001-short-title-slug. Enables stable cross-references across artifacts. **Literature Notes**: Processed insights from sources, maintaining attribution. Zettelkasten concept applied to research papers. **Permanent Notes**: Refined ideas building on literature notes, expressing original synthesis. Second-order knowledge artifacts. **Map of Content (MoC)**: Curated collection of related notes on a topic. Provides navigation and context within knowledge base. **Claims Index**: Inventory of assertions in documentation showing which have citation backing and which need research. Artifact: .aiwg/research/knowledge/claims-index.md. **Quality Score**: Multi-dimensional assessment of source quality following GRADE-inspired criteria. Guides source selection and evidence strength. **FAIR Compliance Score**: Automated assessment of metadata quality against FAIR principles. Uses F-UJI patterns for evaluation. **Provenance Log**: Complete audit trail of research operations following W3C PROV. Enables reproducibility and verification. **Reproducibility Package**: Containerized workflow including search strategies, screening decisions, analysis code, and data. Enables independent replication. **Gap Report**: Automated analysis identifying missing research areas or under-researched topics. Generated from citation network analysis and topic modeling. --- ### 12.3 Agent Terms **Discovery Agent**: Specialized agent for research search, screening, gap detection, and preregistration support. Interfaces with Semantic Scholar API. **Acquisition Agent**: Agent for PDF download, metadata extraction, FAIR validation, and source organization. Maintains .aiwg/research/sources/. **Documentation Agent**: Agent for LLM-powered summarization, data extraction, and literature/permanent note generation. Uses RAG patterns to prevent hallucination. **Citation Agent**: Agent for citation formatting, claim backing, network building, and bibliography generation. Supports 9,000+ citation styles via Zotero patterns. **Quality Agent**: Agent for GRADE scoring, FAIR assessment, and reproducibility validation. Maintains quality dashboards. **Provenance Agent**: Agent for W3C PROV logging, lineage tracking, and integrity verification. Ensures full audit trail. **Archival Agent**: Agent for OAIS-compliant preservation, version control, and lifecycle management. Maintains long-term accessibility. **Workflow Agent**: Agent for pipeline orchestration, DAG visualization, and reproducibility package generation. Ensures repeatable processes. --- ### 12.4 Technical Terms **RAG (Retrieval-Augmented Generation)**: LLM pattern that grounds generation in retrieved sources, reducing hallucination. Critical for research summarization. **LitLLM**: Pattern for literature review generation using RAG-based approaches. Ensures summaries reference actual sources. **F-UJI**: FAIR Data Assessment Tool. Automated scoring system for FAIR compliance. **BibTeX/RIS**: Standard citation export formats. BibTeX (LaTeX ecosystem), RIS (Reference Manager format). **DOI (Digital Object Identifier)**: Persistent identifier for scholarly works. Ensures long-term accessibility despite URL changes. **Semantic Scholar**: Free academic search engine and API by Allen Institute for AI. Indexes 200M+ papers with citation data. **Zotero**: Free, open-source reference manager. Supports 9,000+ citation styles, metadata extraction, PDF management. **Obsidian**: Markdown-based personal knowledge management tool. Popular for Zettelkasten implementations. **Neo4j**: Graph database for storing and querying knowledge graphs. Optional integration for advanced network analysis. --- ### 12.5 Workflow Terms **Discovery Stage**: Research lifecycle phase focused on finding relevant papers through search, screening, and gap analysis. **Acquisition Stage**: Phase focused on obtaining papers, extracting metadata, and validating FAIR compliance. **Documentation Stage**: Phase focused on summarization, extraction, note-taking, and knowledge organization. **Integration Stage**: Phase focused on citation formatting, claim backing, network building, and application. **Archival Stage**: Phase focused on preservation, version control, integrity verification, and long-term accessibility. **Preregistration**: Open science practice of publicly documenting research protocol before execution. Prevents p-hacking and increases transparency. **Screening**: Process of evaluating papers for inclusion/exclusion based on eligibility criteria. Can be automated with LLM assistance (60%+ time savings). **Progressive Summarization**: Iterative distillation of content, highlighting key insights in multiple passes. Building a Second Brain concept. **Bidirectional Linking**: Notes reference each other in both directions, enabling knowledge graph navigation. Core to Zettelkasten method. **Metadata Extraction**: Automated extraction of bibliographic information (authors, DOI, year, venue, abstract) from PDFs or APIs. --- ## 13. Approval & Sign-Off ### 13.1 Vision Approval **Project Sponsor**: Joseph Magly (AIWG Maintainer) - **Status**: PENDING REVIEW - **Approval Date**: TBD - **Comments**: _Vision aligns with comprehensive research foundation (15 topics, 50+ sources). Addresses identified pain points (matric-memory #154, #155). Success metrics are measurable and achievable. Timeline accommodates solo development velocity._ **Technical Lead**: Joseph Magly - **Status**: PENDING REVIEW - **Approval Date**: TBD - **Comments**: _Architecture informed by established standards (PRISMA, FAIR, GRADE, OAIS). Agent specializations follow AIWG patterns. Technology stack leverages existing tools (Semantic Scholar, Zotero, LLMs). Technical feasibility validated._ **Product Owner**: Joseph Magly (representing matric-memory, matric-eval, AIWG use cases) - **Status**: PENDING REVIEW - **Approval Date**: TBD - **Comments**: _User journeys validate stakeholder needs. MoSCoW prioritization ensures MVP delivers value. Professionalization use case (matric-memory) addresses immediate need. Positioning differentiates from alternatives._ --- ### 13.2 Stakeholder Validation **Primary Stakeholders** (AIWG Users, AI Researchers, Docs Writers): - **Status**: VALIDATION PENDING - **Method**: User story validation sessions (Elaboration phase) - **Timeline**: Weeks 3-4 - **Responsible**: Product Owner **Secondary Stakeholders** (AIWG Maintainers, Contributors, Agent Developers): - **Status**: VALIDATION PENDING - **Method**: Architecture review meetings (Elaboration phase) - **Timeline**: Weeks 3-5 - **Responsible**: Technical Lead **Tertiary Stakeholders** (matric-memory, token-lifecycle, research-papers repo): - **Status**: VALIDATION PENDING - **Method**: Use case validation interviews (Elaboration phase) - **Timeline**: Weeks 4-6 - **Responsible**: Integration Lead --- ### 13.3 Vision Governance **Document Owner**: Product Owner (Vision Owner role) **Review Frequency**: Quarterly (or when major assumptions change) **Update Process**: 1. Identify vision drift or invalidated assumptions 2. Consult stakeholders for input 3. Update vision document with changes 4. Communicate changes to all stakeholders 5. Update dependent documents (requirements, architecture) **Version History**: | Version | Date | Changes | Approver | |---------|------|---------|----------| | 1.0 (Draft) | 2026-01-25 | Initial vision document | Pending | --- ### 13.4 Next Steps (Post-Approval) **Immediate** (Week 1): - [ ] **Stakeholder Review**: Present vision to AIWG maintainers, gather feedback - [ ] **Vision Approval**: Obtain formal sign-off from sponsor, technical lead, product owner - [ ] **Communication**: Announce vision to community (Discord, GitHub issue) **Near-Term** (Weeks 2-4): - [ ] **Use Case Specification**: Create UC-001 through UC-008 for core capabilities - [ ] **User Story Creation**: Develop user stories for each persona and lifecycle stage - [ ] **NFR Definition**: Define non-functional requirements (performance, security, usability) - [ ] **Agent Specifications**: Design 8 core agents with tools, expertise, responsibilities **Medium-Term** (Weeks 5-8): - [ ] **Architecture Design**: Create Software Architecture Document (SAD), ADRs, data models - [ ] **Test Strategy**: Define unit, integration, UAT testing approaches - [ ] **Prototype Development**: Build MVP (discovery + acquisition) for validation **Long-Term** (Weeks 9-20): - [ ] **Phased Development**: Execute Phases 2-6 per project intake timeline - [ ] **Continuous Validation**: User testing, stakeholder feedback, iteration - [ ] **Documentation**: User guides, agent docs, API reference - [ ] **Launch Preparation**: Release notes, announcement, community onboarding --- ## References **Project Artifacts**: - @$AIWG_ROOT/agentic/code/frameworks/research-complete/inception/project-intake.md - Comprehensive project foundation - @$AIWG_ROOT/agentic/code/frameworks/research-complete/inception/solution-profile.md - Technical solution overview - @$AIWG_ROOT/agentic/code/frameworks/research-complete/inception/stakeholder-analysis.md - Stakeholder needs and engagement - @$AIWG_ROOT/agentic/code/frameworks/research-complete/flow-status.yaml - Current flow status and phase gates **Research Foundation**: - @.aiwg/research/research-framework-findings.md - 15 topic areas, 50+ sources **AIWG Framework**: - @CLAUDE.md - Project-specific guidance - @$AIWG_ROOT/docs/extensions/overview.md - Extension system architecture - @$AIWG_ROOT/agentic/code/frameworks/sdlc-complete/README.md - SDLC framework patterns - @$AIWG_ROOT/agentic/code/frameworks/media-marketing-kit/README.md - Media curator patterns (differentiation) **External Standards**: - [PRISMA Statement](https://www.prisma-statement.org/) - Systematic review reporting - [FAIR Principles](https://www.go-fair.org/fair-principles/) - Data management - [W3C PROV](https://www.w3.org/TR/prov-overview/) - Provenance ontology - [OAIS Reference Model](https://www.oclc.org/research/publications/2000/lavoie-oais.html) - Digital preservation - [GRADE Framework](https://www.gradeworkinggroup.org/) - Evidence quality assessment **External Tools**: - [Semantic Scholar API](https://www.semanticscholar.org/product/api) - Paper discovery - [Zotero](https://www.zotero.org/) - Reference management - [Obsidian](https://obsidian.md/) - Personal knowledge management --- **Document Status**: Draft - Awaiting Stakeholder Review and Approval **Next Artifact**: @$AIWG_ROOT/agentic/code/frameworks/research-complete/requirements/use-cases/UC-001-discovery-workflow.md **Owner**: Vision Owner (Product Owner) **Last Updated**: 2026-01-25