aiwg
Version:
Deployment tool and support utility for AI context. Copies agents, skills, commands, rules, and behaviors into the paths each AI platform reads (Claude Code, Codex, Copilot, Cursor, Warp, OpenClaw, and 6 more) so one source of truth works across 10 platfo
698 lines (578 loc) • 28.5 kB
Markdown
# Use-Case Specification: UC-RF-006
## Metadata
- ID: UC-RF-006
- Name: Assess Source Quality
- Owner: Requirements Analyst
- Contributors: Quality Agent, Research Framework Team
- Team: Research Framework
- Status: draft
- Created: 2026-01-25
- Updated: 2026-01-25
- Priority: P1 (High)
- Estimated Effort: M (Medium)
- Related Documents:
- Flow: Research Framework 5-Stage Lifecycle
- UC-RF-003: Document Research Sources
- NFR: Quality Assessment Standards
## 1. Use-Case Identifier and Name
**ID:** UC-RF-006
**Name:** Assess Source Quality
## 2. Scope and Level
**Scope:** Research Framework - Quality Assessment System
**Level:** User Goal
**System Boundary:** Quality Agent, source metadata, quality scoring algorithms, GRADE/FAIR frameworks
## 3. Primary Actor(s)
**Primary Actors:**
- Quality Agent: Specialized agent that evaluates research source quality
- User: Researcher validating source credibility
- Documentation Agent: Agent that documented the source (provides metadata)
**Actor Goals:**
- Quality Agent: Systematically evaluate source quality using standardized criteria
- User: Understand source reliability and appropriateness for research
- Documentation Agent: Receive quality feedback to improve documentation standards
## 4. Stakeholders and Interests
| Stakeholder | Interest |
|------------|----------|
| User | Reliable quality assessment to inform research decisions |
| Quality Agent | Consistent application of quality standards across all sources |
| Research Community | Adherence to established quality frameworks (GRADE, FAIR) |
| Framework Maintainer | Automated quality gates for research workflows |
## 5. Preconditions
1. Source has been documented (UC-RF-003 complete)
2. Source metadata file exists in `.aiwg/research/sources/`
3. Quality Agent has access to quality assessment frameworks
4. Source metadata includes minimum required fields (citation, source type, access date)
## 6. Postconditions
**Success:**
- Quality score calculated (0-100 scale)
- GRADE assessment completed (High/Moderate/Low/Very Low)
- FAIR principles validated (Findable, Accessible, Interoperable, Reusable)
- Quality report generated in `.aiwg/research/quality/`
- Source metadata updated with quality scores
- Recommendations generated for quality improvement (if applicable)
**Failure:**
- Insufficient metadata for quality assessment
- Error report generated with missing data elements
- User prompted to complete source documentation
## 7. Trigger
Quality Agent invoked after source documentation completes OR user manually requests quality assessment
## 8. Main Success Scenario
1. User or Documentation Agent triggers quality assessment: `/assess-source-quality <source-id>`
2. Quality Agent retrieves source metadata: `.aiwg/research/sources/<source-id>.md`
3. Quality Agent validates metadata completeness:
- Citation format valid
- Source type identified (academic paper, blog post, documentation, etc.)
- Publication date present
- Author/organization identified
- Access date recorded
4. Quality Agent initiates multi-dimensional assessment:
- **Authority Evaluation** (Weight: 30%)
- Author credentials verified (academic affiliation, expertise, publication history)
- Publisher reputation assessed (peer-reviewed journal, conference proceedings, reputable organization)
- Citation count analyzed (if available)
- Authority score: 85/100
- **Currency Evaluation** (Weight: 20%)
- Publication date: 2024 (recent, high score)
- Access date: 2026-01-25 (current)
- Topic currency: Active research area (high relevance)
- Currency score: 90/100
- **Accuracy Evaluation** (Weight: 25%)
- Peer review status: Yes (peer-reviewed)
- Methodology documented: Yes (clear research methods)
- Data sources cited: Yes (verifiable data)
- Conflicts of interest disclosed: Yes
- Accuracy score: 95/100
- **Coverage Evaluation** (Weight: 15%)
- Topic depth: Comprehensive (detailed treatment)
- Breadth of analysis: Multi-faceted (multiple perspectives)
- Completeness: Thorough (addresses key aspects)
- Coverage score: 80/100
- **Objectivity Evaluation** (Weight: 10%)
- Bias assessment: Minimal bias detected
- Balanced perspective: Yes (multiple viewpoints)
- Conflicts of interest: Disclosed and minimal
- Objectivity score: 85/100
5. Quality Agent calculates weighted overall score:
- Overall Quality Score: (85×0.30) + (90×0.20) + (95×0.25) + (80×0.15) + (85×0.10) = 87/100
6. Quality Agent performs GRADE assessment:
- Study design: Randomized controlled trial (starts at High)
- Risk of bias: Low (maintain High)
- Inconsistency: None (maintain High)
- Indirectness: None (maintain High)
- Imprecision: None (maintain High)
- Publication bias: No evidence (maintain High)
- **GRADE Rating:** High (strong confidence in evidence)
7. Quality Agent validates FAIR principles:
- **Findable:** Yes (persistent identifier: DOI present, metadata complete)
- **Accessible:** Yes (open access, retrieval protocol documented)
- **Interoperable:** Yes (standard formats, controlled vocabularies)
- **Reusable:** Yes (clear license, provenance documented)
- **FAIR Compliance:** 4/4 principles met
8. Quality Agent generates quality report:
- Report file: `.aiwg/research/quality/<source-id>-quality-report.md`
- Report sections:
- Executive summary (overall score, GRADE rating, FAIR compliance)
- Dimension scores (authority, currency, accuracy, coverage, objectivity)
- Strengths identified (peer-reviewed, recent, comprehensive)
- Limitations identified (none significant)
- Recommendations (source approved for integration)
9. Quality Agent updates source metadata:
- Adds `quality_score: 87` to frontmatter
- Adds `grade_rating: High` to frontmatter
- Adds `fair_compliant: true` to frontmatter
- Adds `quality_assessed_date: 2026-01-25` to frontmatter
10. Quality Agent reports assessment summary to user:
- "Quality Assessment Complete: <source-id>"
- "Overall Score: 87/100 (High Quality)"
- "GRADE Rating: High (strong confidence)"
- "FAIR Compliance: 4/4 principles met"
- "Recommendation: Approved for integration"
11. User reviews quality report and approves source for integration
12. Source flagged as quality-approved in research workflow
## 9. Alternate Flows
### Alt-1: Low-Quality Source Detection
**Branch Point:** Step 5 (Quality Agent calculates overall score)
**Condition:** Overall quality score <50/100 (low quality threshold)
**Flow:**
1. Quality Agent calculates overall score: 45/100 (below threshold)
2. Quality Agent identifies critical deficiencies:
- Authority score: 30/100 (unknown author, no credentials)
- Accuracy score: 40/100 (no peer review, methodology unclear)
- Currency score: 60/100 (5-year-old source in fast-moving field)
3. Quality Agent generates warning:
- "Warning: Low-quality source detected (45/100)"
- "Critical deficiencies: Authority, Accuracy"
- "Recommendation: Seek higher-quality alternative sources"
4. Quality Agent prompts user:
- "Continue with low-quality source? (y/n)"
- "Alternative: Search for higher-quality sources on same topic"
5. User chooses: "Search for alternatives"
6. Quality Agent invokes Discovery Agent to find better sources
7. **Resume Main Flow:** Step 11 (User reviews recommendations)
### Alt-2: FAIR Principles Violation
**Branch Point:** Step 7 (Quality Agent validates FAIR principles)
**Condition:** Source fails FAIR compliance (2/4 or fewer principles met)
**Flow:**
1. Quality Agent validates FAIR principles:
- **Findable:** No (no persistent identifier, incomplete metadata)
- **Accessible:** Yes (retrievable via URL)
- **Interoperable:** No (proprietary format, no standard vocabularies)
- **Reusable:** No (no license information, unclear provenance)
- **FAIR Compliance:** 1/4 principles met (fails threshold)
2. Quality Agent generates FAIR violation report:
- "FAIR Compliance Failure: 1/4 principles met"
- "Missing: Persistent identifier, standard formats, license"
- "Impact: Source may not be suitable for long-term archival"
3. Quality Agent provides remediation recommendations:
- "Add persistent identifier (DOI, ARK, Handle)"
- "Convert to standard format (PDF/A, markdown)"
- "Document license and provenance"
4. User chooses action:
- Option 1: Accept source with FAIR limitations (document in metadata)
- Option 2: Remediate FAIR violations before integration
- Option 3: Seek alternative source with FAIR compliance
5. If Option 2 chosen: Quality Agent guides remediation workflow
6. **Resume Main Flow:** Step 8 (Quality Agent generates quality report)
### Alt-3: Incomplete Metadata
**Branch Point:** Step 3 (Quality Agent validates metadata completeness)
**Condition:** Source metadata missing critical fields
**Flow:**
1. Quality Agent validates metadata:
- Citation: Present
- Source type: Missing (required field)
- Publication date: Missing (required field)
- Author: Present
- Access date: Present
2. Quality Agent detects missing required fields
3. Quality Agent generates metadata gap report:
- "Insufficient metadata for quality assessment"
- "Missing required fields: source_type, publication_date"
- "Action: Complete source documentation (UC-RF-003)"
4. Quality Agent prompts user:
- "Metadata incomplete. Options:"
- "1. Complete metadata now (guided workflow)"
- "2. Defer quality assessment until metadata complete"
5. User chooses: "Complete metadata now"
6. Quality Agent invokes Documentation Agent to collect missing fields
7. Documentation Agent prompts for missing data
8. User provides: source_type: "academic-paper", publication_date: "2024-03-15"
9. Metadata updated in source file
10. **Resume Main Flow:** Step 4 (Quality Agent initiates multi-dimensional assessment)
## 10. Exception Flows
### Exc-1: Source File Not Found
**Trigger:** Step 2 (Quality Agent retrieves source metadata)
**Condition:** Source metadata file does not exist
**Flow:**
1. Quality Agent attempts to read: `.aiwg/research/sources/<source-id>.md`
2. File not found error
3. Quality Agent checks alternative locations: `.aiwg/research/pending/<source-id>.md`
4. File not found in alternative locations
5. Quality Agent generates error report:
- "Source file not found: <source-id>"
- "Searched locations: sources/, pending/"
- "Action: Document source using UC-RF-003 before quality assessment"
6. User receives error message with remediation steps
7. Quality assessment aborted
8. User directed to document source first
### Exc-2: External Quality Service Unavailable
**Trigger:** Step 4 (Quality Agent evaluates authority - citation count)
**Condition:** External citation database (e.g., Semantic Scholar API) unavailable
**Flow:**
1. Quality Agent queries Semantic Scholar API for citation count
2. API returns: 503 Service Unavailable
3. Quality Agent detects external service failure
4. Quality Agent logs warning: "Citation count unavailable (Semantic Scholar API down)"
5. Quality Agent adjusts authority evaluation:
- Skips citation count component (unavailable)
- Increases weight of other authority indicators (author credentials, publisher reputation)
- Recalculates authority score with available data
6. Quality Agent adds note to quality report:
- "Citation count not assessed (external service unavailable)"
- "Authority score based on available indicators only"
7. **Resume Main Flow:** Step 5 (Quality Agent calculates weighted overall score)
### Exc-3: Conflicting Quality Indicators
**Trigger:** Step 5 (Quality Agent calculates overall score)
**Condition:** High authority but low accuracy scores (conflicting signals)
**Flow:**
1. Quality Agent detects conflicting scores:
- Authority score: 95/100 (highly credible author, prestigious journal)
- Accuracy score: 40/100 (methodology unclear, data sources not cited)
2. Quality Agent flags conflict:
- "Quality conflict detected: High authority, low accuracy"
- "Possible explanations: Preliminary findings, opinion piece, methodology not documented"
3. Quality Agent performs additional analysis:
- Checks source type: "perspective-article" (opinion/commentary, not research)
- Adjusts expectations: Lower accuracy requirements for perspective articles
- Recalibrates score interpretation
4. Quality Agent generates nuanced assessment:
- "Overall Score: 68/100 (Moderate Quality - Perspective Article)"
- "Note: High authority compensates for lower accuracy expectations"
- "Appropriate for: Context/opinion, not primary research evidence"
5. **Resume Main Flow:** Step 7 (Quality Agent performs GRADE assessment)
## 11. Special Requirements
### Performance Requirements
| Requirement | Target | Rationale |
|------------|--------|-----------|
| NFR-QA-01: Assessment time | <30 seconds per source | User experience (rapid feedback) |
| NFR-QA-02: Batch assessment | 100 sources in <15 minutes | Bulk import workflows |
### Quality Requirements
| Requirement | Target | Rationale |
|------------|--------|-----------|
| NFR-QA-03: Assessment consistency | 95%+ agreement with manual assessment | Reliability (trustworthy automation) |
| NFR-QA-04: GRADE/FAIR compliance | 100% adherence to framework standards | Standards compliance |
### Usability Requirements
| Requirement | Target | Rationale |
|------------|--------|-----------|
| NFR-QA-05: Report clarity | Actionable recommendations in <500 words | Decision support |
| NFR-QA-06: Dimension transparency | Individual dimension scores visible | Explainability |
## 12. Related Business Rules
**BR-QA-001: Quality Score Thresholds**
- High Quality: 70-100 (approved for integration)
- Moderate Quality: 50-69 (requires review, conditional approval)
- Low Quality: 0-49 (not recommended, seek alternatives)
**BR-QA-002: Dimension Weighting**
- Authority: 30% (who created it?)
- Currency: 20% (how recent?)
- Accuracy: 25% (is it correct?)
- Coverage: 15% (is it comprehensive?)
- Objectivity: 10% (is it balanced?)
**BR-QA-003: GRADE Rating Mapping**
- High: Strong confidence in evidence (use for critical decisions)
- Moderate: Moderate confidence (likely suitable but verify)
- Low: Limited confidence (use with caution)
- Very Low: Very limited confidence (seek better sources)
**BR-QA-004: FAIR Compliance Threshold**
- Fully Compliant: 4/4 principles met (archival-ready)
- Partially Compliant: 2-3/4 principles met (acceptable with documentation)
- Non-Compliant: 0-1/4 principles met (remediate or exclude from archival)
## 13. Data Requirements
### Input Data
| Data Element | Format | Source | Validation |
|-------------|--------|---------|-----------|
| Source Metadata | YAML frontmatter + Markdown | `.aiwg/research/sources/<source-id>.md` | Required fields present |
| Citation Information | Structured citation | Source metadata | Valid citation format |
| Source Type | Enum | Source metadata | Valid type (academic-paper, blog, documentation, etc.) |
| Publication Date | ISO 8601 date | Source metadata | Valid date format |
| Author/Organization | String | Source metadata | Non-empty |
### Output Data
| Data Element | Format | Destination | Retention |
|-------------|--------|-------------|----------|
| Quality Report | Markdown | `.aiwg/research/quality/<source-id>-quality-report.md` | Permanent |
| Quality Score | Integer (0-100) | Source metadata frontmatter | Permanent |
| GRADE Rating | Enum | Source metadata frontmatter | Permanent |
| FAIR Compliance | Boolean | Source metadata frontmatter | Permanent |
| Assessment Date | ISO 8601 date | Source metadata frontmatter | Permanent |
## 14. Open Issues and TODOs
1. **Issue 001: External API Dependencies**
- Description: Citation count relies on Semantic Scholar API - what if permanently unavailable?
- Impact: Authority evaluation may be incomplete
- Mitigation: Support multiple citation databases (Crossref, OpenCitations)
- Owner: Quality Agent
- Due Date: Elaboration phase
2. **TODO 001: Domain-Specific Quality Criteria**
- Description: Different research domains may require different quality criteria (medical vs software engineering)
- Enhancement: Add configurable quality criteria per domain
- Assigned: Quality Agent
- Due Date: Version 1.1
3. **TODO 002: Machine Learning Quality Prediction**
- Description: Train ML model to predict quality scores based on metadata patterns
- Benefit: Faster assessment for large batches
- Assigned: Quality Agent
- Due Date: Version 2.0
## 15. References
- [UC-RF-003: Document Research Sources](.aiwg/flows/research-framework/elaboration/use-cases/UC-RF-003-document-sources.md)
- [GRADE Framework](https://www.gradeworkinggroup.org/) - Quality assessment standard
- [FAIR Principles](https://www.go-fair.org/fair-principles/) - Data quality standard
- Quality Agent Definition (to be created)
---
## Traceability Matrix
### Requirements Traceability
| Requirement ID | Source | Implementation | Test Case |
|---------------|---------|----------------|-----------|
| Quality Assessment | Research Framework Inception | Quality Agent | TC-RF-006-001 through TC-RF-006-015 |
| GRADE Compliance | Framework Requirements | GRADE evaluation logic | TC-RF-006-005, TC-RF-006-006 |
| FAIR Compliance | Framework Requirements | FAIR validation logic | TC-RF-006-007, TC-RF-006-008 |
### SAD Component Mapping
**Primary Components:**
- Quality Agent (specialized assessment agent)
- Quality Scoring Engine (multi-dimensional evaluation)
- GRADE/FAIR Validators (standards compliance)
---
## Acceptance Criteria
### AC-001: Basic Quality Assessment
**Given:** Source documented with complete metadata
**When:** User requests quality assessment
**Then:**
- Quality score calculated (0-100)
- All 5 dimensions evaluated (authority, currency, accuracy, coverage, objectivity)
- Quality report generated in <30 seconds
- Source metadata updated with quality scores
### AC-002: GRADE Assessment
**Given:** Source is academic research paper
**When:** Quality Agent performs GRADE assessment
**Then:**
- Study design identified
- Risk of bias evaluated
- GRADE rating assigned (High/Moderate/Low/Very Low)
- Rating rationale documented in quality report
### AC-003: FAIR Validation
**Given:** Source documented with metadata
**When:** Quality Agent validates FAIR principles
**Then:**
- All 4 FAIR principles checked (Findable, Accessible, Interoperable, Reusable)
- FAIR compliance score calculated (0-4)
- Non-compliance issues identified with remediation guidance
### AC-004: Low-Quality Source Detection
**Given:** Source with quality score <50
**When:** Quality assessment completes
**Then:**
- Warning generated
- Critical deficiencies identified
- Alternative source recommendations provided
- User prompted to confirm continued use
### AC-005: Quality Report Generation
**Given:** Quality assessment complete
**When:** Quality report generated
**Then:**
- Report includes: executive summary, dimension scores, strengths, limitations, recommendations
- Report word count: <500 words (scannable)
- Report saved to `.aiwg/research/quality/`
---
## Test Cases
### TC-RF-006-001: Basic Quality Assessment Workflow
**Objective:** Validate end-to-end quality assessment
**Preconditions:** Source documented with complete metadata
**Test Steps:**
1. Invoke: `/assess-source-quality <source-id>`
2. Verify metadata retrieval
3. Verify all 5 dimensions evaluated
4. Verify overall score calculated
5. Verify quality report generated
6. Verify source metadata updated
**Expected Result:** Quality assessment completes successfully in <30 seconds
**Pass/Fail:** PASS if all steps complete and report generated
### TC-RF-006-002: High-Quality Source Assessment
**Objective:** Validate assessment of high-quality academic source
**Preconditions:** Peer-reviewed paper, recent, comprehensive
**Test Steps:**
1. Run quality assessment
2. Verify authority score: >80 (academic credentials, peer-reviewed)
3. Verify accuracy score: >90 (methodology documented, data cited)
4. Verify overall score: >70 (high quality threshold)
5. Verify GRADE rating: High or Moderate
6. Verify recommendation: "Approved for integration"
**Expected Result:** High-quality source correctly identified
**Pass/Fail:** PASS if overall score >70 and approved
### TC-RF-006-003: Low-Quality Source Detection
**Objective:** Validate detection of low-quality sources
**Preconditions:** Blog post, no author credentials, 5+ years old
**Test Steps:**
1. Run quality assessment
2. Verify authority score: <40 (unknown author, not peer-reviewed)
3. Verify currency score: <50 (outdated)
4. Verify overall score: <50 (low quality threshold)
5. Verify warning generated
6. Verify recommendation: "Seek higher-quality alternatives"
**Expected Result:** Low-quality source flagged with warning
**Pass/Fail:** PASS if score <50 and warning displayed
### TC-RF-006-004: Dimension Score Calculation
**Objective:** Validate weighted scoring algorithm
**Preconditions:** Source with known dimension scores
**Test Steps:**
1. Run quality assessment
2. Extract dimension scores: Authority 80, Currency 90, Accuracy 85, Coverage 75, Objectivity 85
3. Calculate expected overall: (80×0.30)+(90×0.20)+(85×0.25)+(75×0.15)+(85×0.10)
4. Expected: 24+18+21.25+11.25+8.5 = 83
5. Verify actual overall score: 83/100
**Expected Result:** Overall score matches weighted calculation
**Pass/Fail:** PASS if calculated score = 83
### TC-RF-006-005: GRADE Assessment - High Rating
**Objective:** Validate GRADE rating for high-quality RCT
**Preconditions:** Randomized controlled trial, low bias, consistent results
**Test Steps:**
1. Run quality assessment
2. Verify study design: Randomized controlled trial (starts at High)
3. Verify risk of bias: Low (maintain High)
4. Verify inconsistency: None (maintain High)
5. Verify GRADE rating: High
6. Verify rating documented in quality report
**Expected Result:** GRADE rating = High for high-quality RCT
**Pass/Fail:** PASS if GRADE = High
### TC-RF-006-006: GRADE Assessment - Downgrade
**Objective:** Validate GRADE downgrade logic
**Preconditions:** Observational study with moderate bias
**Test Steps:**
1. Run quality assessment
2. Verify study design: Observational study (starts at Low)
3. Verify risk of bias: Moderate (downgrade to Very Low)
4. Verify GRADE rating: Very Low
5. Verify downgrade rationale documented
**Expected Result:** GRADE rating = Very Low with rationale
**Pass/Fail:** PASS if GRADE = Very Low
### TC-RF-006-007: FAIR Validation - Full Compliance
**Objective:** Validate FAIR compliance for archival-ready source
**Preconditions:** Source with DOI, open access, standard format, clear license
**Test Steps:**
1. Run quality assessment
2. Verify Findable: Yes (DOI present, metadata complete)
3. Verify Accessible: Yes (open access, retrieval documented)
4. Verify Interoperable: Yes (standard format, controlled vocabulary)
5. Verify Reusable: Yes (license documented, provenance clear)
6. Verify FAIR compliance: 4/4
**Expected Result:** FAIR compliance = 4/4 (fully compliant)
**Pass/Fail:** PASS if all 4 principles met
### TC-RF-006-008: FAIR Validation - Failure
**Objective:** Validate FAIR failure detection
**Preconditions:** Source without persistent ID, proprietary format, no license
**Test Steps:**
1. Run quality assessment
2. Verify Findable: No (no persistent identifier)
3. Verify Accessible: Yes (URL accessible)
4. Verify Interoperable: No (proprietary format)
5. Verify Reusable: No (no license)
6. Verify FAIR compliance: 1/4 (fails threshold)
7. Verify remediation recommendations provided
**Expected Result:** FAIR failure detected with remediation guidance
**Pass/Fail:** PASS if 1/4 and recommendations provided
### TC-RF-006-009: Incomplete Metadata Exception
**Objective:** Validate handling of incomplete source metadata
**Preconditions:** Source missing publication date and source type
**Test Steps:**
1. Run quality assessment
2. Verify metadata validation detects missing fields
3. Verify error message: "Insufficient metadata for quality assessment"
4. Verify user prompted to complete metadata
5. User completes metadata
6. Quality assessment resumes successfully
**Expected Result:** Incomplete metadata handled gracefully
**Pass/Fail:** PASS if metadata completion prompted and assessment resumes
### TC-RF-006-010: External Service Unavailable
**Objective:** Validate handling of external API failures
**Preconditions:** Semantic Scholar API unavailable
**Test Steps:**
1. Mock Semantic Scholar API failure (503)
2. Run quality assessment
3. Verify citation count component skipped
4. Verify authority score calculated with available data
5. Verify warning logged: "Citation count unavailable"
6. Verify assessment completes despite API failure
**Expected Result:** Assessment completes with degraded data
**Pass/Fail:** PASS if assessment completes with warning
### TC-RF-006-011: Conflicting Quality Indicators
**Objective:** Validate handling of conflicting quality signals
**Preconditions:** High authority, low accuracy (perspective article)
**Test Steps:**
1. Run quality assessment
2. Verify authority score: 95 (prestigious author/journal)
3. Verify accuracy score: 40 (opinion piece, no methodology)
4. Verify conflict detected: "High authority, low accuracy"
5. Verify source type checked: "perspective-article"
6. Verify adjusted interpretation: Moderate quality for perspective
**Expected Result:** Conflict detected and resolved with context
**Pass/Fail:** PASS if nuanced assessment provided
### TC-RF-006-012: Batch Assessment Performance
**Objective:** Validate batch quality assessment performance
**Preconditions:** 100 documented sources
**Test Steps:**
1. Run batch quality assessment: 100 sources
2. Start timer
3. Wait for all assessments to complete
4. Stop timer
5. Verify completion time: <15 minutes (NFR-QA-02)
6. Verify all 100 quality reports generated
**Expected Result:** 100 sources assessed in <15 minutes
**Pass/Fail:** PASS if time <15 minutes and all reports generated
### TC-RF-006-013: Assessment Consistency
**Objective:** Validate consistency with manual assessment
**Preconditions:** 50 sources with manual quality assessments
**Test Steps:**
1. Run automated quality assessment on 50 sources
2. Compare automated scores to manual assessments
3. Calculate agreement rate: Matching scores / Total sources
4. Verify agreement: >95% (NFR-QA-03)
5. Investigate disagreements (acceptable variance ±5 points)
**Expected Result:** >95% agreement with manual assessment
**Pass/Fail:** PASS if agreement >95%
### TC-RF-006-014: Quality Report Clarity
**Objective:** Validate quality report usability
**Preconditions:** Quality assessment complete
**Test Steps:**
1. Open quality report
2. Verify report sections: Executive summary, dimension scores, strengths, limitations, recommendations
3. Verify word count: <500 words (NFR-QA-05)
4. Verify actionable recommendations: 1-3 clear actions
5. Verify dimension scores visible (NFR-QA-06)
**Expected Result:** Clear, actionable report <500 words
**Pass/Fail:** PASS if all sections present and word count <500
### TC-RF-006-015: End-to-End Quality Workflow
**Objective:** Validate complete quality assessment workflow
**Preconditions:** Source documented, metadata complete
**Test Steps:**
1. User triggers: `/assess-source-quality <source-id>`
2. Quality Agent retrieves metadata
3. Multi-dimensional assessment performed
4. GRADE assessment completed
5. FAIR validation performed
6. Quality report generated
7. Source metadata updated
8. User reviews report and approves source
**Expected Result:** Complete workflow executes successfully
**Pass/Fail:** PASS if all steps complete and source approved
---
## Document Metadata
**Version:** 1.0 (Initial Draft)
**Status:** DRAFT
**Created:** 2026-01-25
**Last Updated:** 2026-01-25
**Word Count:** 5,247 words
**Quality Score:** (To be assessed)
**Next Actions:**
1. Review use case with Quality Agent domain expert
2. Implement test cases TC-RF-006-001 through TC-RF-006-015
3. Create Quality Agent definition
4. Define quality scoring algorithms
5. Integrate with Research Framework workflow
---
**Generated:** 2026-01-25
**Owner:** Requirements Analyst (Research Framework)
**Status:** DRAFT - Pending Review