aiwg/agentic/code/frameworks/sdlc-complete/templates/test/regression-report.md

Cognitive architecture for AI-augmented software development with structured memory, ensemble validation, and closed-loop correction. FAIR-aligned artifacts, 84% cost reduction via human-in-the-loop, standards adopted by 100+ organizations.

# Regression Analysis Report

## Purpose

Document analysis of regression defects discovered during testing to identify patterns, trends, and systemic quality issues. Regression reports inform process improvements and help prevent future regressions by understanding their root causes and triggers.

## Ownership

- Owner: Test Engineer (regression detection and analysis)
- Contributors: Test Architect (trend analysis), Quality Manager (process improvement)
- Reviewers: Technical Lead (root cause review), Product Owner (prioritization)

## Phase 1: Summary (ESSENTIAL)

### Report Metadata

- Report ID: REG-REPORT-{YYYY-MM-DD}
- Reporting Period: {YYYY-MM-DD} to {YYYY-MM-DD}
- Generated By: {name/role}
- Date Generated: {YYYY-MM-DD}

<!-- EXAMPLE:
- Report ID: REG-REPORT-2026-01-15
- Reporting Period: 2025-12-01 to 2026-01-15
- Generated By: Test Engineer (automated)
- Date: 2026-01-15
-->

### Executive Summary

**Total Regressions Detected**: {count}

<!-- EXAMPLE: Total Regressions Detected: 12 -->

**Severity Breakdown**:
- Critical (P0): {count}
- High (P1): {count}
- Medium (P2): {count}
- Low (P3): {count}

<!-- EXAMPLE:
- Critical (P0): 2
- High (P1): 5
- Medium (P2): 3
- Low (P3): 2
-->

**Status Summary**:
- Active (unresolved): {count}
- Fixed (awaiting verification): {count}
- Verified (resolved): {count}
- Deferred: {count}

<!-- EXAMPLE:
- Active: 7
- Fixed: 3
- Verified: 2
- Deferred: 0
-->

**Top Affected Area**: {component/module}

<!-- EXAMPLE: Top Affected Area: Authentication module (5 regressions) -->

**Regression Rate**: {percentage}

<!-- EXAMPLE: Regression Rate: 8.3% (12 regressions / 145 changes) -->

**Mean Time to Detect (MTTD)**: {duration}

<!-- EXAMPLE: Mean Time to Detect: 2.3 days (from code change to detection) -->

**Mean Time to Resolve (MTTR)**: {duration}

<!-- EXAMPLE: Mean Time to Resolve: 4.7 days (from detection to fix verification) -->

## Phase 2: Detailed Analysis (EXPAND WHEN READY)

<details>
<summary>Click to expand individual regression details</summary>

### Individual Regressions

For each regression discovered during the reporting period, provide:

#### REG-{ID}: {Brief Description}

**Introduced**: {YYYY-MM-DD}
**Detected**: {YYYY-MM-DD}
**MTTD**: {days}

<!-- EXAMPLE:
#### REG-042: Session timeout breaks remember-me functionality

**Introduced**: 2025-12-15
**Detected**: 2025-12-18
**MTTD**: 3 days
-->

**Type**: {Functional | Performance | Security | Usability | Compatibility | Data | Integration}

<!-- EXAMPLE: Type: Functional -->

**Severity**: {Critical | High | Medium | Low}

<!-- EXAMPLE: Severity: High -->

**Status**: {Active | Fixed | Verified | Deferred}

<!-- EXAMPLE: Status: Fixed -->

**Trigger**: {What code change caused this regression?}

<!-- EXAMPLE:
Trigger: PR #3241 "Refactor session management" introduced stricter timeout validation that invalidates remember-me tokens after 30 minutes instead of 30 days.
-->

**Affected Component**: {module/service/file}

<!-- EXAMPLE: Affected Component: Authentication Service (src/auth/SessionManager.ts) -->

**User Impact**: {Describe how users are affected}

<!-- EXAMPLE:
User Impact: Users with "Remember Me" enabled are forced to re-login after 30 minutes, defeating the purpose of the feature. Affects estimated 60% of active users.
-->

**Root Cause**: {Why did this regression occur?}

<!-- EXAMPLE:
Root Cause: Developer changed SESSION_TTL constant but inadvertently applied it to both regular sessions and remember-me sessions. Previously, remember-me sessions had separate REMEMBER_ME_TTL constant (30 days) that was removed during refactoring.
-->

**Detection Method**: {How was this discovered?}

<!-- EXAMPLE:
Detection Method: Automated regression test TC-AUTH-089 "Verify remember-me persists across 7 days" failed in nightly test run.
-->

**Fix Status**: {If fixed, describe fix}

<!-- EXAMPLE:
Fix Status: FIXED - Restored separate REMEMBER_ME_TTL constant (30 days) and updated SessionManager to use appropriate TTL based on remember-me flag. Fix deployed to staging 2025-12-20, verified 2025-12-21.
-->

**Related Defects**: {Link to defect cards}

<!-- EXAMPLE: Related Defects: DEF-SHOP-158 -->

**Related Test**: {Link to regression test}

<!-- EXAMPLE: Related Test: TC-AUTH-089 (added to TS-REG-001) -->

---

<!-- ANTI-PATTERN: Vague description -->
#### REG-999: Login broken

**Type**: Bug
**Impact**: Users can't log in

<!-- BETTER: Specific and actionable -->
#### REG-999: OAuth SSO redirect fails with 404 after login server upgrade

**Introduced**: 2025-12-10
**Detected**: 2025-12-12
**Type**: Integration
**Severity**: Critical
**Trigger**: Upgrade to Auth0 library v5.2.0 changed default redirect URI from /callback to /auth/callback
**Affected Component**: SSO Integration (src/auth/OAuth2Provider.ts)
**User Impact**: All SSO users (enterprise customers, ~40% of user base) cannot log in
**Root Cause**: Library upgrade documentation not reviewed; redirect URI mismatch not detected in staging
**Detection Method**: Production monitoring alert (login success rate dropped to 60%)

<!-- WHY: Specific details enable root cause analysis and prevention -->

</details>

## Phase 3: Trends and Recommendations (ADVANCED)

<details>
<summary>Click to expand trend analysis and improvement recommendations</summary>

### Trend Analysis

**Regressions Over Time**:

| Period | Regressions | Changes | Rate |
|--------|-------------|---------|------|
| {Month 1} | {count} | {count} | {%} |
| {Month 2} | {count} | {count} | {%} |
| {Month 3} | {count} | {count} | {%} |

<!-- EXAMPLE:
| Period | Regressions | Changes | Rate |
|--------|-------------|---------|------|
| 2025-10 | 8 | 120 | 6.7% |
| 2025-11 | 10 | 135 | 7.4% |
| 2025-12 | 12 | 145 | 8.3% |

Trend: Regression rate increasing month-over-month (+1.6pp over 3 months)
-->

**Most Affected Components**:

| Component | Regressions | % of Total |
|-----------|-------------|------------|
| {component 1} | {count} | {%} |
| {component 2} | {count} | {%} |
| {component 3} | {count} | {%} |

<!-- EXAMPLE:
| Component | Regressions | % of Total |
|-----------|-------------|------------|
| Authentication | 5 | 42% |
| Payment Processing | 3 | 25% |
| User Profile | 2 | 17% |
| Search | 2 | 17% |

Pattern: Authentication module has highest regression rate (42%)
-->

**Regressions by Type**:

| Type | Count | % of Total |
|------|-------|------------|
| Functional | {count} | {%} |
| Performance | {count} | {%} |
| Security | {count} | {%} |
| Integration | {count} | {%} |
| Other | {count} | {%} |

<!-- EXAMPLE:
| Type | Count | % of Total |
|------|-------|------------|
| Functional | 7 | 58% |
| Performance | 2 | 17% |
| Integration | 2 | 17% |
| Security | 1 | 8% |

Pattern: Most regressions are functional (58%), indicating inadequate functional test coverage
-->

**Detection Method Distribution**:

| Method | Count | % of Total | Avg MTTD |
|--------|-------|------------|----------|
| Automated Test | {count} | {%} | {days} |
| Manual Testing | {count} | {%} | {days} |
| Production Monitoring | {count} | {%} | {days} |
| User Report | {count} | {%} | {days} |

<!-- EXAMPLE:
| Method | Count | % of Total | Avg MTTD |
|--------|-------|------------|----------|
| Automated Test | 6 | 50% | 1.2 days |
| Manual Testing | 3 | 25% | 3.5 days |
| Production Monitoring | 2 | 17% | 0.8 days |
| User Report | 1 | 8% | 7.0 days |

Pattern: Automated tests detect 50% of regressions with fastest MTTD. User-reported regressions have longest MTTD (7 days).
-->

**Common Root Causes**:

| Root Cause Category | Count | % of Total |
|---------------------|-------|------------|
| Incomplete testing | {count} | {%} |
| Refactoring side effects | {count} | {%} |
| Dependency changes | {count} | {%} |
| Configuration drift | {count} | {%} |
| Code merge conflicts | {count} | {%} |

<!-- EXAMPLE:
| Root Cause | Count | % of Total |
|------------|-------|------------|
| Incomplete test coverage | 5 | 42% |
| Refactoring side effects | 4 | 33% |
| Dependency upgrade issues | 2 | 17% |
| Configuration mismatch | 1 | 8% |

Pattern: 42% of regressions due to inadequate test coverage before code changes
-->

### Quality Metrics

**Regression Escape Rate**: {percentage}

Definition: Regressions that escaped to production / Total regressions detected

<!-- EXAMPLE: Regression Escape Rate: 16.7% (2 production escapes / 12 total regressions) -->

**Test Coverage vs Regression Correlation**:

| Component | Test Coverage | Regression Count |
|-----------|---------------|------------------|
| {component 1} | {%} | {count} |
| {component 2} | {%} | {count} |

<!-- EXAMPLE:
| Component | Test Coverage | Regression Count |
|-----------|---------------|------------------|
| Authentication | 68% | 5 |
| Payment | 85% | 3 |
| Profile | 92% | 2 |

Correlation: Components with lower test coverage have higher regression rates
-->

**MTTD Trend**:

- Current period MTTD: {days}
- Previous period MTTD: {days}
- Change: {+/- days or %}

<!-- EXAMPLE:
- Current period MTTD: 2.3 days
- Previous period MTTD: 1.8 days
- Change: +0.5 days (+28%)

Concern: MTTD increasing, indicating slower detection
-->

**MTTR Trend**:

- Current period MTTR: {days}
- Previous period MTTR: {days}
- Change: {+/- days or %}

<!-- EXAMPLE:
- Current period MTTR: 4.7 days
- Previous period MTTR: 3.2 days
- Change: +1.5 days (+47%)

Concern: MTTR increasing significantly, indicating slower fix cycle
-->

### Recommendations

Based on trend analysis, the following improvements are recommended:

#### High Priority

1. **{Recommendation 1}**
   - **Rationale**: {Why is this important?}
   - **Action**: {What specific steps?}
   - **Owner**: {Who should implement?}
   - **Timeline**: {When?}

<!-- EXAMPLE:
1. **Expand test coverage for Authentication module**
   - **Rationale**: Authentication has 68% coverage but highest regression rate (42% of all regressions). Gap analysis shows session management and OAuth flows lack adequate test coverage.
   - **Action**: Add 25 new test cases covering session lifecycle, remember-me functionality, and OAuth edge cases. Target 90% coverage.
   - **Owner**: Test Engineer + Software Implementer (auth team)
   - **Timeline**: Complete by end of Q1 2026
-->

2. **{Recommendation 2}**

<!-- EXAMPLE:
2. **Implement pre-merge regression smoke test**
   - **Rationale**: 75% of regressions would have been caught by running existing regression suite before merge. Currently regression tests only run nightly.
   - **Action**: Configure CI to run priority P0/P1 regression tests on all PRs before merge approval. Subset runs in <10 minutes.
   - **Owner**: DevOps Engineer
   - **Timeline**: Implement by 2026-02-01
-->

#### Medium Priority

3. **{Recommendation 3}**

<!-- EXAMPLE:
3. **Establish dependency upgrade review process**
   - **Rationale**: 17% of regressions caused by dependency upgrades (libraries, SDKs). Current process lacks systematic compatibility review.
   - **Action**: Create checklist for dependency upgrades including: changelog review, breaking change scan, staging validation, rollback plan.
   - **Owner**: Technical Lead
   - **Timeline**: Document process by 2026-02-15
-->

#### Low Priority

4. **{Recommendation 4}**

<!-- EXAMPLE:
4. **Improve production monitoring alerting**
   - **Rationale**: Production monitoring detected 2 regressions with fastest MTTD (0.8 days avg). Expanding monitoring could improve early detection.
   - **Action**: Add monitoring for key user flows (login success rate, checkout completion rate, API error rates) with automated alerts.
   - **Owner**: Reliability Engineer
   - **Timeline**: Phase 1 by 2026-03-01
-->

### Process Improvements

**Prevent Regressions**:

- [ ] {Prevention measure 1}
- [ ] {Prevention measure 2}

<!-- EXAMPLE:
- [ ] Require test coverage review on PRs affecting critical paths (auth, payment, checkout)
- [ ] Mandate refactoring to include test expansion, not just code changes
- [ ] Add static analysis rules to detect common regression patterns (e.g., constant misuse)
-->

**Detect Regressions Faster**:

- [ ] {Detection improvement 1}
- [ ] {Detection improvement 2}

<!-- EXAMPLE:
- [ ] Run regression suite subset on every PR (not just nightly)
- [ ] Enable canary deployments with automatic rollback on metric degradation
- [ ] Add synthetic monitoring tests for critical user journeys
-->

**Resolve Regressions Faster**:

- [ ] {Resolution improvement 1}
- [ ] {Resolution improvement 2}

<!-- EXAMPLE:
- [ ] Establish regression hotfix SLA: P0 within 4 hours, P1 within 24 hours
- [ ] Create regression war room protocol for critical issues
- [ ] Pre-allocate 20% of sprint capacity for regression fixes
-->

</details>

## Related Templates and References

- @agentic/code/frameworks/sdlc-complete/templates/test/regression-register.md - Track all regressions
- @agentic/code/frameworks/sdlc-complete/templates/test/defect-card.md - Individual defect tracking
- @agentic/code/frameworks/sdlc-complete/templates/test/test-evaluation-summary-template.md - Overall test results
- @agentic/code/frameworks/sdlc-complete/templates/test/regression-test-set-card.md - Regression test suite
- @agentic/code/frameworks/sdlc-complete/commands/regression-analyze.md - Generate this report (if available)

## Automation Notes

For agents generating regression analysis reports:

### Input Requirements

- Regression register data (all REG-XXX entries)
- Defect tracking data (DEF-XXX cards)
- Test execution history (pass/fail, dates)
- Code change history (commits, PRs, deployment dates)
- Production monitoring data (if available)

### Analysis Steps

1. **Collect regressions** from reporting period
2. **Calculate metrics** (count, severity distribution, MTTD, MTTR)
3. **Identify trends** (compare to previous periods)
4. **Correlate with coverage** (test coverage vs regression rate)
5. **Categorize root causes** (group similar issues)
6. **Generate recommendations** (based on patterns detected)

### Output Artifacts

- Regression analysis report (this template)
- Updated regression register
- Process improvement recommendations
- Trend charts (if visualization available)

## Checklist: Report Completeness

Before finalizing regression analysis report:

- [ ] All regressions from period included
- [ ] Each regression has complete details (intro date, detection date, MTTD, root cause)
- [ ] Severity and status accurate
- [ ] Metrics calculated correctly (regression rate, MTTD, MTTR)
- [ ] Trend analysis compares to previous period(s)
- [ ] Common patterns identified
- [ ] Recommendations specific and actionable
- [ ] Recommendations have assigned owners and timelines
- [ ] Process improvements mapped to root causes
- [ ] Report reviewed by Test Architect or Quality Manager

This report is complete when all checklist items are satisfied and recommendations are ready for stakeholder review.