aiwg
Version:
Cognitive architecture for AI-augmented software development with structured memory, ensemble validation, and closed-loop correction. FAIR-aligned artifacts, 84% cost reduction via human-in-the-loop, standards adopted by 100+ organizations.
414 lines (306 loc) • 15 kB
Markdown
# Regression Register
## Purpose
Maintain a comprehensive, living record of all regression defects discovered across the project lifecycle. The regression register serves as the single source of truth for regression tracking, trend analysis, and quality metrics.
## Ownership
- Owner: Test Engineer (register maintenance)
- Contributors: Test Architect (analysis), Software Implementer (fixes)
- Reviewers: Quality Manager (monthly review), Technical Lead (trend review)
## Phase 1: Active Regressions (ESSENTIAL)
### Active Regressions Table
Current regressions requiring attention:
| ID | Type | Severity | Status | Introduced | Detected | MTTD | Affected Component | Trigger | Assigned To |
|----|------|----------|--------|------------|----------|------|-------------------|---------|-------------|
| REG-{id} | {type} | {P0-P3} | {Active/Fixed} | {date} | {date} | {days} | {component} | {PR/commit} | {owner} |
<!-- EXAMPLE:
| ID | Type | Severity | Status | Introduced | Detected | MTTD | Affected Component | Trigger | Assigned To |
|----|------|----------|--------|------------|----------|------|-------------------|---------|-------------|
| REG-042 | Functional | P1 | Fixed | 2025-12-15 | 2025-12-18 | 3d | Authentication | PR #3241 | Jane Doe |
| REG-043 | Performance | P2 | Active | 2025-12-20 | 2025-12-23 | 3d | Search Service | PR #3289 | John Smith |
| REG-044 | Integration | P0 | Active | 2026-01-05 | 2026-01-06 | 1d | Payment Gateway | Stripe SDK v5.2 | Alice Johnson |
| REG-045 | Security | P1 | Fixed | 2026-01-08 | 2026-01-10 | 2d | Session Management | PR #3402 | Bob Wilson |
| REG-046 | Functional | P2 | Active | 2026-01-12 | 2026-01-15 | 3d | User Profile | PR #3450 | Carol Davis |
-->
<!-- ANTI-PATTERN: Incomplete tracking -->
| ID | Type | Status |
|----|------|--------|
| REG-999 | Bug | Open |
<!-- BETTER: Complete context for triage and resolution -->
| ID | Type | Severity | Status | Introduced | Detected | MTTD | Affected Component | Trigger | Assigned To |
|----|------|----------|--------|------------|----------|------|-------------------|---------|-------------|
| REG-999 | Integration | P0 | Active | 2026-01-10 | 2026-01-12 | 2d | OAuth SSO | Auth0 v5.2.0 upgrade | Security Team |
<!-- WHY: Full context enables prioritization, root cause analysis, and pattern detection -->
### Status Definitions
- **Active**: Regression detected, not yet fixed
- **Fixed**: Fix implemented and deployed to test environment, awaiting verification
- **Verified**: Fix verified by testing, awaiting production deployment
- **Closed**: Fix deployed to production and verified
- **Deferred**: Decided to defer fix to future iteration
### Severity Definitions
- **P0 (Critical)**: System unusable, data loss, security breach, complete feature failure - fix immediately
- **P1 (High)**: Major feature broken, significant user impact, no workaround - fix this iteration
- **P2 (Medium)**: Feature degraded, workaround exists - fix next iteration
- **P3 (Low)**: Cosmetic issue, minor inconvenience - fix when convenient
### Type Definitions
- **Functional**: Feature doesn't work as previously specified
- **Performance**: Unacceptable slowness or resource usage degradation
- **Security**: New vulnerability or security feature regression
- **Usability**: User experience degradation
- **Compatibility**: Breaks compatibility with browser/OS/device
- **Data**: Incorrect calculation or data integrity issue
- **Integration**: Fails to integrate with external system
- **Documentation**: Inaccurate or missing documentation after code change
## Phase 2: Recently Resolved (EXPAND WHEN READY)
<details>
<summary>Click to expand recently resolved regressions (last 30 days)</summary>
### Resolved Regressions Table
Regressions fixed and verified in the last 30 days:
| ID | Type | Severity | Introduced | Detected | Fixed | Verified | MTTD | MTTR | Component | Root Cause Summary |
|----|------|----------|------------|----------|-------|----------|------|------|-----------|-------------------|
| REG-{id} | {type} | {P0-P3} | {date} | {date} | {date} | {date} | {days} | {days} | {component} | {summary} |
<!-- EXAMPLE:
| ID | Type | Severity | Introduced | Detected | Fixed | Verified | MTTD | MTTR | Component | Root Cause |
|----|------|----------|------------|----------|-------|----------|------|------|-----------|------------|
| REG-038 | Functional | P1 | 2025-11-20 | 2025-11-22 | 2025-11-25 | 2025-11-26 | 2d | 4d | Email Service | Config error in SMTP settings |
| REG-039 | Performance | P2 | 2025-11-25 | 2025-11-28 | 2025-12-02 | 2025-12-03 | 3d | 5d | Database | Missing index after schema migration |
| REG-040 | Integration | P1 | 2025-12-01 | 2025-12-02 | 2025-12-05 | 2025-12-06 | 1d | 4d | Analytics | GA4 API version mismatch |
| REG-041 | Security | P0 | 2025-12-10 | 2025-12-10 | 2025-12-11 | 2025-12-12 | 0d | 2d | CORS | Overly permissive origin whitelist |
-->
**Metrics for Resolved Regressions (Last 30 Days)**:
- Total Resolved: {count}
- Average MTTD: {days}
- Average MTTR: {days}
- Fastest Resolution: {REG-ID} ({days})
- Slowest Resolution: {REG-ID} ({days})
<!-- EXAMPLE:
- Total Resolved: 4
- Average MTTD: 1.5 days
- Average MTTR: 3.75 days
- Fastest: REG-041 (2 days)
- Slowest: REG-039 (5 days)
-->
</details>
## Phase 3: Metrics and Patterns (ADVANCED)
<details>
<summary>Click to expand regression metrics and trend analysis</summary>
### Overall Metrics
**All-Time Statistics** (since project start):
- Total Regressions Tracked: {count}
- Active Regressions: {count}
- Resolved Regressions: {count}
- Deferred Regressions: {count}
<!-- EXAMPLE:
- Total: 46
- Active: 5
- Resolved: 38
- Deferred: 3
-->
**Current Regression Rate**: {percentage}
Formula: (Regressions detected in period) / (Code changes in period) × 100
<!-- EXAMPLE: Current Rate: 8.3% (12 regressions / 145 PRs merged in last 30 days) -->
**Regression Escape Rate**: {percentage}
Formula: (Regressions escaped to production) / (Total regressions detected) × 100
<!-- EXAMPLE: Escape Rate: 15% (7 production escapes / 46 total regressions) -->
**Mean Time to Detect (MTTD)**: {days}
Average time from code change introduction to regression detection.
<!-- EXAMPLE: MTTD: 2.1 days (median: 1.5 days) -->
**Mean Time to Resolve (MTTR)**: {days}
Average time from detection to fix verification.
<!-- EXAMPLE: MTTR: 4.3 days (median: 3.5 days) -->
### Trends Over Time
**Regression Count by Month**:
| Month | Regressions | Changes | Rate | MTTD | MTTR |
|-------|-------------|---------|------|------|------|
| {YYYY-MM} | {count} | {count} | {%} | {days} | {days} |
<!-- EXAMPLE:
| Month | Regressions | Changes | Rate | MTTD | MTTR |
|-------|-------------|---------|------|------|------|
| 2025-10 | 8 | 120 | 6.7% | 1.8d | 3.2d |
| 2025-11 | 10 | 135 | 7.4% | 2.0d | 3.8d |
| 2025-12 | 12 | 145 | 8.3% | 2.3d | 4.7d |
| 2026-01 | 5 | 62 | 8.1% | 1.9d | 3.9d |
Trend: Regression rate stable around 8%, but MTTR increasing (concerning)
-->
**Regressions by Severity**:
| Severity | Count | % of Total | Avg MTTR |
|----------|-------|------------|----------|
| P0 (Critical) | {count} | {%} | {days} |
| P1 (High) | {count} | {%} | {days} |
| P2 (Medium) | {count} | {%} | {days} |
| P3 (Low) | {count} | {%} | {days} |
<!-- EXAMPLE:
| Severity | Count | % of Total | Avg MTTR |
|----------|-------|------------|----------|
| P0 | 6 | 13% | 1.8d |
| P1 | 18 | 39% | 3.5d |
| P2 | 15 | 33% | 5.2d |
| P3 | 7 | 15% | 7.8d |
Pattern: Critical regressions resolved fastest (1.8d avg), lower priority takes longer
-->
**Regressions by Type**:
| Type | Count | % of Total | Avg MTTD |
|------|-------|------------|----------|
| Functional | {count} | {%} | {days} |
| Performance | {count} | {%} | {days} |
| Security | {count} | {%} | {days} |
| Integration | {count} | {%} | {days} |
| Other | {count} | {%} | {days} |
<!-- EXAMPLE:
| Type | Count | % of Total | Avg MTTD |
|------|-------|------------|----------|
| Functional | 26 | 57% | 2.3d |
| Performance | 8 | 17% | 3.5d |
| Integration | 6 | 13% | 1.8d |
| Security | 4 | 9% | 1.0d |
| Other | 2 | 4% | 2.5d |
Pattern: Security regressions detected fastest (1.0d), performance slowest (3.5d)
-->
**Most Affected Components**:
| Component | Regressions | % of Total | Latest |
|-----------|-------------|------------|--------|
| {component 1} | {count} | {%} | {REG-ID} |
| {component 2} | {count} | {%} | {REG-ID} |
| {component 3} | {count} | {%} | {REG-ID} |
<!-- EXAMPLE:
| Component | Regressions | % of Total | Latest |
|-----------|-------------|------------|--------|
| Authentication | 12 | 26% | REG-045 |
| Payment Gateway | 8 | 17% | REG-044 |
| Search Service | 6 | 13% | REG-043 |
| User Profile | 5 | 11% | REG-046 |
| Other | 15 | 33% | - |
Pattern: Authentication has highest regression rate (26%), indicating quality concern
-->
**Detection Method Breakdown**:
| Detection Method | Count | % of Total | Avg MTTD |
|------------------|-------|------------|----------|
| Automated Regression Test | {count} | {%} | {days} |
| Manual Testing | {count} | {%} | {days} |
| Production Monitoring | {count} | {%} | {days} |
| User Report | {count} | {%} | {days} |
| Code Review | {count} | {%} | {days} |
<!-- EXAMPLE:
| Detection Method | Count | % of Total | Avg MTTD |
|------------------|-------|------------|----------|
| Automated Test | 22 | 48% | 1.2d |
| Manual Testing | 12 | 26% | 3.5d |
| Production Monitoring | 7 | 15% | 0.8d |
| User Report | 4 | 9% | 7.2d |
| Code Review | 1 | 2% | 0.1d |
Pattern: Automated tests catch nearly half (48%), monitoring fastest (0.8d MTTD)
-->
**Root Cause Distribution**:
| Root Cause Category | Count | % of Total |
|---------------------|-------|------------|
| Inadequate test coverage | {count} | {%} |
| Refactoring side effects | {count} | {%} |
| Dependency/library changes | {count} | {%} |
| Configuration mismatch | {count} | {%} |
| Code merge conflicts | {count} | {%} |
| Design flaw exposed | {count} | {%} |
| Other | {count} | {%} |
<!-- EXAMPLE:
| Root Cause | Count | % of Total |
|------------|-------|------------|
| Inadequate test coverage | 18 | 39% |
| Refactoring side effects | 12 | 26% |
| Dependency changes | 8 | 17% |
| Configuration mismatch | 4 | 9% |
| Merge conflicts | 2 | 4% |
| Design flaw | 2 | 4% |
Pattern: 39% due to test gaps - primary improvement opportunity
-->
### Quality Alerts
**Current Alerts** (thresholds exceeded):
- [ ] Regression rate > 10%: {current rate}
- [ ] MTTD > 3 days: {current MTTD}
- [ ] MTTR > 5 days: {current MTTR}
- [ ] Escape rate > 20%: {current escape rate}
- [ ] Active P0 regressions > 2: {current P0 count}
- [ ] Same component regressed > 3 times in 30 days: {component name}
<!-- EXAMPLE:
Current Alerts:
- [ ] Regression rate: 8.3% (OK, threshold 10%)
- [x] MTTD: 2.3 days (OK, threshold 3 days)
- [x] MTTR: 4.7 days (WARNING - approaching threshold of 5 days)
- [ ] Escape rate: 15% (OK, threshold 20%)
- [x] Active P0: 1 (OK, threshold 2)
- [x] Authentication: 3 regressions in last 30 days (ALERT - investigate)
-->
**Alert Actions**:
When alerts triggered:
1. **Regression rate > 10%**: Review recent changes for quality issues, consider code freeze
2. **MTTD > 3 days**: Improve test automation coverage, increase test frequency
3. **MTTR > 5 days**: Allocate more dev capacity to regression fixes, streamline fix process
4. **Escape rate > 20%**: Review pre-production testing rigor, add smoke tests
5. **Active P0 > 2**: Declare regression crisis, daily standup until resolved
6. **Component repeated regressions**: Deep-dive quality review of component, refactor or rewrite
</details>
## Related Templates and References
- @agentic/code/frameworks/sdlc-complete/templates/test/regression-report.md - Periodic analysis reports
- @agentic/code/frameworks/sdlc-complete/templates/test/defect-card.md - Individual regression details
- @agentic/code/frameworks/sdlc-complete/templates/test/regression-test-set-card.md - Regression test suites
- @agentic/code/frameworks/sdlc-complete/templates/test/test-evaluation-summary-template.md - Test cycle summaries
- @agentic/code/frameworks/sdlc-complete/commands/regression-track.md - Update register (if available)
## Register Maintenance
### Update Frequency
- **Daily**: Add new regressions as detected
- **Weekly**: Update status of active regressions
- **Monthly**: Generate regression analysis report, review trends
### Adding New Regressions
When a regression is detected:
1. Assign next sequential ID: REG-{project}-{number}
2. Add row to Active Regressions table
3. Create detailed defect card (DEF-{id}) if needed
4. Link to regression test case (TC-{id})
5. Assign to responsible developer
6. Update metrics
### Updating Regression Status
When status changes:
1. Update Status column in Active Regressions table
2. If Fixed → add Fix date, calculate MTTR
3. If Verified → move to Recently Resolved table
4. If Closed → archive (can remove from Recently Resolved after 30 days)
5. If Deferred → update with deferral reason and target date
6. Recalculate metrics
### Archival Policy
- **Active Regressions**: Keep all until closed
- **Recently Resolved**: Keep last 30 days
- **Archived**: Move closed regressions >30 days to archive file
- **Metrics**: Retain all-time metrics for trend analysis
## Automation Notes
For agents maintaining the regression register:
### Automated Updates
Agents SHOULD automatically:
1. **Add new regressions** when regression tests fail
2. **Update MTTD** when regression detected
3. **Update status** when fixes merged/deployed
4. **Update MTTR** when fix verified
5. **Move to Resolved** when verified
6. **Recalculate metrics** on each update
7. **Trigger alerts** when thresholds exceeded
### Manual Reviews Required
Humans MUST review:
1. **Severity assignment** - Agent can suggest, human confirms
2. **Root cause analysis** - Requires investigation
3. **Deferral decisions** - Business priority judgment
4. **Trend interpretation** - Context-dependent analysis
5. **Process improvements** - Strategic decisions
### Integration Points
- Test execution results (automated test failures)
- Defect tracking system (DEF-{id} cards)
- Code repository (PRs, commits, deployment dates)
- Production monitoring (alerting, metrics)
- Regression test suites (TS-REG-{id})
## Checklist: Register Health
Monthly register review checklist:
- [ ] All active regressions have assigned owners
- [ ] All regressions have accurate MTTD and status
- [ ] No stale regressions (>30 days in Active without update)
- [ ] Metrics calculated correctly
- [ ] Trends reviewed and documented
- [ ] Alerts addressed or escalated
- [ ] Recently Resolved cleaned (only last 30 days)
- [ ] High-regression components flagged for review
- [ ] Root cause patterns identified
- [ ] Process improvements proposed based on patterns
Register is healthy when all checklist items pass.