@cloudkinetix/bmad-enhanced/expansion-packs/ck-jira-integration/utils/correlation-analysis.md

Cloud-Kinetix enhanced fork of BMAD-METHOD - Breakthrough Method of Agile AI-driven Development with robust versioning and unified validation.

# correlation-analysis

Specialized utility functions for calculating three-way correlation health scores, analyzing BMAD-JIRA-Git relationships, and performing comprehensive gap analysis with evidence-based recommendations.

## Core Correlation Algorithms

### Three-Way Health Score Calculation

**Overall Correlation Health Formula:**

```
Overall Health = (
  (BMAD-JIRA Correlation × 0.3) +
  (JIRA-Git Correlation × 0.4) +
  (BMAD-Git Correlation × 0.3)
)

Where each pairwise correlation = (
  Completeness Score × 0.4 +
  Recency Score × 0.3 +
  Quality Score × 0.2 +
  Consistency Score × 0.1
)
```

**Completeness Score (0-100%):**

- Percentage of items with valid cross-references
- Weighted by importance (epics > stories > tasks)
- Adjusted for expected correlation types

**Recency Score (0-100%):**

- Time decay factor for correlation freshness
- Recent correlations weighted higher
- Stale correlations penalized progressively

**Quality Score (0-100%):**

- Strength of correlation evidence
- Explicit references > implicit mentions > inferred links
- Validation through multiple confirmation sources

**Consistency Score (0-100%):**

- Status alignment across systems
- Priority consistency validation
- Timeline and milestone alignment

### Epic-Level Correlation Assessment

**Epic Health Categories:**

1. **Complete (90-100% health)**: All three systems fully aligned with strong evidence
2. **Excellent (80-89% health)**: Minor gaps in one system, strong overall alignment
3. **Good (70-79% health)**: Moderate gaps in 1-2 systems, core alignment maintained
4. **Partial (50-69% health)**: Significant gaps but basic correlation exists
5. **Incomplete (25-49% health)**: Major alignment issues across multiple systems
6. **Critical (0-24% health)**: Minimal or no correlation, requires immediate attention

**Epic Assessment Factors:**

- **Story Coverage**: Percentage of epic stories with JIRA tickets
- **Implementation Progress**: Git activity correlation with planned work
- **Status Synchronization**: Alignment of completion status across systems
- **Priority Consistency**: Strategic importance reflected consistently
- **Timeline Alignment**: Milestone and deadline coordination

### Story-Level Correlation Analysis

**Story Correlation Patterns:**

- **Perfect Correlation**: BMAD story → JIRA ticket → Git commits with clear traceability
- **Forward Correlation**: Story and ticket exist, implementation in progress
- **Reverse Correlation**: Implementation exists, documentation created retroactively
- **Partial Correlation**: Some connections exist but significant gaps remain
- **No Correlation**: Story exists in isolation without tracking or implementation

**Story Health Indicators:**

- **Ticket Association**: Valid JIRA ticket linked to story
- **Implementation Evidence**: Git commits referencing story or ticket
- **Acceptance Criteria Alignment**: JIRA description matches story requirements
- **Status Consistency**: Progress reflected accurately across systems
- **Timeline Adherence**: Development progress aligns with planned schedule

## Gap Analysis Functions

### Missing Correlation Detection

**BMAD Stories Without JIRA Tickets:**

```python
def detect_missing_jira_tickets(bmad_stories, jira_tickets):
    missing_tickets = []
    for story in bmad_stories:
        if not has_jira_reference(story) or not validate_jira_link(story, jira_tickets):
            missing_tickets.append({
                'story': story,
                'epic': extract_epic(story),
                'priority': calculate_story_priority(story),
                'suggested_action': generate_ticket_creation_recommendation(story)
            })
    return prioritize_by_impact(missing_tickets)
```

**Orphaned Git Commits:**

```python
def detect_orphaned_commits(git_commits, jira_tickets, bmad_stories):
    orphaned = []
    for commit in git_commits:
        jira_ref = extract_jira_reference(commit)
        bmad_ref = infer_bmad_story(commit, bmad_stories)

        if not jira_ref and not bmad_ref:
            orphaned.append({
                'commit': commit,
                'suggested_story': suggest_story_correlation(commit, bmad_stories),
                'suggested_ticket': suggest_ticket_correlation(commit, jira_tickets),
                'impact_assessment': assess_orphan_impact(commit)
            })
    return orphaned
```

**Documentation Drift Detection:**

```python
def detect_documentation_drift(git_commits, bmad_stories):
    drift_cases = []
    for story in bmad_stories:
        related_commits = find_related_commits(story, git_commits)
        implementation_scope = analyze_commit_scope(related_commits)
        documented_scope = extract_story_scope(story)

        drift_score = calculate_scope_divergence(implementation_scope, documented_scope)
        if drift_score > DRIFT_THRESHOLD:
            drift_cases.append({
                'story': story,
                'drift_score': drift_score,
                'implementation_changes': implementation_scope,
                'documentation_gaps': identify_documentation_gaps(implementation_scope, documented_scope),
                'update_recommendation': generate_doc_update_plan(story, implementation_scope)
            })
    return drift_cases
```

### Correlation Quality Assessment

**Reference Strength Scoring:**

- **Explicit Reference (100 points)**: Direct ticket ID or story filename mention
- **Strong Implicit (80 points)**: Clear description matching with high confidence
- **Moderate Implicit (60 points)**: Probable connection based on timing and content
- **Weak Implicit (40 points)**: Possible connection with low confidence
- **Inferred (20 points)**: Algorithm-suggested correlation requiring validation

**Temporal Correlation Validation:**

```python
def validate_temporal_correlation(bmad_story, jira_ticket, git_commits):
    story_date = extract_story_creation_date(bmad_story)
    ticket_date = extract_ticket_creation_date(jira_ticket)
    commit_dates = [extract_commit_date(c) for c in git_commits]

    # Validate logical sequence: story → ticket → commits
    sequence_validity = validate_chronological_sequence(story_date, ticket_date, commit_dates)

    # Check for reasonable time gaps
    time_gap_validity = validate_time_gaps(story_date, ticket_date, commit_dates)

    return {
        'sequence_valid': sequence_validity,
        'timing_reasonable': time_gap_validity,
        'confidence_score': calculate_temporal_confidence(sequence_validity, time_gap_validity)
    }
```

## Evidence-Based Analysis Functions

### Development Activity Correlation

**Commit Pattern Analysis:**

```python
def analyze_commit_patterns(commits, jira_tickets, bmad_stories):
    patterns = {
        'ticket_referenced_commits': count_commits_with_jira_refs(commits),
        'story_correlated_commits': count_commits_with_story_correlation(commits, bmad_stories),
        'orphaned_commits': count_orphaned_commits(commits, jira_tickets, bmad_stories),
        'commit_message_quality': assess_commit_message_quality(commits),
        'development_velocity': calculate_development_velocity(commits),
        'correlation_trends': analyze_correlation_trends_over_time(commits)
    }
    return patterns
```

**Implementation Scope Validation:**

```python
def validate_implementation_scope(bmad_story, git_commits):
    story_acceptance_criteria = extract_acceptance_criteria(bmad_story)
    story_scope = parse_story_scope(bmad_story)

    implementation_evidence = []
    for commit in git_commits:
        changed_files = extract_changed_files(commit)
        implementation_impact = assess_implementation_impact(changed_files, commit.message)
        scope_alignment = calculate_scope_alignment(implementation_impact, story_scope)

        implementation_evidence.append({
            'commit': commit,
            'scope_alignment': scope_alignment,
            'criteria_coverage': assess_criteria_coverage(implementation_impact, story_acceptance_criteria)
        })

    return {
        'overall_scope_alignment': calculate_overall_alignment(implementation_evidence),
        'criteria_coverage_percentage': calculate_criteria_coverage_percentage(implementation_evidence),
        'scope_creep_indicators': detect_scope_creep(implementation_evidence, story_scope)
    }
```

### Quality and Consistency Analysis

**Status Synchronization Assessment:**

```python
def assess_status_synchronization(bmad_stories, jira_tickets, git_commits):
    sync_assessment = []

    for story in bmad_stories:
        story_status = extract_story_status(story)
        related_tickets = find_related_jira_tickets(story, jira_tickets)
        related_commits = find_related_commits(story, git_commits)

        ticket_statuses = [extract_ticket_status(t) for t in related_tickets]
        implementation_progress = assess_implementation_progress(related_commits)

        sync_score = calculate_status_sync_score(story_status, ticket_statuses, implementation_progress)
        consistency_issues = identify_consistency_issues(story_status, ticket_statuses, implementation_progress)

        sync_assessment.append({
            'story': story,
            'sync_score': sync_score,
            'consistency_issues': consistency_issues,
            'recommended_actions': generate_sync_recommendations(consistency_issues)
        })

    return sync_assessment
```

**Priority Alignment Validation:**

```python
def validate_priority_alignment(bmad_epics, jira_project):
    alignment_analysis = []

    for epic in bmad_epics:
        bmad_priority = extract_epic_priority(epic)
        bmad_strategic_importance = assess_strategic_importance(epic)

        related_jira_epic = find_related_jira_epic(epic, jira_project)
        jira_priority = extract_jira_epic_priority(related_jira_epic) if related_jira_epic else None

        priority_alignment_score = calculate_priority_alignment(bmad_priority, jira_priority)
        strategic_consistency = assess_strategic_consistency(bmad_strategic_importance, jira_priority)

        alignment_analysis.append({
            'epic': epic,
            'priority_alignment_score': priority_alignment_score,
            'strategic_consistency': strategic_consistency,
            'alignment_recommendations': generate_priority_alignment_recommendations(bmad_priority, jira_priority)
        })

    return alignment_analysis
```

## Trend Analysis and Forecasting

### Correlation Health Trends

**Historical Health Tracking:**

```python
def track_correlation_health_over_time(historical_data, current_data):
    health_trends = {
        'overall_health_trend': calculate_health_trend(historical_data, current_data),
        'epic_level_trends': analyze_epic_health_trends(historical_data, current_data),
        'correlation_type_trends': analyze_correlation_type_trends(historical_data, current_data),
        'improvement_velocity': calculate_improvement_velocity(historical_data, current_data),
        'regression_detection': detect_health_regressions(historical_data, current_data)
    }
    return health_trends
```

**Predictive Health Modeling:**

```python
def predict_correlation_health(historical_trends, current_trajectory, planned_improvements):
    prediction_models = {
        'linear_projection': project_linear_trend(historical_trends, current_trajectory),
        'improvement_impact': model_improvement_impact(planned_improvements),
        'risk_adjusted_forecast': apply_risk_adjustments(current_trajectory, identify_risk_factors()),
        'scenario_analysis': generate_scenario_forecasts(historical_trends, current_trajectory),
        'confidence_intervals': calculate_prediction_confidence(historical_trends)
    }

    return {
        'predicted_health_30_days': prediction_models['linear_projection']['30_day'],
        'predicted_health_90_days': prediction_models['linear_projection']['90_day'],
        'improvement_recommendations': identify_high_impact_improvements(prediction_models),
        'risk_mitigation_priorities': prioritize_risk_mitigation(prediction_models['risk_adjusted_forecast'])
    }
```

## Actionable Recommendations Engine

### Prioritized Action Generation

**Critical Gap Prioritization:**

```python
def prioritize_correlation_improvements(correlation_gaps, business_context, resource_constraints):
    prioritized_actions = []

    for gap in correlation_gaps:
        business_impact = assess_business_impact(gap, business_context)
        implementation_effort = estimate_implementation_effort(gap, resource_constraints)
        risk_level = assess_gap_risk_level(gap)

        priority_score = calculate_priority_score(business_impact, implementation_effort, risk_level)

        prioritized_actions.append({
            'gap': gap,
            'priority_score': priority_score,
            'business_impact': business_impact,
            'implementation_plan': generate_implementation_plan(gap, resource_constraints),
            'success_metrics': define_success_metrics(gap),
            'timeline_estimate': estimate_completion_timeline(gap, resource_constraints)
        })

    return sort_by_priority(prioritized_actions)
```

**Process Improvement Recommendations:**

```python
def generate_process_improvements(correlation_patterns, team_metrics, historical_effectiveness):
    improvements = {
        'immediate_wins': identify_quick_wins(correlation_patterns),
        'systematic_improvements': identify_systematic_improvements(correlation_patterns, team_metrics),
        'tool_integration_opportunities': identify_tool_integration_needs(correlation_patterns),
        'training_recommendations': identify_training_needs(team_metrics, historical_effectiveness),
        'policy_updates': recommend_policy_updates(correlation_patterns, historical_effectiveness)
    }

    return prioritize_improvements(improvements)
```

This correlation analysis utility provides the mathematical and algorithmic foundation for generating meaningful insights from three-way correlation data, enabling the JIRA agent to deliver precise, actionable, and evidence-based recommendations for improving organizational alignment and development process effectiveness.