@cloudkinetix/bmad-enhanced/expansion-packs/ck-gitlab-cicd-automation/tasks/monitor-pipeline-status.md

Cloud-Kinetix enhanced fork of BMAD-METHOD - Breakthrough Method of Agile AI-driven Development with robust versioning and unified validation.

# monitor-pipeline-status

## Task: Real-Time GitLab Pipeline Monitoring

**Purpose**: Autonomous monitoring of GitLab CI/CD pipeline status with intelligent alerting and cross-pack integration coordination.

**When to Use**:

- Continuous pipeline health monitoring
- Pre-merge CI validation
- Integration status checks with JIRA/parallel-dev
- Automated pipeline health reporting

---

## Task Configuration

### Input Parameters

- `branch` (optional): Target branch to monitor (default: current branch)
- `monitor_mode` (optional): continuous, snapshot, or alert-only (default: snapshot)
- `integration_sync` (optional): Enable cross-pack integration sync (default: true)
- `alert_threshold` (optional): Alert on status changes (default: true)

### Expected Outputs

- Pipeline status summary with health assessment
- Job-level status breakdown with failure analysis
- Integration sync status (JIRA, parallel-dev)
- Actionable recommendations for pipeline issues

---

## Task Execution

### Phase 1: Pipeline Discovery and Authentication

```bash
# Import utilities for robust operations
source .bmad-core/utils/gitlab-commands.md
source .bmad-core/utils/gitlab-api-fallback.md

# Enable debug mode if requested
export GITLAB_DEBUG="${GITLAB_DEBUG:-false}"

# Verify GitLab CLI authentication
echo "🔐 Verifying GitLab CLI Authentication..."
glab auth status || {
  echo "❌ GitLab CLI not authenticated"
  echo "   Run: glab auth login"
  exit 1
}

# Validate GitLab permissions (new robust check)
echo "🔍 Validating GitLab permissions..."
validate_gitlab_permissions

# Initialize API fallback environment
gitlab_api_init || {
  echo "⚠️ API fallback initialization failed"
  echo "   Continuing with glab commands only"
}

# Auto-detect current context
CURRENT_BRANCH=${branch:-$(git branch --show-current)}
echo "📍 Monitoring branch: $CURRENT_BRANCH"

# Check for GitLab CI configuration
if [ ! -f ".gitlab-ci.yml" ] && [ ! -f "gitlab-ci.yml" ]; then
  echo "⚠️ No GitLab CI configuration detected"
  echo "   Looking for: .gitlab-ci.yml or gitlab-ci.yml"
fi
```

### Phase 2: Pipeline Status Assessment

```bash
# Get comprehensive pipeline data using robust fallback strategy
echo "🔍 Fetching pipeline data..."

# Use the robust pipeline info function from gitlab-commands.md
PIPELINE_DATA=$(get_pipeline_info_robust "$CURRENT_BRANCH")

if [ $? -eq 0 ] && [ "$PIPELINE_DATA" != "" ] && [ "$PIPELINE_DATA" != '{"error":"Limited GitLab access","status":"unknown","ref":"'$CURRENT_BRANCH'"}' ]; then
  # Extract key metrics using ci-status-parser utility
  source .bmad-core/utils/ci-status-parser.md

  echo "📊 Pipeline Status Summary:"
  echo "=========================="
  parse_pipeline_summary "$CURRENT_BRANCH" "standard"

  echo ""
  echo "📋 Job Status Breakdown:"
  echo "========================"
  parse_job_summary "$CURRENT_BRANCH" "standard"

  # Analyze pipeline health using pipeline-analyzer utility
  echo ""
  echo "🏥 Pipeline Health Analysis:"
  echo "============================"
  source .bmad-core/utils/pipeline-analyzer.md
  calculate_pipeline_health "$CURRENT_BRANCH"

else
  # Fallback: Try API directly if glab fails
  echo "⚠️ Limited GitLab access detected, trying API fallback..."

  if [[ -n "$GITLAB_TOKEN" ]] && [[ -n "$CI_PROJECT_ID" ]]; then
    PIPELINE_DATA=$(gitlab_get_latest_pipeline "$CURRENT_BRANCH")

    if [[ -n "$PIPELINE_DATA" ]] && [[ "$PIPELINE_DATA" != "null" ]]; then
      echo "✅ Retrieved pipeline data via API"
      echo "📊 Pipeline Status: $(echo "$PIPELINE_DATA" | jq -r '.status // "unknown"')"
      echo "🆔 Pipeline ID: $(echo "$PIPELINE_DATA" | jq -r '.id // "N/A"')"
      echo "🌿 Branch: $CURRENT_BRANCH"
      echo "🔗 URL: $(echo "$PIPELINE_DATA" | jq -r '.web_url // "N/A"')"
    else
      echo "❌ No pipeline data available for branch: $CURRENT_BRANCH"
      echo "   Possible causes:"
      echo "   - No pipeline has been triggered for this branch"
      echo "   - Branch does not exist on remote"
      echo "   - GitLab API permissions issue"
      exit 1
    fi
  else
    echo "❌ Cannot access pipeline data"
    echo "   Both glab and API methods failed"
    echo "   Please check:"
    echo "   - GitLab authentication (glab auth login)"
    echo "   - GITLAB_TOKEN environment variable"
    echo "   - Project permissions"
    exit 1
  fi
fi
```

### Phase 3: Failure Analysis (if applicable)

```bash
# Check for failed jobs and provide detailed analysis
PIPELINE_ID=$(echo "$PIPELINE_DATA" | jq -r '.id // ""')

if [[ -n "$PIPELINE_ID" ]]; then
  # Get failed jobs with enhanced error handling
  echo "🔍 Checking for failed jobs..."

  # Method 1: Try to get from pipeline data
  FAILED_JOBS=$(echo "$PIPELINE_DATA" | jq -r '.jobs[] | select(.status == "failed") | "\(.id):\(.name)"' 2>/dev/null)

  # Method 2: If no jobs in pipeline data, use API
  if [[ -z "$FAILED_JOBS" ]] && [[ -n "$GITLAB_TOKEN" ]]; then
    debug_log "No jobs in pipeline data, trying API method"
    FAILED_JOBS=$(gitlab_get_failed_job_logs "$PIPELINE_ID" | grep "^=== Failed Job:" | sed 's/=== Failed Job: //' | sed 's/ (ID: /:/g' | sed 's/)$//')
  fi

  if [ -n "$FAILED_JOBS" ]; then
    echo ""
    echo "🚨 Failure Analysis:"
    echo "==================="

    analyze_failure_patterns "$CURRENT_BRANCH"

    echo ""
    echo "📋 Failed Job Details:"
    while IFS=: read -r job_id job_name; do
      echo ""
      echo "--- $job_name (ID: $job_id) ---"

      # Use the robust job log retrieval function
      echo "📝 Recent logs:"

      # Try enhanced method from gitlab-commands.md
      if command -v get_job_logs >/dev/null 2>&1; then
        get_job_logs "$PIPELINE_ID" "$job_name" 2>&1 | tail -50 || {
          # Fallback to direct trace with numeric ID
          debug_log "get_job_logs failed, trying direct trace with ID: $job_id"
          glab ci trace "$job_id" 2>&1 | tail -50 || {
            # Final fallback to API
            debug_log "glab trace failed, trying API method"
            gitlab_get_job_trace "$job_id" 2>/dev/null | tail -50 || echo "   ❌ No logs available"
          }
        }
      else
        # Direct trace with numeric ID (discovered pattern)
        glab ci trace "$job_id" 2>&1 | tail -50 || echo "   ❌ No logs available"
      fi

      echo ""
      echo "🔗 Full logs: Use 'glab ci trace $job_id' or check GitLab web interface"
    done <<< "$FAILED_JOBS"
  else
    echo "✅ No failed jobs found"
  fi
else
  echo "⚠️ Cannot analyze failures - no pipeline ID available"
fi
```

### Phase 4: Cross-Pack Integration Sync

```bash
# Sync with other expansion packs if enabled
if [ "$integration_sync" = "true" ]; then
  echo ""
  echo "🔗 Cross-Pack Integration Sync:"
  echo "==============================="

  # Use integration bridge utility
  source .bmad-core/utils/gitlab-integration-bridge.md

  # Detect available integrations
  detect_expansion_packs

  # Auto-detect integration context
  auto_detect_integration_context

  # Sync with JIRA if available
  if [ "$JIRA_INTEGRATION" = "true" ]; then
    echo ""
    echo "🎯 JIRA Integration Status:"
    sync_ci_status_to_jira "$CURRENT_BRANCH" false
  fi

  # Coordinate with parallel development if applicable
  if [ "$PARALLEL_DEV_INTEGRATION" = "true" ]; then
    echo ""
    echo "🔀 Parallel Development Coordination:"
    coordinate_parallel_ci
  fi
fi
```

### Phase 5: Continuous Monitoring (if requested)

```bash
# Continuous monitoring mode
if [ "$monitor_mode" = "continuous" ]; then
  echo ""
  echo "🔄 Entering continuous monitoring mode..."
  echo "   Press Ctrl+C to stop monitoring"
  echo ""

  while true; do
    CURRENT_STATUS=$(glab ci get --output json --branch "$CURRENT_BRANCH" 2>/dev/null | jq -r '.status // "unknown"')
    TIMESTAMP=$(date '+%H:%M:%S')

    case "$CURRENT_STATUS" in
      "running")
        echo "[$TIMESTAMP] 🔄 Pipeline running..."
        # Show job progress
        glab ci get --output json --branch "$CURRENT_BRANCH" 2>/dev/null | jq -r '.jobs[] | "  \(.name): \(.status)"' 2>/dev/null
        ;;
      "success")
        echo "[$TIMESTAMP] ✅ Pipeline completed successfully!"
        if [ "$integration_sync" = "true" ]; then
          echo "   Triggering integration sync..."
          sync_ci_status_to_jira "$CURRENT_BRANCH" false
        fi
        break
        ;;
      "failed")
        echo "[$TIMESTAMP] ❌ Pipeline failed!"
        echo "   Running failure analysis..."
        analyze_failure_patterns "$CURRENT_BRANCH"
        break
        ;;
      *)
        echo "[$TIMESTAMP] ℹ️ Status: $CURRENT_STATUS"
        ;;
    esac

    sleep 30
  done
fi
```

### Phase 6: Actionable Recommendations

```bash
# Provide intelligent recommendations based on pipeline status
echo ""
echo "🎯 Actionable Recommendations:"
echo "=============================="

CURRENT_STATUS=$(echo "$PIPELINE_DATA" | jq -r '.status')

case "$CURRENT_STATUS" in
  "success")
    echo "✅ Pipeline Status: SUCCESS"
    echo ""
    echo "Recommended Actions:"
    echo "  1. 🔀 Ready for merge/deployment"
    echo "  2. 🎯 Update JIRA issues (if applicable)"
    echo "  3. 📋 Consider code review if not done"
    echo "  4. 🚀 Proceed with deployment workflow"

    if [ "$PARALLEL_DEV_INTEGRATION" = "true" ]; then
      echo "  5. 🔀 Coordinate with other parallel branches"
    fi
    ;;

  "failed")
    echo "❌ Pipeline Status: FAILED"
    echo ""
    echo "Recommended Actions:"
    echo "  1. 🔍 Review failed job logs above"
    echo "  2. 🔧 Fix identified issues"
    echo "  3. 📝 Update commit with fixes"
    echo "  4. 🔄 Push to trigger new pipeline"

    if [ "$JIRA_INTEGRATION" = "true" ]; then
      echo "  5. 🎯 Update JIRA with failure details"
    fi
    ;;

  "running")
    echo "🔄 Pipeline Status: RUNNING"
    echo ""
    echo "Monitoring Actions:"
    echo "  1. ⏱️ Monitor progress (use --monitor-mode continuous)"
    echo "  2. 👀 Watch for failures or completion"
    echo "  3. 🔄 Be ready for post-completion actions"
    ;;

  *)
    echo "ℹ️ Pipeline Status: $CURRENT_STATUS"
    echo ""
    echo "General Actions:"
    echo "  1. 🔍 Investigate unusual status"
    echo "  2. 📋 Check GitLab pipeline page"
    echo "  3. 🔄 Consider re-triggering if needed"
    ;;
esac
```

---

## Integration Hooks

### JIRA Integration Points

- Automatic issue status updates based on CI results
- Failure details added to JIRA comments
- Success notifications for deployment tracking

### Parallel Development Integration Points

- Multi-worktree CI status coordination
- Merge readiness assessment across branches
- Aggregate CI health reporting

### Core BMAD Integration Points

- CI status context for development workflows
- Pipeline health gates for story completion
- Integration with architecture decision workflows

---

## Error Handling and Recovery

### Common Issues and Solutions

**Issue**: GitLab CLI authentication failure
**Solution**: Run `glab auth login` to re-authenticate

**Issue**: glab commands fail with 403/404 errors
**Solution**: Task now automatically falls back to direct API calls. Ensure GITLAB_TOKEN is set.

**Issue**: No pipeline data for branch
**Solution**:

- Check branch exists remotely and has commits that trigger CI
- Task will try multiple methods: glab ci get → glab ci list → direct API

**Issue**: Job log retrieval fails with job name
**Solution**:

- Task now uses numeric job IDs (discovered pattern)
- Automatically resolves job name to ID before trace
- Falls back to API if glab trace fails

**Issue**: Limited GitLab permissions (can list but not view details)
**Solution**:

- Task detects permission levels and uses appropriate fallbacks
- API methods often work when glab commands fail
- Set GITLAB_DEBUG=true for detailed troubleshooting

**Issue**: Integration sync fails
**Solution**: Verify other expansion packs are properly installed and configured

---

## Usage Examples

### Basic Pipeline Monitoring

```bash
# Monitor current branch
monitor-pipeline-status

# Monitor specific branch
monitor-pipeline-status --branch develop

# Snapshot mode without integration sync
monitor-pipeline-status --integration-sync false
```

### Continuous Monitoring

```bash
# Continuous monitoring with alerts
monitor-pipeline-status --monitor-mode continuous

# Alert-only mode (minimal output)
monitor-pipeline-status --monitor-mode alert-only
```

### Integration-Focused Monitoring

```bash
# Focus on JIRA integration
monitor-pipeline-status --integration-sync true

# Parallel development coordination
monitor-pipeline-status --monitor-mode continuous --integration-sync true
```

### Debug Mode for Troubleshooting

```bash
# Enable debug logging to troubleshoot command failures
GITLAB_DEBUG=true monitor-pipeline-status

# Debug with specific branch
GITLAB_DEBUG=true monitor-pipeline-status --branch develop

# Debug with API token override
GITLAB_TOKEN="your-token" GITLAB_DEBUG=true monitor-pipeline-status
```

---

## Success Criteria

- ✅ Successfully retrieves and displays pipeline status
- ✅ Provides intelligent failure analysis when applicable
- ✅ Integrates seamlessly with available expansion packs
- ✅ Delivers actionable recommendations
- ✅ Operates autonomously without user interaction prompts
- ✅ Handles errors gracefully with helpful guidance
- ✅ **NEW**: Implements cascading command strategies with automatic fallbacks
- ✅ **NEW**: Uses numeric job IDs for reliable log retrieval
- ✅ **NEW**: Falls back to direct API when glab commands fail
- ✅ **NEW**: Provides debug mode for troubleshooting permission issues

## Dependencies

- **GitLab CLI** (`glab`) with authentication
- **Utilities**:
  - ci-status-parser
  - pipeline-analyzer
  - gitlab-integration-bridge
  - **NEW**: gitlab-commands (enhanced with fallback strategies)
  - **NEW**: gitlab-api-fallback (direct API integration)
- **Optional**: JIRA integration, parallel-dev integration
- **Git repository** with GitLab remote configured
- **Environment Variables** (optional but recommended):
  - `GITLAB_TOKEN` - For API fallback when glab fails
  - `GITLAB_DEBUG` - For troubleshooting command failures
  - `CI_PROJECT_ID` - Auto-detected or manually set