agentic-data-stack-community
Version:
AI Agentic Data Stack Framework - Community Edition. Open source data engineering framework with 4 core agents, essential templates, and 3-dimensional quality validation.
298 lines (268 loc) • 11.5 kB
YAML
workflow:
id: analytics-workflow
name: Analytics Pipeline Development
description: >-
End-to-end workflow for developing analytics solutions from business requirements
through dashboard deployment. Supports descriptive, diagnostic, predictive, and prescriptive analytics.
type: greenfield
project_types:
- business-intelligence
- operational-analytics
- customer-analytics
- financial-reporting
- performance-dashboards
sequence:
- step: business_requirements_analysis
agent: data-analyst
action: analyze-business-requirements
creates: analytics-requirements.md
duration: 1-2 days
notes: |
Conduct comprehensive business requirements analysis:
- Stakeholder interviews and needs assessment
- Key performance indicators (KPI) definition
- Success metrics and measurement criteria
- Reporting and dashboard requirements
- User personas and access patterns
SAVE OUTPUT: Copy final analytics-requirements.md to your project's docs/ folder.
- step: data_requirements_specification
agent: data-product-manager
action: create-analytics-data-contract
creates: analytics-data-contract.md
requires: analytics-requirements.md
duration: 1 day
notes: |
Define data requirements for analytics:
- Required data sources and datasets
- Data granularity and aggregation needs
- Historical data requirements and retention
- Data quality requirements for analytics
- Real-time vs batch processing requirements
SAVE OUTPUT: Copy final analytics-data-contract.md to your project's docs/ folder.
- step: analytical_design
agent: data-scientist
action: design-analytical-approach
creates: analytical-design.md
requires: [analytics-requirements.md, analytics-data-contract.md]
duration: 1-2 days
notes: |
Design analytical methodology and approach:
- Statistical methods and analytical techniques
- Model selection and validation strategies
- Feature engineering and data preparation
- Hypothesis testing and validation frameworks
- Performance metrics and evaluation criteria
- step: data_architecture_design
agent: data-architect
action: design-analytics-architecture
creates: analytics-architecture.md
requires: [analytics-data-contract.md, analytical-design.md]
duration: 1-2 days
notes: |
Design technical architecture for analytics:
- Data modeling for analytics (dimensional, OLAP)
- Processing architecture (batch, streaming, hybrid)
- Storage optimization for analytical workloads
- Query performance optimization strategies
- Scalability and resource planning
SAVE OUTPUT: Copy final analytics-architecture.md to your project's docs/ folder.
- step: user_experience_design
agent: data-experience-designer
action: design-analytics-interface
creates: analytics-ux-design.md
requires: analytics-requirements.md
duration: 1-2 days
notes: |
Design user experience for analytics consumption:
- Dashboard wireframes and user interface design
- Information architecture and navigation design
- Visualization selection and design principles
- Interactive features and drill-down capabilities
- Mobile and responsive design considerations
- step: data_model_implementation
agent: data-engineer
action: implement-analytics-data-model
creates: analytics-data-model
requires: [analytics-architecture.md, analytics-data-contract.md]
duration: 2-3 days
notes: |
Implement analytical data models:
- Dimensional model implementation (facts, dimensions)
- Data transformation and aggregation logic
- Calculated measures and KPI definitions
- Data lineage and metadata documentation
- Performance optimization and indexing
- step: analytical_implementation
agent: data-scientist
action: implement-analytics
creates: analytical-models
requires: [analytical-design.md, analytics-data-model]
duration: 2-3 days
notes: |
Implement analytical models and calculations:
- Statistical model implementation and training
- Business metric calculations and formulas
- Trend analysis and forecasting models
- Segmentation and classification algorithms
- Model validation and performance testing
- step: visualization_implementation
agent: data-experience-designer
action: implement-dashboards
creates: analytics-dashboards
requires: [analytics-ux-design.md, analytical-models]
duration: 2-3 days
notes: |
Implement dashboards and visualizations:
- Interactive dashboard development
- Chart and visualization implementation
- Drill-down and filtering capabilities
- Export and sharing functionality
- Performance optimization for large datasets
- step: quality_validation
agent: data-quality-engineer
action: validate-analytics-quality
validates: [analytics-data-model, analytical-models, analytics-dashboards]
duration: 1-2 days
notes: |
Comprehensive quality validation:
- Data accuracy validation in analytical models
- Calculation verification and testing
- Dashboard functionality and performance testing
- Cross-validation with known business metrics
- User acceptance testing coordination
- step: performance_optimization
agent: data-engineer
action: optimize-analytics-performance
requires: [analytics-data-model, analytics-dashboards]
duration: 1 day
notes: |
Optimize analytical performance:
- Query optimization and indexing strategies
- Caching implementation for frequently accessed data
- Resource allocation and scaling configuration
- Response time optimization for interactive features
- Load testing and capacity planning
- step: user_acceptance_testing
agent: data-analyst
action: conduct-analytics-uat
validates: [analytics-dashboards, analytical-models]
duration: 1-2 days
notes: |
Business user acceptance testing:
- Stakeholder validation of analytical outputs
- Dashboard usability and functionality testing
- Business logic verification and sign-off
- Training material development and delivery
- Feedback collection and issue resolution
- step: deployment_and_rollout
agent: data-engineer
action: deploy-analytics-solution
creates: production-analytics
requires: [analytics-dashboards, analytical-models]
duration: 1 day
notes: |
Deploy analytics solution to production:
- Production environment configuration
- Security and access control setup
- Monitoring and alerting configuration
- User provisioning and permission setup
- Rollout communication and training
- step: monitoring_and_governance
agent: data-governance-owner
action: setup-analytics-governance
creates: analytics-governance-framework
requires: production-analytics
duration: 0.5 day
notes: |
Establish ongoing governance and monitoring:
- Usage monitoring and analytics adoption tracking
- Data lineage documentation and maintenance
- Compliance monitoring and audit procedures
- Change management and version control
- Performance monitoring and optimization alerts
validation_gates:
- gate: requirements_validation
criteria:
- Business stakeholders approve analytical requirements
- Success metrics and KPIs are clearly defined
- Data requirements are feasible and available
- User experience requirements are documented
- gate: design_validation
criteria:
- Analytical approach is scientifically sound
- Architecture supports performance requirements
- User experience design meets usability standards
- Integration points are well-defined
- gate: implementation_validation
criteria:
- All analytical models meet accuracy requirements
- Dashboard performance meets response time SLAs
- Data quality validation passes all tests
- User acceptance testing completed successfully
- gate: deployment_validation
criteria:
- Production deployment completed without issues
- All security and access controls are functional
- Monitoring and alerting systems are operational
- User training and documentation are complete
success_criteria:
business:
- Key stakeholders actively use analytics solution
- Decision-making speed improves measurably
- Business insights lead to actionable outcomes
- ROI targets for analytics investment are met
technical:
- Dashboard response times meet SLA requirements
- Data accuracy meets business quality standards
- System availability exceeds 99% uptime
- User adoption reaches target thresholds
analytical:
- Model predictions meet accuracy thresholds
- Statistical significance is maintained
- Business metrics align with external benchmarks
- Trend analysis provides actionable insights
workflow_variants:
descriptive_analytics:
focus: Historical analysis and reporting
additional_steps:
- Historical data validation and cleansing
- Trend analysis and pattern identification
- Comparative analysis and benchmarking
typical_duration: 2-3 weeks
predictive_analytics:
focus: Forecasting and prediction models
additional_steps:
- Machine learning model development
- Model validation and backtesting
- Prediction accuracy monitoring
typical_duration: 4-6 weeks
prescriptive_analytics:
focus: Optimization and recommendation systems
additional_steps:
- Optimization algorithm development
- Scenario modeling and simulation
- Decision support system integration
typical_duration: 6-8 weeks
escalation_procedures:
- condition: Analytical accuracy below thresholds
action: Escalate to Data Scientist and Data Architect
timeline: Within 24 hours
- condition: Performance issues affecting user experience
action: Escalate to Data Engineer and Infrastructure Team
timeline: Within 4 hours
- condition: Business requirement conflicts or changes
action: Escalate to Data Product Manager and Business Sponsors
timeline: Within 1 business day
post_deployment_activities:
- activity: Usage analytics and adoption monitoring
frequency: Weekly for first month, then monthly
responsible: Data Product Manager
- activity: Model performance monitoring
frequency: Daily for statistical models, weekly for business metrics
responsible: Data Scientist
- activity: User feedback and satisfaction surveys
frequency: 30, 60, and 90 days post-deployment
responsible: Data Experience Designer
- activity: Business impact assessment
frequency: Quarterly
responsible: Data Product Manager and Business Stakeholders