@cloudkinetix/bmad-enhanced/expansion-packs/ck-llm-agent-dev/tasks/prompt-testing-setup.md

Cloud-Kinetix enhanced fork of BMAD-METHOD - Breakthrough Method of Agile AI-driven Development with robust versioning and unified validation.

# Research-Driven Prompt Testing Setup

This task guides the research and setup of systematic prompt testing using current best practices and appropriate frameworks for the specific project context.

## Research-First Approach

[[LLM: Begin by researching current prompt testing methodologies and tools. Understand the project's specific testing requirements, constraints, and goals before selecting frameworks and approaches.]]

### 1. Research Testing Frameworks

**Research Current Options**:

- Investigate current prompt testing frameworks and tools (PromptFoo, Weights & Biases, LangSmith, custom solutions)
- Research evaluation methodologies appropriate for the specific use case
- Study testing patterns and best practices from recent AI projects
- Analyze cost, complexity, and maintenance trade-offs for different approaches

**Framework Selection Criteria**:

- Project requirements and constraints
- Team expertise and learning curve
- Integration capabilities with existing tools
- Cost and scalability considerations
- Community support and documentation quality

### 2. Research Implementation Patterns

**Configuration Research**:

- Research current provider configuration patterns and best practices
- Investigate multi-model testing strategies and their effectiveness
- Study security patterns for API key management and environment setup
- Analyze testing organization patterns (functional, safety, performance)

**Testing Methodology Research**:

- Research current evaluation metrics and their effectiveness for different use cases
- Investigate A/B testing approaches and statistical significance patterns
- Study automated vs manual evaluation trade-offs and hybrid approaches
- Analyze continuous integration patterns for prompt testing

### 3. Research-Based Implementation

[[LLM: Based on your research findings, implement the testing setup using current best practices. Focus on:

1. **Framework Setup**: Use researched approaches to set up the selected testing framework
2. **Test Design**: Apply research-backed methodologies for test case design and organization
3. **Evaluation Metrics**: Implement evaluation approaches based on current literature and best practices
4. **Integration**: Set up CI/CD integration using current patterns for automated testing
5. **Monitoring**: Implement performance tracking using researched observability patterns

Document your implementation choices and rationale based on the research conducted.]]

### 4. Continuous Improvement

**Research-Driven Evolution**:

- Regularly research new testing methodologies and tools
- Investigate emerging evaluation frameworks and metrics
- Study performance optimization techniques and apply relevant findings
- Analyze testing effectiveness and adapt based on latest best practices

---

**Note**: This task emphasizes research-driven testing setup over prescriptive tool configurations. Always research current best practices and adapt to your specific project context and requirements.