@debugmcp/mcp-debugger
Version:
Run-time step-through debugging for LLM agents.
147 lines (115 loc) • 5.64 kB
Markdown
# Task 7: Fix E2E Test Timing Issues - Summary
## 🎯 Objective
Fix timing issues in E2E tests by implementing proper event-based waiting mechanisms to replace fixed timeouts and race conditions.
## 📊 Current Status: Partially Complete
### ✅ Completed Items
1. **Event-Based Waiting Utilities Created** (`tests/e2e/test-event-utils.ts`)
- `waitForSessionState()` - Polls session state until expected state is reached
- `waitForStoppedEvent()` - Waits for session to enter paused state
- `waitForContinuedEvent()` - Waits for session to enter running state
- `waitForTerminatedEvent()` - Waits for session to enter stopped state
- `waitForAnyState()` - Waits for any of multiple states
- `executeAndWaitForState()` - Executes operation and waits for state change
- `smartWaitAfterOperation()` - Handles common patterns intelligently
- Event recording for debugging flaky tests
2. **Improved Error Handling** (`tests/e2e/smoke-test-utils.ts`)
- Enhanced `callToolSafely()` to catch all MCP errors properly
- Better error message handling for various error formats
3. **Server Validation Added** (`src/server.ts`)
- Added `validateSession()` method to check session exists and is not closed
- Applied validation to all debug operations
- Fixed method signatures and missing implementations
### ❌ Remaining Issues
1. **Session Validation Not Working Properly**
- Closed sessions (state='stopped') still allow operations like setting breakpoints
- The validation throws errors but they're not propagating to the client correctly
- This causes the error scenarios test to fail
2. **Full Debug Session Tests Timing Out**
- Mock debugging tests are taking too long (>30s)
- Variable retrieval returns unexpected values
- Need to investigate why mock adapter isn't behaving as expected
3. **Path Translation Issues**
- Container path translation test failing on Windows
- Absolute paths are being displayed instead of relative paths
## 🔍 Root Cause Analysis
### Session Validation Issue
The test revealed that when a session is closed:
- Session state correctly changes to 'stopped'
- Session remains in the session store (by design)
- `validateSession()` is called but the error isn't reaching the client
- Operations succeed when they should fail
This suggests the error is being caught and suppressed somewhere in the call chain.
### Timing Issues
The event-based utilities are working correctly, but:
- Some operations complete faster than expected
- Some operations never complete (infinite wait)
- Mock adapter behavior differs from Python adapter
## 📝 Code Changes Made
### 1. Event Utilities (tests/e2e/test-event-utils.ts)
```typescript
// Poll-based waiting with exponential backoff
export async function waitForSessionState(
client: Client,
sessionId: string,
expectedState: string,
options: WaitOptions = {}
): Promise<boolean>
// Smart waiting for common patterns
export async function smartWaitAfterOperation(
client: Client,
sessionId: string,
operation: 'step_over' | 'step_into' | 'step_out' | 'continue',
options: WaitOptions = {}
): Promise<{ success: boolean; finalState?: string }>
```
### 2. Error Handling (tests/e2e/smoke-test-utils.ts)
```typescript
// Enhanced to catch all error types
export async function callToolSafely(
mcpClient: Client,
toolName: string,
args: Record<string, unknown>
): Promise<ParsedToolResult>
```
### 3. Session Validation (src/server.ts)
```typescript
private validateSession(sessionId: string): void {
const session = this.sessionManager.getSession(sessionId);
if (!session) {
throw new McpError(McpErrorCode.InvalidParams, `Session not found: ${sessionId}`);
}
if (session.state === SessionState.STOPPED) {
throw new McpError(McpErrorCode.InvalidRequest, `Session is closed: ${sessionId}`);
}
}
```
## 🚧 Next Steps
### 1. Fix Session Validation
- Investigate why MCP errors aren't propagating from server methods
- Check if SessionManager is catching and suppressing errors
- Ensure proper error handling in the MCP request handler
### 2. Fix Mock Adapter Timing
- Review mock adapter's variable handling
- Ensure proper event sequence for debugging operations
- Add logging to track operation timing
### 3. Fix Path Translation
- Review path normalization in breakpoint messages
- Ensure consistent path handling across platforms
## 🎓 Lessons Learned
1. **Event-based waiting is essential** - Fixed timeouts are unreliable
2. **Error propagation is complex** - Multiple layers can suppress errors
3. **Mock adapters need careful design** - Must match real adapter behavior
4. **Cross-platform testing is critical** - Path handling varies by OS
## 💡 Recommendations
1. **Add comprehensive logging** to trace error propagation
2. **Create integration tests** for error scenarios at SessionManager level
3. **Standardize error handling** across all layers
4. **Document expected timing** for all debug operations
## 📊 Test Results Summary
- **Adapter Switching**: ✅ Passing
- **Container Path Translation**: ❌ Failing (path display issue)
- **Error Scenarios**: ❌ Failing (session validation issue)
- **Full Debug Session**: ❌ Failing (timeout issues)
- **Smoke Tests**: ✅ Passing
## 🏁 Conclusion
Task 7 successfully implemented event-based waiting utilities, which was the primary goal. However, the testing revealed deeper issues with session validation and mock adapter behavior that need to be addressed. The event utilities provide a solid foundation for reliable E2E testing once these underlying issues are resolved.