UNPKG

@debugmcp/mcp-debugger

Version:

Run-time step-through debugging for LLM agents.

147 lines (115 loc) 5.64 kB
# Task 7: Fix E2E Test Timing Issues - Summary ## 🎯 Objective Fix timing issues in E2E tests by implementing proper event-based waiting mechanisms to replace fixed timeouts and race conditions. ## 📊 Current Status: Partially Complete ### ✅ Completed Items 1. **Event-Based Waiting Utilities Created** (`tests/e2e/test-event-utils.ts`) - `waitForSessionState()` - Polls session state until expected state is reached - `waitForStoppedEvent()` - Waits for session to enter paused state - `waitForContinuedEvent()` - Waits for session to enter running state - `waitForTerminatedEvent()` - Waits for session to enter stopped state - `waitForAnyState()` - Waits for any of multiple states - `executeAndWaitForState()` - Executes operation and waits for state change - `smartWaitAfterOperation()` - Handles common patterns intelligently - Event recording for debugging flaky tests 2. **Improved Error Handling** (`tests/e2e/smoke-test-utils.ts`) - Enhanced `callToolSafely()` to catch all MCP errors properly - Better error message handling for various error formats 3. **Server Validation Added** (`src/server.ts`) - Added `validateSession()` method to check session exists and is not closed - Applied validation to all debug operations - Fixed method signatures and missing implementations ### ❌ Remaining Issues 1. **Session Validation Not Working Properly** - Closed sessions (state='stopped') still allow operations like setting breakpoints - The validation throws errors but they're not propagating to the client correctly - This causes the error scenarios test to fail 2. **Full Debug Session Tests Timing Out** - Mock debugging tests are taking too long (>30s) - Variable retrieval returns unexpected values - Need to investigate why mock adapter isn't behaving as expected 3. **Path Translation Issues** - Container path translation test failing on Windows - Absolute paths are being displayed instead of relative paths ## 🔍 Root Cause Analysis ### Session Validation Issue The test revealed that when a session is closed: - Session state correctly changes to 'stopped' - Session remains in the session store (by design) - `validateSession()` is called but the error isn't reaching the client - Operations succeed when they should fail This suggests the error is being caught and suppressed somewhere in the call chain. ### Timing Issues The event-based utilities are working correctly, but: - Some operations complete faster than expected - Some operations never complete (infinite wait) - Mock adapter behavior differs from Python adapter ## 📝 Code Changes Made ### 1. Event Utilities (tests/e2e/test-event-utils.ts) ```typescript // Poll-based waiting with exponential backoff export async function waitForSessionState( client: Client, sessionId: string, expectedState: string, options: WaitOptions = {} ): Promise<boolean> // Smart waiting for common patterns export async function smartWaitAfterOperation( client: Client, sessionId: string, operation: 'step_over' | 'step_into' | 'step_out' | 'continue', options: WaitOptions = {} ): Promise<{ success: boolean; finalState?: string }> ``` ### 2. Error Handling (tests/e2e/smoke-test-utils.ts) ```typescript // Enhanced to catch all error types export async function callToolSafely( mcpClient: Client, toolName: string, args: Record<string, unknown> ): Promise<ParsedToolResult> ``` ### 3. Session Validation (src/server.ts) ```typescript private validateSession(sessionId: string): void { const session = this.sessionManager.getSession(sessionId); if (!session) { throw new McpError(McpErrorCode.InvalidParams, `Session not found: ${sessionId}`); } if (session.state === SessionState.STOPPED) { throw new McpError(McpErrorCode.InvalidRequest, `Session is closed: ${sessionId}`); } } ``` ## 🚧 Next Steps ### 1. Fix Session Validation - Investigate why MCP errors aren't propagating from server methods - Check if SessionManager is catching and suppressing errors - Ensure proper error handling in the MCP request handler ### 2. Fix Mock Adapter Timing - Review mock adapter's variable handling - Ensure proper event sequence for debugging operations - Add logging to track operation timing ### 3. Fix Path Translation - Review path normalization in breakpoint messages - Ensure consistent path handling across platforms ## 🎓 Lessons Learned 1. **Event-based waiting is essential** - Fixed timeouts are unreliable 2. **Error propagation is complex** - Multiple layers can suppress errors 3. **Mock adapters need careful design** - Must match real adapter behavior 4. **Cross-platform testing is critical** - Path handling varies by OS ## 💡 Recommendations 1. **Add comprehensive logging** to trace error propagation 2. **Create integration tests** for error scenarios at SessionManager level 3. **Standardize error handling** across all layers 4. **Document expected timing** for all debug operations ## 📊 Test Results Summary - **Adapter Switching**: ✅ Passing - **Container Path Translation**: ❌ Failing (path display issue) - **Error Scenarios**: ❌ Failing (session validation issue) - **Full Debug Session**: ❌ Failing (timeout issues) - **Smoke Tests**: ✅ Passing ## 🏁 Conclusion Task 7 successfully implemented event-based waiting utilities, which was the primary goal. However, the testing revealed deeper issues with session validation and mock adapter behavior that need to be addressed. The event utilities provide a solid foundation for reliable E2E testing once these underlying issues are resolved.