UNPKG

major-ai-skills

Version:

Installable agentic skills / AI agent skills (SKILL.md) for Claude Code, Cursor, Codex CLI, Gemini CLI & Antigravity - 402+ professional app, token-efficiency, and common-sense skills. SEO/GEO ready.

135 lines (105 loc) 5.13 kB
--- name: ansi-strip-logger description: "Remove ANSI colors and terminal control sequences from logs before passing them into model context or structured reports." category: efficiency risk: safe source: self source_type: self date_added: "2026-08-26" tags: ["ansi-strip", "terminal-logs", "regex", "token-optimization", "pytest", "jest", "agent-runtime"] tools: ["claude", "cursor", "gemini", "codex", "lmstudio"] --- # ANSI Escape Sequence Stripping & Output Normalization ## Overview When CLI tools and test runners (Jest, Pytest, Webpack, Docker, Cargo) execute in a terminal, they emit rich **ANSI Escape Sequences** (e.g. `\x1b[32m`, `\x1b[0m`, `\033[2K`) to render colors, bold text, spinners, and cursor movements. LLM byte-pair tokenizers cannot parse escape codes semantically; instead, they fragment each escape string into **4 to 8 individual tokens** (*`\`*, *`x1b`*, *`[`*, *`32`*, *`m`*). A simple 50-line test run can bloat from 500 clean tokens to **over 2,500 noisy tokens**, degrading model reasoning and wasting context budget. The **ANSI Stripping Protocol** intercepts and sanitizes stdout/stderr streams before they are injected into agent context windows. --- ## Token Fragmentation from ANSI Escape Codes ``` ┌─────────────────────────────────────────────────────────────┐ ANSI Token Fragmentation Raw Terminal Output with Color Bytes: \x1b[32mPASS\x1b[0m \x1b[90msrc/auth.test.ts\x1b[0m Tokenizer splits into: [\, x1b, [, 32, m, PASS, \, x1b, [, 0, m, \, x1b, [, 90, m, src, /, auth, ., test, ...] Total: 38 Tokens for 1 Line! Sanitized Plain Text Output: PASS src/auth.test.ts Total: 5 Tokens (86.8% Reduction!) └─────────────────────────────────────────────────────────────┘ ``` --- ## The Industrial ANSI Stripping Regex To strip all standard 7-bit and 8-bit ANSI / VT100 escape sequences, cursor movements, and OSC operating system commands: ```python import re # Comprehensive ECMA-48 / ANSI Escape Stripper ANSI_REGEX = re.compile( r""" \x1B(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~]) # Standard CSI codes & SGR colors |\x1B\][0-9];[^\a\x1B]*[\a\x1B\\] # OSC hyperlinks & title sequences |\r(?!\n) # Standalone carriage returns (spinners) """, re.VERBOSE ) def strip_ansi(text: str) -> str: """Removes all ANSI color codes and control sequences from terminal output.""" return ANSI_REGEX.sub("", text) ``` --- ## TypeScript / Node.js Agent Middleware ```typescript /** * Strips ANSI codes from command execution buffers before sending to LLM. */ const ANSI_REGEX = /[\u001b\u009b][[()#;?]*(?:[0-9]{1,4}(?:;[0-9]{0,4})*)?[0-9A-ORZcf-nqry=><]/g; export function sanitizeTerminalOutput(rawOutput: string): string { return rawOutput .replace(ANSI_REGEX, '') .replace(/\r\n/g, '\n') .replace(/\r/g, '\n') .trim(); } ``` --- ## Native Environment Variable Suppressors Before running CLI commands in agent subprocesses, pass these environment variables to disable ANSI output at the source: ```bash # Set in agent execution runtime export NO_COLOR=1 # Standard supported by 100+ modern CLI tools export TERM=dumb # Signals non-interactive terminal (disables spinners) export FORCE_COLOR=0 # Disables chalk/color in Node.js export CI=true # Forces automated, non-interactive output ``` ### Example Subprocess Execution in Python: ```python import subprocess import os env = os.environ.copy() env.update({ "NO_COLOR": "1", "TERM": "dumb", "CI": "true", "FORCE_COLOR": "0" }) result = subprocess.run( ["pytest", "--tb=short"], capture_output=True, text=True, env=env ) clean_output = strip_ansi(result.stdout) ``` --- ## Benchmark Comparison | Test Runner | Raw Output Tokens | Sanitized Tokens | Token Reduction | | :--- | :--- | :--- | :--- | | **Jest (12 Test Suites)** | 3,420 tokens | 610 tokens | **82.2% Reduction** | | **Pytest (Verbose Mode)** | 2,180 tokens | 540 tokens | **75.2% Reduction** | | **Cargo Build Logs** | 1,890 tokens | 480 tokens | **74.6% Reduction** | --- ## Agent Operational Directive > **MANDATORY**: Any agent tool that captures terminal stdout/stderr (`run_command`, `exec_shell`, `bash`) MUST pass outputs through an ANSI sanitizer before appending the result to the conversation transcript.