arela/RESEARCH/prompts/code-summarization-research-prompt.md

AI-powered CTO with multi-agent orchestration, code summarization, visual testing (web + mobile) for blazing fast development.

# Code Summarization Research Prompt

**Date:** 2025-11-15  
**Goal:** Research best practices for code summarization to reduce token usage while maintaining semantic meaning

---

## Research Questions

### 1. Code Summarization Techniques

**What are the state-of-the-art techniques for summarizing code?**

Research:
- Academic papers on code summarization
- Industry approaches (GitHub Copilot, Cursor, etc.)
- LLM-based vs rule-based approaches
- AST-based extraction vs full-text analysis

**Key questions:**
- What information should be extracted from code?
- How to maintain semantic meaning in summaries?
- What's the optimal summary length? (tokens vs information density)
- How to handle different code types (utilities, components, APIs, etc.)?

---

### 2. AST Extraction for Summarization

**What should we extract from the AST?**

Research:
- TypeScript AST structure (ts-morph, @typescript-eslint/parser)
- Key elements to extract:
  - Functions (name, parameters, return type, JSDoc)
  - Classes (name, methods, properties, inheritance)
  - Exports (what's public API)
  - Imports (dependencies)
  - Type definitions (interfaces, types)
  - Comments (JSDoc, inline comments)

**Key questions:**
- What's the minimal set of information needed?
- How to handle large files (>1000 lines)?
- How to prioritize information (public API > private helpers)?
- How to extract semantic relationships (calls, dependencies)?

---

### 3. LLM Prompts for Code Summarization

**What prompts work best for code summarization?**

Research:
- Prompt engineering for code understanding
- Few-shot vs zero-shot approaches
- Chain-of-thought for complex code
- Structured output formats (JSON, markdown)

**Example prompts to test:**
```
Prompt 1 (Brief):
"Summarize this code in 2-3 sentences. Focus on what it does, not how."

Prompt 2 (Structured):
"Analyze this code and provide:
1. Purpose (1 sentence)
2. Key functions/classes (list)
3. Dependencies (list)
4. Public API (list exports)"

Prompt 3 (Technical):
"Extract the technical summary:
- Main responsibility
- Input/output contracts
- Side effects
- Key algorithms/patterns used"
```

**Key questions:**
- Which format produces the most useful summaries?
- How to balance brevity vs completeness?
- How to ensure consistency across different code types?
- How to validate summary quality?

---

### 4. Token Efficiency

**How much token savings can we achieve?**

Research:
- Benchmarks: original code tokens vs summary tokens
- Target compression ratios (2x? 5x? 10x?)
- Trade-offs: compression vs information loss
- When to summarize vs when to include full code

**Key questions:**
- What's the optimal compression ratio?
- At what file size does summarization become worth it? (>500 lines? >1000 lines?)
- How to measure information retention?
- How to handle edge cases (generated code, minified code, etc.)?

---

### 5. Validation & Quality

**How to ensure summaries maintain semantic meaning?**

Research:
- Semantic similarity metrics (cosine similarity, BLEU, ROUGE)
- LLM-based validation (can model answer questions from summary?)
- Human evaluation criteria
- Automated testing approaches

**Key questions:**
- How to measure summary quality?
- What's acceptable information loss? (90% retention? 95%?)
- How to detect bad summaries automatically?
- How to improve summaries iteratively?

---

### 6. Existing Solutions

**What tools/libraries already exist?**

Research:
- GitHub Copilot's approach
- Cursor's context compression
- CodeBERT, GraphCodeBERT (code understanding models)
- Tree-sitter (universal AST parser)
- ts-morph (TypeScript AST manipulation)

**Key questions:**
- Can we use existing libraries?
- What can we learn from commercial tools?
- Are there open-source implementations?
- What are the licensing considerations?

---

### 7. Implementation Strategy

**How should we implement this in Arela?**

Research:
- Architecture: where does summarization fit in the pipeline?
- Caching: should we cache summaries?
- Incremental updates: how to handle file changes?
- Multi-language support: TypeScript, JavaScript, Python, Go?

**Key questions:**
- Should summarization be on-demand or pre-computed?
- How to invalidate cached summaries?
- How to handle different programming languages?
- What's the performance target? (<3s per file?)

---

## Success Criteria

A successful research outcome should provide:

1. **Clear technique** - Specific approach to code summarization
2. **Concrete prompts** - Tested prompts that produce good summaries
3. **Token benchmarks** - Expected compression ratios with examples
4. **Quality metrics** - How to measure summary quality
5. **Implementation plan** - Step-by-step guide to build it

---

## Deliverables

After research, create:

1. **Research report** - `RESEARCH/Code Summarization for Arela.md`
2. **Prompt library** - Collection of tested prompts
3. **Benchmarks** - Token savings on real code samples
4. **Implementation spec** - Technical design for `src/summarization/`

---

## Example Test Cases

Test summarization on these real files from Arela:

1. **Small utility** - `src/utils/update-checker.ts` (~140 lines)
2. **Medium component** - `src/meta-rag/classifier.ts` (~200 lines)
3. **Large module** - `src/memory/hexi-memory.ts` (~300 lines)
4. **Complex integration** - `src/context-router.ts` (~164 lines)

For each:
- Original token count
- Summary token count
- Compression ratio
- Information retention (can you answer questions from summary?)

---

## Research Sources

**Academic:**
- "A Survey of Automatic Source Code Summarization" (2020)
- "Deep Learning for Source Code Modeling and Generation" (2021)
- "CodeBERT: A Pre-Trained Model for Programming and Natural Languages" (2020)

**Industry:**
- GitHub Copilot documentation
- Cursor AI blog posts
- OpenAI Codex papers
- Anthropic Claude code understanding

**Tools:**
- ts-morph (TypeScript AST)
- tree-sitter (universal parser)
- @typescript-eslint/parser
- babel (JavaScript AST)

---

## Timeline

**Research phase:** 2-3 hours
**Report writing:** 1 hour
**Prompt testing:** 1 hour
**Total:** 4-5 hours

---

## Next Steps After Research

1. Review research report
2. Select best approach
3. Create implementation ticket
4. Build prototype
5. Test on real code
6. Integrate into Meta-RAG pipeline

---

**Ready to research!** 🔍