@agentled/cli
Version:
CLI for Agentled — manage workflows, apps, and knowledge from the command line. Zero context-window cost for AI agents.
498 lines (422 loc) • 21.6 kB
Markdown
# 09 — Reports, sharing, and knowledge-graph storage: closing the loop
**Problem**: Agents produce analysis as free-form prose in AI step output, then either never display it or drop it into a raw JSON view. Results aren't shareable with stakeholders, aren't comparable across runs, and don't feed back into the Knowledge Graph for trend analysis or future scoring calibration.
**Why it fails silently**: Without a `renderer`, an AI step's structured output is unreadable in the workspace UI. Without a `share` step, the output has no URL that can be sent to a stakeholder. Without a `knowledgeSync` step, each run's data is discarded — the second run can't learn from the first, and KPIs have no history to trend against.
---
## The three-part closing loop
```
... upstream steps → AI report step (with Config renderer)
→ [optional] share step (public URL)
→ [optional] composed email step (delivery)
→ knowledgeSync (persist to KG for trending)
→ milestone
```
The three pieces are independent and optional, but they compound. A report with a renderer is readable. A report with a renderer + share step is forwardable. A report with all three + knowledgeSync is how KPI dashboards actually get built.
## 1. Report AI step with Config renderer
The Config renderer turns structured AI output into a KPI dashboard view. Key blocks: `kpiRow`, `markdown`, `table`, `signalList`.
```json
{
"id": "generate-report",
"type": "aiAction",
"name": "Generate Report",
"pipelineStepPrompt": {
"template": "Analyze the data and produce a structured report.\n\nData: {{steps.evaluate.items}}",
"responseStructure": {
"summary": "string — executive summary",
"kpis": "object { total: number, qualified: number, avgScore: number }",
"items": "array of { name, score, decision }",
"insights": "array of strings"
}
},
"renderer": {
"type": "Config",
"config": {
"layout": {
"title": "Run Report",
"blocks": [
{ "blockType": "kpiRow", "kpis": [
{ "label": "Total", "valuePath": "kpis.total", "icon": "Hash" },
{ "label": "Qualified", "valuePath": "kpis.qualified", "icon": "CheckCircle" },
{ "label": "Avg Score", "valuePath": "kpis.avgScore", "format": "score" }
]},
{ "blockType": "markdown", "contentPath": "summary" },
{ "blockType": "table", "arrayPath": "items", "searchable": true, "columns": [
{ "header": "Name", "field": "name" },
{ "header": "Score", "field": "score", "display": "score", "sortable": true },
{ "header": "Decision", "field": "decision", "display": "badge" }
]},
{ "blockType": "signalList", "title": "Insights", "arrayPath": "insights", "variant": "signal" }
]
}
}
},
"creditCost": 10,
"next": { "stepId": "share-report" }
}
```
**Rule**: the `responseStructure` keys must match the renderer's `valuePath`/`arrayPath`/`contentPath` references. Drift between the two silently produces empty KPI tiles and blank tables.
## 2. Share step for a public URL
```json
{
"id": "share-report",
"type": "share",
"name": "Create Public Report Link",
"shareConfig": {
"outputSteps": ["generate-report"],
"expiresInDays": 30,
"visibility": "public"
},
"next": { "stepId": "send-report-email" }
}
```
Outputs `{ shareId, shareUrl, expiresAt }`. Downstream steps use `{{steps.share-report.shareUrl}}` to reference the public URL (e.g., in an email body).
**When to add a share step**:
- Anyone outside the workspace needs to see the report.
- The report should remain accessible after the execution's detail page expires.
- The workflow delivers the report via email and the body should link to the full report.
**When to skip it**: internal-only dashboards where viewers already have workspace access.
## 3. Knowledge-graph storage for trending
Without this, each run's data vanishes. With it, you can:
- Compare "today's MRR" against last month's.
- Calibrate future AI scoring on past outcomes (see `kg.retrieve-scoring-memory`).
- Build a timeline view of any metric.
```json
{
"id": "store-history",
"type": "knowledgeSync",
"name": "Store to KPI History",
"knowledgeSync": {
"source": { "stepId": "generate-report" },
"listKey": "kpi_history",
"fieldMapping": {
"mrr": "mrr",
"burn": "burn",
"runway_months": "runway_months",
"executedAt": "executedAt"
}
},
"next": { "stepId": "done" }
}
```
If you're inside a loop (scoring many items per run), set `source.resultsPath: "items"` so each loop iteration produces one KG row.
## Anti-patterns
**AI step with no renderer:**
```yaml
type: aiAction
responseStructure:
summary: string
kpis: { total, avgScore }
# ❌ No renderer — the workspace UI shows raw JSON. Nobody reads it.
```
**Share step without `outputSteps`:**
```yaml
type: share
shareConfig:
visibility: public
# ❌ Missing outputSteps — share URL renders nothing
```
**`knowledgeSync` without a clear schema:**
If `listKey` doesn't already exist, the KG creates an implicit schema from the first write. Subsequent writes with different field shapes fail to index. For trending, pre-create the list with a typed schema.
**Reusing `fieldMapping` values with source-side renames:**
`fieldMapping` keys are source field names (from the prior step's output), values are target field names (in the KG). Flipping them silently stores the wrong data.
```yaml
# ✅ Correct: source → target
fieldMapping:
mrr: monthly_revenue # steps.extract.mrr → row.monthly_revenue
burn: monthly_expenses
```
## Checklist for any "report" workflow
- [ ] `responseStructure` keys match every renderer `valuePath` / `arrayPath` / `contentPath`
- [ ] Renderer `blocks` cover at least one KPI + one tabular view
- [ ] Share step exists if report is forwardable to non-workspace users
- [ ] If delivered via email, the email template embeds `{{steps.share.shareUrl}}`
- [ ] `knowledgeSync` persists to a list with a typed schema (not implicit)
- [ ] `executedAt` (or a similar timestamp) is stored so rows can be trended
---
## Don't (anti-patterns AI authors hit constantly)
These four mistakes turn a report into a wall-of-text "looks AI-generated" output:
- ❌ **Wall of `markdown` blocks for a scored report.** A scored AI step produces a numeric verdict — surface it. Use `scoringHeader` + `dimensionScores` at the top, body markdown after.
- ❌ **`section` block with nested `blocks: [...]`.** The `section` renderer only accepts `fields: [{ name, label, display? }]`. Nested `blocks` arrays are silently dropped — every section that uses them renders as `null`. Use `grid` (which DOES accept `blocks[]`) or convert to a flat `markdown` block with a `title`.
- ❌ **Markdown block with `{{var}}` in `title`.** Markdown block titles are NOT template-resolved (see `MarkdownBlockRenderer.tsx:22`). Only `section` block titles are. Use a `section` block with `display: 'text'` if you need a runtime-resolved heading, or have the AI emit the resolved title as a separate response field and reference it via the section title.
- ❌ **Skipping `thresholds` on `scoringHeader` / `dimensionScores`.** Without thresholds the score chip renders gray and unreadable. Always wire threshold-to-color rules.
✅ **scoringHeader + dimensionScores + rubric table** for any 0–100 scored output.
✅ **`funnel` block** for any orchestrator digest with stage attrition.
✅ **`banner` block at the bottom** for the CTA (apply, contact, upgrade).
---
## Pattern A — Scored report (rubric + scoringHeader + dimensionScores)
Use case: any AI step that produces a 0–100 score plus per-dimension breakdown — startup scoring, lead scoring, GBP audits, candidate evaluation, fit reviews.
**Visual hierarchy:**
1. `scoringHeader` — big colored hero with score + decision label + identity badges
2. `dimensionScores` — per-rubric colored bars (each with their own thresholds)
3. `table` "Why these scores" with rationale per dimension
4. Body sections (markdown / section blocks)
5. `banner` CTA at the bottom
**Threshold convention:**
- Total score (0–100): `>= 70` emerald, `>= 40` amber, `< 40` rose.
- Dimension score (0–20): `>= 14` emerald, `>= 8` amber, `< 8` rose.
### Required `responseStructure` shape
```jsonc
{
"score_total": "REQUIRED integer 0-100",
"public_verdict": "REQUIRED short label e.g. 'Strong fit' | 'Promising' | 'Refine and resubmit'",
"rubric": "REQUIRED array of {dimension, label, score 0-20, max 20, rationale}, ordered to match dimensionScores",
"company_name": "REQUIRED string for scoringHeader.titlePath",
"founder_name": "REQUIRED string for identity badge",
"stage": "REQUIRED string for identity badge",
"themesObserved": "REQUIRED comma-separated string",
"headline": "6-10 word warm verdict",
"summary": "2-3 sentence executive summary",
// ...body fields referenced by markdown / section blocks
}
```
### Hard rules
1. **`rubric` array order MUST match `dimensions[]` order in dimensionScores.** `valuePath: "rubric.0.score"` means the rubric's first entry. Misalignment silently shows the wrong score next to each label.
2. **Sum of dimension scores SHOULD equal `score_total`.** State this explicitly in the prompt — without that constraint the LLM produces inconsistent numbers between hero and rubric.
3. **`decisionPath` should resolve to a SHORT public-friendly label.** Don't echo internal status values (`qualified` / `declined` / `dead`). Map them in the prompt: `qualified → 'Strong fit'`, `declined → 'Promising — refine'`, `dead → 'Refine and resubmit'`.
4. **Use `scoringHeader.identity.badges[]` for cover-card metadata** (founder, stage, themes, country, source) instead of an extra `kpiRow`. Avoids the disconnected "big card on top, separate text card below" feeling.
### Live worked example
Reference workflow: AngelHive Pitch Review step `founder_report` (`99a8f552-b822-40c3-855c-16d5bfa0fe1f`). Pull the current config:
```bash
agentled steps get 99a8f552-b822-40c3-855c-16d5bfa0fe1f founder_report --source live
```
Renderer config (verbatim, abbreviated for readability):
```jsonc
{
"type": "Config",
"config": {
"downloadPdf": true,
"layout": {
"title": "Your AngelHive Review",
"subtitle": "{{steps.save_submission.company_name}} — {{steps.save_submission.themes}}",
"blocks": [
// 1. Big colored hero — score 0-100, decision label, identity badges
{
"blockType": "scoringHeader",
"scorePath": "score_total",
"max": 100,
"decisionPath": "public_verdict",
"summaryPath": "headline",
"titlePath": "company_name",
"thresholds": [
{ "min": 70, "color": "emerald" },
{ "min": 40, "color": "amber" },
{ "min": 0, "color": "rose" }
],
"identity": {
"namePath": "company_name",
"badges": [
{ "label": "Founder", "valuePath": "founder_name" },
{ "label": "Stage", "valuePath": "stage" },
{ "label": "Themes", "valuePath": "themesObserved" }
]
}
},
// 2. Per-rubric colored bars — each dimension /20 with its own thresholds
{
"blockType": "dimensionScores",
"title": "Pitch readiness rubric",
"dimensions": [
{ "label": "Product clarity", "valuePath": "rubric.0.score", "max": 20,
"thresholds": [{"min":14,"color":"emerald"},{"min":8,"color":"amber"},{"min":0,"color":"rose"}] },
{ "label": "Market opportunity", "valuePath": "rubric.1.score", "max": 20, "thresholds": [/*…same…*/] },
{ "label": "Team strength", "valuePath": "rubric.2.score", "max": 20, "thresholds": [/*…same…*/] },
{ "label": "Traction signals", "valuePath": "rubric.3.score", "max": 20, "thresholds": [/*…same…*/] },
{ "label": "Pitch readiness", "valuePath": "rubric.4.score", "max": 20, "thresholds": [/*…same…*/] }
]
},
// 3. Rationale table — readable breakdown of WHY each score
{
"blockType": "table",
"title": "Why these scores",
"arrayPath": "rubric",
"columns": [
{ "field": "label", "header": "Dimension" },
{ "field": "score", "header": "Score", "display": "score", "sortable": true },
{ "field": "max", "header": "Max" },
{ "field": "rationale", "header": "Why this score" }
]
},
// 4. Body sections — section blocks (titles ARE template-resolved, so score can appear inline)
{ "blockType": "markdown", "title": "Our Read", "contentPath": "summary" },
{
"blockType": "grid", "columns": 2,
"blocks": [
{ "blockType": "section", "title": "Product · {{rubric.0.score}}/20",
"fields": [{ "name": "productAnalysis", "label": "", "display": "text" }] },
{ "blockType": "section", "title": "Market · {{rubric.1.score}}/20",
"fields": [{ "name": "marketAnalysis", "label": "", "display": "text" }] }
]
},
// …Team/Traction, Business Model/Go-to-Market grids…
// 5. Banner CTA at the bottom
{
"blockType": "banner",
"variant": "upsell",
"icon": "Sparkles",
"title": "Apply for an upcoming AngelHive Pitch Night",
"contentPath": "aboutAngelHive",
"actions": [
{ "label": "Apply for a Pitch Night",
"url": "https://angelhive.pynn.ai/pitch-nights",
"icon": "ExternalLink", "variant": "primary" }
]
}
]
}
}
}
```
### Variant: scored report with radar chart
For audits that compare a target against a benchmark + competitor average (e.g. SEO/GBP audits), add a `chart` block of `chartType: "radar"`. Reference workflow: Agwanet GBP audit step `score-target-3` and `generate-full-report` — uses `dimensionScores` with per-dimension `thresholds` PLUS a radar comparing `cible` / `meilleur concurrent` / `moyenne zone`:
```jsonc
{
"blockType": "chart",
"chartType": "radar",
"title": "Comparison by dimension",
"arrayPath": "radar_data",
"categoryField": "dimension",
"valueFields": [
{ "field": "cible", "label": "Target", "color": "#f43f5e" },
{ "field": "meilleur", "label": "Leader", "color": "#10b981" },
{ "field": "moyenne", "label": "Average", "color": "#f59e0b" }
]
}
```
Pulled live with:
```bash
agentled steps get <agwanet-workflow-id> score-target-3 --source live
```
---
## Pattern B — Funnel report (orchestrator digest)
Use case: orchestrator workflow that processes N items and produces a digest report — daily deal flow, weekly sourcing summary, batch outreach reports, pipeline health dashboards.
**Visual hierarchy:**
1. `funnel` block — stage-by-stage attrition with conversion percentages and bottleneck highlighting
2. `kpiRow` — totals (Total processed, Qualified, Contacted, Paid)
3. `banner` (variant `info` or `warning`) — operator focus / "this run" summary
4. `markdown` — executive summary
5. `list` (style `card`) or `table` — itemized rows
6. `signalList` (variant `risk`) — bottlenecks
7. `list` (style `numbered`) — recommended actions
### Funnel block schema
```jsonc
{
"blockType": "funnel",
"title": "Pitch Night Funnel",
"description": "Sourcing → Scheduled conversion",
"stages": [
{ "label": "Sourced", "valuePath": "report.funnel.sourced", "icon": "Search" },
{ "label": "Qualified", "valuePath": "report.funnel.qualified", "icon": "CheckCircle" },
{ "label": "Contacted", "valuePath": "report.funnel.contacted", "icon": "Mail" },
{ "label": "Paid", "valuePath": "report.funnel.paid", "icon": "CreditCard" },
{ "label": "Scheduled", "valuePath": "report.funnel.scheduled", "icon": "Calendar" }
],
"showAbsolute": true, // show raw counts next to bars
"showConversion": true, // show stage-to-stage % between bars
"emphasizeBottleneck": true // highlight the largest drop in red
}
```
The funnel block uses `stages: [{ label, valuePath, icon? }]` — NOT `arrayPath`. Each stage's count is resolved by walking `valuePath` against the AI step's output. Bottleneck detection is automatic: the largest stage-to-stage drop is highlighted rose; anti-funnel growth is highlighted emerald.
### Required `responseStructure` shape
```jsonc
{
"report": {
"title": "string",
"headline": "string — most urgent operator next action",
"summary": "string — 2 sentences",
"funnel": {
"sourced": "REQUIRED number, never null",
"qualified": "REQUIRED number, never null",
"contacted": "REQUIRED number, never null",
"paid": "REQUIRED number, never null",
"scheduled": "REQUIRED number, never null"
},
"weeklyDeltas": { "sourced": 0, "contacted": 0, "paid": 0 },
"thisRun": { "scored": 0, "qualified_total": 0, "to_contact": 0, "outreach_queued": 0 },
"bottlenecks": ["string"],
"recommendations": ["string"]
}
}
```
**Hard rule on null counts:** the funnel block renders `null` as a dash, which breaks the conversion math and the bottleneck detection. State this explicitly in the prompt: *"For every count field, output `0` if the underlying metric is null/blank/missing. Never emit null."* Verify the rule at the bottom of the prompt as a final-pass check.
### Live worked example
Reference workflow: AngelHive Daily Funnel step `generate_report` (`58bb623f-cbc2-4e5c-bcb4-2855ce64bb56`). Pull the current config:
```bash
agentled steps get 58bb623f-cbc2-4e5c-bcb4-2855ce64bb56 generate_report --source live
```
Renderer config (verbatim, abbreviated):
```jsonc
{
"type": "Config",
"config": {
"layout": {
"blocks": [
// 1. Funnel — stages with conversion + bottleneck
{
"blockType": "funnel",
"title": "Pitch Night Funnel",
"description": "Sourcing → Scheduled conversion",
"stages": [
{ "icon": "Search", "label": "Sourced", "valuePath": "report.funnel.sourced" },
{ "icon": "CheckCircle", "label": "Qualified", "valuePath": "report.funnel.qualified" },
{ "icon": "Mail", "label": "Contacted", "valuePath": "report.funnel.contacted" },
{ "icon": "CreditCard", "label": "Paid", "valuePath": "report.funnel.paid" },
{ "icon": "Calendar", "label": "Scheduled", "valuePath": "report.funnel.scheduled" }
],
"showAbsolute": true,
"showConversion": true,
"emphasizeBottleneck": true
},
// 2. KPI row — this-run + weekly deltas
{
"blockType": "kpiRow",
"kpis": [
{ "icon": "Sparkles", "label": "Scored this run", "valuePath": "report.thisRun.scored", "format": "number" },
{ "icon": "CheckCircle", "label": "Outreach queued", "valuePath": "report.thisRun.outreach_queued", "format": "number" },
{ "icon": "TrendingUp", "label": "Sourced 7d", "valuePath": "report.weeklyDeltas.sourced", "format": "number" },
{ "icon": "Mail", "label": "Contacted 7d", "valuePath": "report.weeklyDeltas.contacted", "format": "number" },
{ "icon": "CreditCard", "label": "Paid 7d", "valuePath": "report.weeklyDeltas.paid", "format": "number" }
]
},
// 3. Banners — this-run summary (info) + operator focus (warning)
{
"blockType": "banner", "variant": "info", "icon": "Activity",
"title": "This run", "contentPath": "report.thisRunSummary"
},
{
"blockType": "banner", "variant": "warning", "icon": "TriangleAlert",
"title": "Operator focus", "contentPath": "report.headline"
},
// 4. Executive summary
{ "blockType": "markdown", "title": "Executive Summary", "contentPath": "report.summary" },
// 5. Itemized rows (card-list style for variable-length items)
{
"blockType": "list", "style": "card",
"title": "Upcoming Editions",
"arrayPath": "report.upcomingEditions",
"itemFields": [ /* per-item fields with thresholds + display hints */ ]
},
// 6. Side-by-side: source breakdown + bottlenecks
{
"blockType": "grid", "columns": 2,
"blocks": [
{ "blockType": "table", "title": "Source Breakdown", "arrayPath": "report.sourceBreakdown",
"columns": [
{ "field": "source", "header": "Source" },
{ "field": "count", "header": "Count", "display": "threshold",
"thresholds": { "red": 0, "orange": 3, "green": 8 }, "sortable": true }
]
},
{ "blockType": "signalList", "variant": "risk",
"title": "Bottlenecks", "arrayPath": "report.bottlenecks" }
]
},
// 7. Recommendations — numbered list at the bottom
{
"blockType": "list", "style": "numbered",
"title": "Recommended Actions",
"arrayPath": "report.recommendations"
}
]
}
}
}
```