@awesomate/hosting-mcp/skill/awesomate-knowledge/references/ingestion-sources.md

Awesomate MCP server — lets Claude manage your Awesomate WordPress hosting, plan, limits, n8n automations, and build Node/static apps + databases

# Ingestion — what goes in, caps, cost and time

Source of truth: the platform contract (api-v1.md, frozen 2026-08-27). The
platform enforces the real numbers; these are for setting expectations and
estimating BEFORE asking approval.

## What can be ingested

| Kind | Via | Notes |
|---|---|---|
| Web page / blog post | `add {url}` | One page per URL; extraction skips pages under ~200 chars of real text |
| Whole site / blog | `add {sitemap, since?}` | sitemap.xml; default cap 500 pages per job — a bigger sitemap FAILS the job loudly rather than truncating silently. `since` skips entries with older `<lastmod>` |
| YouTube / hosted video, podcast audio | `add {url}` | **Not transcribed today**: a link to a recording is read as a web page (title and description). Pro and above only; below Pro, or in a trial, it is refused (403 `media_ingest`). To transcribe a recording, upload the file |
| PDF / book / document / markdown | hub upload page | Text layer used; scanned pages OCR'd automatically (`auto` mode, capped at 300 OCR pages per job — over that the job fails BEFORE any vision spend) |
| Video/audio FILES | hub upload page (≤ 100 MB) | Transcribed (Deepgram, language defaults to English). Pro and above, within the month's media hours. A longer recording: split or compress it and upload each part |
| Images | hub upload page | Vision-described, then indexed |
| Facebook export | hub upload page | The platform's facebook-export lane |
| FAQs | Markdown file via `awesomate_knowledge_upload` | One `##` per question, so each becomes its own citable section. **Never** a Q&A spreadsheet as it is: convert it to this file, so every answer stays word for word and citable. Full recipe: [faqs.md](faqs.md) |
| Spreadsheet (`.csv`, `.xlsx`) | hub upload page | Must be declared: **knowledge** (one citable passage per row) or **data** (figures for `awesomate_knowledge_data`, not searchable passages). The hub's upload page asks the owner. FAQs in a sheet: convert to Markdown instead, see [faqs.md](faqs.md) |

Local files go through `awesomate_knowledge_upload`, which streams them from
disk so their contents never enter the conversation (the owner can also use
the hub upload page). 100 MB per file.

## Job lifecycle

`add` returns a job id (202). Poll `awesomate_knowledge_sources
{action:'jobs'}` — statuses `queued → running → succeeded | failed`.
Re-adding identical content is cheap: unchanged pages and already-seen
media are skipped by content hash (`unchanged` / `transcript_cached` in the
job summary). Typical time-to-searchable: minutes for pages; media takes
longer (roughly the media's own duration as a worst case).

## Cost expectations (for the approval conversation)

- Text (pages, posts, books): cents to a few dollars for a typical corpus.
- Media is the big line: transcription ≈ $0.30/media-hour plus enrichment —
  a 200-hour channel is real money AND real quota (media-hours).
- Scanned PDFs needing OCR cost per page — the 300-page OCR cap fails the
  job before surprise spend.
- Everything counts against the plan allowance first
  ([quotas-and-packs.md](quotas-and-packs.md)); a `pack_required` response
  means the allowance is exhausted, and nothing was purchased or queued.

Always: estimate → state it plainly ("about X hours of video — that's ~$Y
and most of this month's media allowance") → explicit yes → queue.