voyageai-cli
Version:
CLI for Voyage AI embeddings, reranking, and MongoDB Atlas Vector Search
879 lines (860 loc) • 110 kB
JavaScript
'use strict';
// Gracefully handle missing picocolors (e.g., in packaged Electron app)
let pc;
try {
pc = require('picocolors');
} catch {
// Fallback: no-op functions that return input unchanged
pc = new Proxy({}, { get: () => (s) => s });
}
/**
* Map of concept key → explanation object.
* Each has: title, summary, content (formatted string), links, tryIt.
*/
const concepts = {
embeddings: {
title: 'Embeddings',
summary: 'What are vector embeddings?',
content: [
`${pc.cyan('Vector embeddings')} are numerical representations of text (or images) as arrays`,
`of floating-point numbers — typically 256 to 2048 dimensions. They capture the`,
`${pc.cyan('semantic meaning')} of the input, not just keywords.`,
``,
`When you embed text, a neural network reads the entire input and produces a`,
`fixed-size vector. Texts with similar meanings end up ${pc.cyan('close together')} in this`,
`high-dimensional space, even if they share no words at all.`,
``,
`${pc.bold('Why dimensions matter:')} Higher dimensions capture more nuance but cost more to`,
`store and search. Voyage 4 models default to 1024 and support only discrete`,
`dimensions (256, 512, 1024, 2048) via ${pc.cyan('Matryoshka representation learning')} — you can`,
`truncate embeddings to these specific sizes without retraining, trading accuracy`,
`for efficiency. Arbitrary dimensions like 768 or 384 are not supported.`,
``,
`${pc.bold('Input types:')} When embedding for retrieval, use ${pc.cyan('--input-type query')} for search`,
`queries and ${pc.cyan('--input-type document')} for corpus text. The model prepends different`,
`internal prompts for each, optimizing the embedding for asymmetric retrieval.`,
``,
`All Voyage 4 series models (voyage-4-large, voyage-4, voyage-4-lite) share the`,
`same embedding space — you can embed queries with one model and documents with`,
`another for cost optimization.`,
].join('\n'),
links: ['https://docs.vaicli.com/models/text-embeddings/'],
tryIt: [
'vai embed "hello world" --model voyage-4-large',
'vai embed --file document.txt --input-type document',
],
},
reranking: {
title: 'Reranking',
summary: 'Two-stage retrieval with rerankers',
content: [
`${pc.cyan('Reranking')} is the process of re-scoring a set of candidate documents against a`,
`query to improve precision. It's the "second stage" of two-stage retrieval.`,
``,
`${pc.bold('Why embeddings alone aren\'t enough:')} Embedding models encode queries and documents`,
`${pc.cyan('independently')} — each text gets its own vector without seeing the other. This is`,
`fast but can miss subtle relevance signals. A reranker uses ${pc.cyan('cross-attention')} — it`,
`reads the query and each document ${pc.cyan('together')}, producing a much more accurate`,
`relevance score.`,
``,
`${pc.bold('The two-stage pattern:')}`,
` ${pc.dim('1.')} Embedding search retrieves a broad set of candidates (high recall)`,
` ${pc.dim('2.')} Reranker re-scores and reorders them (high precision)`,
``,
`${pc.bold('Instruction-following:')} The ${pc.cyan('rerank-2.5')} model supports natural-language`,
`instructions in the query, like "Find documents about database performance, not`,
`pricing." This lets you guide relevance beyond keyword matching.`,
``,
`${pc.bold('When to skip reranking:')} If your embedding search already returns highly relevant`,
`results, or latency is critical and you can't afford the extra round-trip,`,
`single-stage retrieval may be sufficient.`,
].join('\n'),
links: ['https://docs.vaicli.com/models/rerankers/'],
tryIt: [
'vai rerank --query "database performance" --documents "MongoDB is fast" "Redis is cached"',
'vai rerank --query "query" --documents-file candidates.json --top-k 5',
],
},
'vector-search': {
title: 'Vector Search',
summary: 'MongoDB Atlas Vector Search',
content: [
`${pc.cyan('Vector search')} finds documents whose embeddings are closest to a query embedding.`,
`Instead of matching keywords, it matches ${pc.cyan('meaning')}.`,
``,
`${pc.bold('How it works in Atlas:')} MongoDB Atlas Vector Search uses the ${pc.cyan('$vectorSearch')}`,
`aggregation stage. Under the hood, it performs ${pc.cyan('Approximate Nearest Neighbor')}`,
`(ANN) search using a Hierarchical Navigable Small World (HNSW) graph index.`,
`ANN trades a tiny amount of accuracy for massive speed gains over brute-force.`,
``,
`${pc.bold('Similarity functions:')}`,
` ${pc.cyan('cosine')} — Measures direction, ignoring magnitude. Best default for text.`,
` ${pc.cyan('dotProduct')} — Like cosine but magnitude-sensitive. Use with normalized vectors.`,
` ${pc.cyan('euclidean')} — Measures straight-line distance. Better for some spatial data.`,
``,
`${pc.bold('Tuning numCandidates:')} This controls how many candidates the ANN index considers`,
`before returning the top results. Higher values improve recall but add latency.`,
`A good starting point is ${pc.cyan('20× your limit')} (e.g., numCandidates=200 for limit=10).`,
`MongoDB recommends at least 20× to increase accuracy and reduce discrepancies`,
`between exact and approximate nearest neighbor results.`,
``,
`${pc.bold('Pre-filters:')} You can filter documents ${pc.cyan('before')} vector search runs (e.g., by`,
`category, date, or tenant). Pre-filters narrow the search space efficiently.`,
].join('\n'),
links: ['https://www.mongodb.com/docs/atlas/atlas-vector-search/vector-search-stage/'],
tryIt: [
'vai search --query "cloud database" --db myapp --collection docs --field embedding',
'vai index create --db myapp --collection docs --field embedding --dimensions 1024',
],
},
rag: {
title: 'RAG (Retrieval-Augmented Generation)',
summary: 'Retrieval-Augmented Generation',
content: [
`${pc.cyan('RAG')} is a pattern that combines retrieval with LLM generation: instead of`,
`relying on the LLM's training data alone, you ${pc.cyan('retrieve')} relevant context from`,
`your own data and include it in the prompt.`,
``,
`${pc.bold('Why RAG beats fine-tuning for most use cases:')}`,
` ${pc.dim('•')} No retraining needed — just update your document store`,
` ${pc.dim('•')} Citations and sources are traceable`,
` ${pc.dim('•')} Works with any LLM (swap models freely)`,
` ${pc.dim('•')} Keeps proprietary data out of model weights`,
``,
`${pc.bold('The pattern:')}`,
` ${pc.cyan('1. Embed')} your corpus → store vectors in Atlas`,
` ${pc.cyan('2. Retrieve')} → embed the user's question, run $vectorSearch`,
` ${pc.cyan('3. Generate')} → pass retrieved documents + question to an LLM`,
``,
`${pc.bold('How reranking improves RAG:')} After retrieval, reranking re-scores the candidates`,
`so only the most relevant documents go into the LLM context window. This`,
`reduces noise, improves answer quality, and saves tokens. The pattern becomes:`,
` ${pc.dim('embed → retrieve (top-100) → rerank (top-5) → generate')}`,
``,
`RAG with Voyage AI embeddings and Atlas Vector Search is one of the most`,
`effective ways to build grounded, up-to-date AI applications.`,
].join('\n'),
links: ['https://docs.vaicli.com/tutorials/rag-voyageai-mongodb/'],
tryIt: [
'vai store --db myapp --collection docs --field embedding --text "your document"',
'vai search --query "your question" --db myapp --collection docs --field embedding',
],
},
'cosine-similarity': {
title: 'Cosine Similarity',
summary: 'Measuring vector distance',
content: [
`${pc.cyan('Cosine similarity')} measures the angle between two vectors, ignoring their`,
`magnitude (length). Two vectors pointing in the same direction have a cosine`,
`similarity of ${pc.cyan('1')}, perpendicular vectors score ${pc.cyan('0')}, and opposite vectors score`,
`${pc.cyan('-1')}.`,
``,
`${pc.bold('Why it\'s the default for text embeddings:')} Text embedding models typically`,
`produce ${pc.cyan('normalized vectors')} (unit length), so cosine similarity and dot product`,
`give identical rankings. Cosine is preferred because it's intuitive: it measures`,
`how similar the ${pc.cyan('direction')} (meaning) is, regardless of scale.`,
``,
`${pc.bold('Intuition:')} Think of two documents about "databases" — one is a paragraph, one`,
`is a full article. Their embeddings point in a similar direction (similar topic)`,
`even though one input is much longer. Cosine captures this.`,
``,
`${pc.bold('When to use alternatives:')}`,
` ${pc.cyan('dotProduct')} — Equivalent to cosine for normalized vectors. Slightly faster`,
` on some hardware. Use when you know vectors are unit-length.`,
` ${pc.cyan('euclidean')} — Measures straight-line distance. Can be better when magnitude`,
` carries meaning (e.g., term frequency vectors, spatial data).`,
``,
`For Voyage AI embeddings, ${pc.cyan('cosine')} is almost always the right choice.`,
].join('\n'),
links: ['https://www.mongodb.com/docs/atlas/atlas-vector-search/vector-search-stage/'],
tryIt: [
'vai embed "hello world" --model voyage-4-large',
'vai embed "hi there" --model voyage-4-large',
],
},
'two-stage-retrieval': {
title: 'Two-Stage Retrieval',
summary: 'The embed → search → rerank pattern',
content: [
`${pc.cyan('Two-stage retrieval')} is the standard pattern for high-quality semantic search:`,
`a fast first stage for ${pc.cyan('recall')}, then a precise second stage for ${pc.cyan('precision')}.`,
``,
`${pc.bold('Stage 1 — Embedding search (recall):')}`,
`Embed the query, run ANN search against your vector index, and retrieve a`,
`broad set of candidates (e.g., top 100). This is fast (milliseconds) because`,
`ANN indexes are optimized for throughput, not perfect accuracy.`,
``,
`${pc.bold('Stage 2 — Reranking (precision):')}`,
`Feed the query + candidates to a reranker model that reads each pair with`,
`${pc.cyan('cross-attention')}. The reranker produces fine-grained relevance scores and`,
`reorders the results. Return the top 5–10 to the user (or to an LLM for RAG).`,
``,
`${pc.bold('Why two stages?')} Embedding search is ${pc.cyan('fast but approximate')} — it encodes`,
`query and document independently. Reranking is ${pc.cyan('slow but precise')} — it reads`,
`them together. Combining both gives you speed ${pc.cyan('and')} accuracy.`,
``,
`${pc.bold('Typical numbers:')} top-100 → rerank → top-10. The reranker adds ~50–200ms`,
`of latency but dramatically improves result quality.`,
``,
`${pc.bold('When single-stage is fine:')} Simple use cases, low-stakes search, or when`,
`latency budgets are extremely tight (<50ms total).`,
].join('\n'),
links: ['https://docs.vaicli.com/models/rerankers/'],
tryIt: [
'vai search --query "your question" --db myapp --collection docs --field embedding',
'vai rerank --query "your question" --documents "doc1" "doc2" "doc3" --top-k 3',
],
},
'input-type': {
title: 'Input Type',
summary: 'Query vs document embedding types',
content: [
`The ${pc.cyan('input_type')} parameter tells the embedding model whether the text is a`,
`${pc.cyan('search query')} or a ${pc.cyan('document')} being indexed. This matters for retrieval quality.`,
``,
`${pc.bold('⚠ Do not omit this parameter for retrieval tasks.')} The official docs emphasize`,
`that omitting input_type degrades retrieval accuracy.`,
``,
`${pc.bold('How it works:')} Voyage AI models internally prepend a specific prompt prefix`,
`to your text based on input_type:`,
` ${pc.dim('• query →')} ${pc.cyan('"Represent the query for retrieving supporting documents: "')}`,
` ${pc.dim('• document →')} ${pc.cyan('"Represent the document for retrieval: "')}`,
``,
`These prefixes bias the embedding to be ${pc.cyan('asymmetric')} — query embeddings are`,
`optimized to find relevant documents, and document embeddings are optimized`,
`to be found by relevant queries.`,
``,
`${pc.bold('Asymmetric retrieval:')} Queries are typically short ("What is MongoDB?") while`,
`documents are long (paragraphs, pages). They have fundamentally different`,
`characteristics, so embedding them differently improves matching quality.`,
``,
`${pc.bold('When to use each:')}`,
` ${pc.cyan('query')} — When embedding a search query or question`,
` ${pc.cyan('document')} — When embedding text to be stored and searched later`,
` ${pc.dim('(omit)')} — Only for clustering, classification, or symmetric similarity`,
``,
`${pc.bold('Tip:')} Always use ${pc.cyan('--input-type document')} when running ${pc.cyan('vai store')} or`,
`${pc.cyan('vai ingest')}, and ${pc.cyan('--input-type query')} when running ${pc.cyan('vai search')}.`,
].join('\n'),
links: ['https://docs.vaicli.com/models/text-embeddings/'],
tryIt: [
'vai embed "What is MongoDB?" --input-type query',
'vai embed --file article.txt --input-type document',
],
},
models: {
title: 'Models',
summary: 'Choosing the right model',
content: [
`Voyage AI offers several model families through MongoDB Atlas, each optimized`,
`for different use cases.`,
``,
`${pc.bold('Voyage 4 Series')} ${pc.dim('(general-purpose text embeddings):')}`,
` ${pc.cyan('voyage-4-large')} — Best quality, 1024 dims (256–2048), $0.12/1M tokens`,
` ${pc.cyan('voyage-4')} — Balanced quality/cost, same dimensions, $0.06/1M tokens`,
` ${pc.cyan('voyage-4-lite')} — Lowest cost, same dimensions, $0.02/1M tokens`,
` All three share the ${pc.cyan('same embedding space')} — you can mix models (e.g., embed`,
` documents with voyage-4-lite, queries with voyage-4-large).`,
``,
`${pc.bold('Domain-Specific:')}`,
` ${pc.cyan('voyage-code-3')} — Optimized for code search and understanding`,
` ${pc.cyan('voyage-finance-2')} — Financial text (reports, filings, analysis)`,
` ${pc.cyan('voyage-law-2')} — Legal documents (contracts, case law, statutes)`,
``,
`${pc.bold('Multimodal:')}`,
` ${pc.cyan('voyage-multimodal-3.5')} — Embeds both text and images in the same space`,
``,
`${pc.bold('Rerankers:')}`,
` ${pc.cyan('rerank-2.5')} — Best reranking quality, instruction-following`,
` ${pc.cyan('rerank-2.5-lite')} — Faster, lower cost reranking`,
``,
`${pc.bold('How to choose:')} Start with ${pc.cyan('voyage-4')} for general use. Use domain models when`,
`your data is specialized. Add reranking when precision matters.`,
].join('\n'),
links: ['https://docs.vaicli.com/models/'],
tryIt: [
'vai models',
'vai models --type embedding',
'vai embed "hello" --model voyage-4-large --dimensions 512',
],
},
'api-keys': {
title: 'API Keys',
summary: 'Managing API keys in Atlas',
content: [
`To use Voyage AI models, you need a ${pc.cyan('Model API key')} from MongoDB Atlas. This is`,
`different from your Atlas API keys (which manage infrastructure).`,
``,
`${pc.bold('Where to create one:')}`,
` ${pc.dim('1.')} Log in to ${pc.cyan('MongoDB Atlas')} (cloud.mongodb.com)`,
` ${pc.dim('2.')} Navigate to ${pc.cyan('AI Models')} in the left sidebar`,
` ${pc.dim('3.')} Click ${pc.cyan('Create API Key')}`,
` ${pc.dim('4.')} Copy the key — it starts with ${pc.dim('pa-')} and is shown only once`,
``,
`${pc.bold('Key types:')}`,
` ${pc.cyan('Model API Key')} — Authenticates to ai.mongodb.com/v1/ (for vai)`,
` ${pc.dim('Atlas API Key')} — Authenticates to Atlas Admin API (for infrastructure)`,
` ${pc.dim('Connection String')} — Connects to your MongoDB cluster (MONGODB_URI)`,
``,
`${pc.bold('Rate limits and usage tiers:')}`,
` ${pc.cyan('Free tier')} — 200M tokens across most models (no credit card needed)`,
` Paid tiers scale with your Atlas plan`,
` Rate limits apply per key — check the Atlas dashboard for your current usage`,
``,
`${pc.bold('Security tips:')}`,
` ${pc.dim('•')} Never commit keys to git — use environment variables or ${pc.cyan('vai config set')}`,
` ${pc.dim('•')} Use ${pc.cyan('echo "key" | vai config set api-key --stdin')} to avoid shell history`,
` ${pc.dim('•')} Rotate keys periodically in the Atlas dashboard`,
].join('\n'),
links: ['https://docs.vaicli.com/management/api-keys/'],
tryIt: [
'vai config set api-key "your-key"',
'vai ping',
],
},
'api-access': {
title: 'API Access Methods',
summary: 'MongoDB Atlas vs. Voyage AI platform keys',
content: [
`There are ${pc.cyan('two ways')} to access Voyage AI's embedding and reranking models,`,
`each with its own API key type, endpoint, and trade-offs.`,
``,
`${pc.bold('Option 1 — MongoDB Atlas (recommended for vai)')}`,
` ${pc.cyan('Endpoint:')} https://ai.mongodb.com/v1/`,
` ${pc.cyan('Key prefix:')} pa- (Model API Key)`,
` ${pc.cyan('Get a key:')} Atlas → AI Models → Create API Key`,
``,
` ${pc.dim('•')} Integrated with your Atlas billing and usage dashboards`,
` ${pc.dim('•')} ${pc.cyan('Free tier:')} 200M tokens across most models (no credit card)`,
` ${pc.dim('•')} Seamless pairing with Atlas Vector Search (store + search + rerank)`,
` ${pc.dim('•')} Same models, same quality — just a different access point`,
` ${pc.dim('•')} This is what ${pc.cyan('vai')} uses by default (VOYAGE_API_KEY + ai.mongodb.com)`,
``,
`${pc.bold('Option 2 — Voyage AI Platform (direct)')}`,
` ${pc.cyan('Endpoint:')} https://api.voyageai.com/v1/`,
` ${pc.cyan('Key prefix:')} pa- (Voyage AI API Key)`,
` ${pc.cyan('Get a key:')} dash.voyageai.com → API Keys`,
``,
` ${pc.dim('•')} Billed separately through Voyage AI's own platform`,
` ${pc.dim('•')} Same models and API shape — compatible with vai if you override the base URL`,
` ${pc.dim('•')} Free tier: 200M free tokens (separate from Atlas free tier)`,
` ${pc.dim('•')} Useful if you don't use MongoDB or want separate billing`,
``,
`${pc.bold('Key differences at a glance:')}`,
` ${pc.dim('Feature')} ${pc.cyan('Atlas')} ${pc.cyan('Voyage AI Direct')}`,
` ${pc.dim('─────────────────────────────────────────────────────────────────')}`,
` Endpoint ai.mongodb.com/v1 api.voyageai.com/v1`,
` Billing Atlas account Voyage AI account`,
` Free tier 200M tokens 200M tokens`,
` Vector Search pairing Native ($vectorSearch) BYO integration`,
` Key management Atlas dashboard Voyage AI dashboard`,
``,
`${pc.bold('Which should you use?')} If you're already on MongoDB Atlas (or plan to use`,
`Atlas Vector Search), go with the Atlas endpoint — it's one bill, one dashboard,`,
`and ${pc.cyan('vai')} is pre-configured for it. If you only need embeddings without MongoDB,`,
`the Voyage AI platform works fine too.`,
``,
`${pc.bold('Switching endpoints in vai:')} The default base URL is ${pc.cyan('https://ai.mongodb.com/v1/')}.`,
`To use the Voyage AI platform directly, set:`,
` ${pc.dim('$')} ${pc.cyan('vai config set base-url https://api.voyageai.com/v1/')}`,
].join('\n'),
links: [
'https://docs.vaicli.com/management/api-keys/',
'https://docs.voyageai.com/docs/api-key-and-installation',
],
tryIt: [
'vai config set api-key "your-atlas-or-voyage-key"',
'vai config set base-url https://api.voyageai.com/v1/',
'vai ping',
],
},
'batch-processing': {
title: 'Batch Processing',
summary: 'Embedding large datasets efficiently',
content: [
`When embedding large datasets (thousands or millions of documents), efficient`,
`${pc.cyan('batching')} is essential for speed, cost, and reliability.`,
``,
`${pc.bold('The API accepts arrays:')} Voyage AI's embedding endpoint accepts up to`,
`${pc.cyan('1,000 texts per request')} and model-specific token limits per batch. Sending arrays`,
`instead of individual requests dramatically reduces overhead.`,
``,
`${pc.bold('Rate limits:')} The API enforces rate limits (requests/min and tokens/min).`,
`If you hit them, add delays between batches. The vai CLI handles basic`,
`batching automatically when using ${pc.cyan('vai store')} with JSONL input.`,
``,
`${pc.bold('JSONL format for vai store:')} Create a file with one JSON object per line:`,
` ${pc.dim('{"text": "First document...", "metadata": {"source": "docs"}}')}`,
` ${pc.dim('{"text": "Second document...", "metadata": {"source": "blog"}}')}`,
` Then: ${pc.cyan('vai store --db myapp --collection docs --field embedding --file data.jsonl')}`,
``,
`${pc.bold('Chunking strategies:')} For long documents, split into overlapping chunks`,
`(e.g., 512 tokens with 50-token overlap). Voyage 4 models support up to`,
`${pc.cyan('32K tokens')} per input, but shorter chunks often retrieve better.`,
``,
`${pc.bold('Token counting:')} Roughly ${pc.cyan('1 token ≈ 4 characters')} for English text. The API`,
`returns ${pc.cyan('usage.total_tokens')} in every response so you can track consumption.`,
``,
`${pc.bold('Tip:')} Start with a small test batch to validate your pipeline before`,
`processing the full corpus.`,
].join('\n'),
links: ['https://docs.vaicli.com/models/text-embeddings/'],
tryIt: [
'vai store --db myapp --collection docs --field embedding --file documents.jsonl',
'vai embed --file document.txt --input-type document',
],
},
quantization: {
title: 'Quantization & Flexible Dimensions',
summary: 'Reduce storage costs with lower-precision embeddings',
content: [
`${pc.cyan('Quantization')} reduces embedding precision from 32-bit floats to smaller`,
`representations, dramatically cutting storage and search costs with minimal`,
`quality loss. Combined with ${pc.cyan('Matryoshka dimensions')}, you can shrink vectors`,
`by up to ${pc.bold('128×')} (32× from binary × 4× from fewer dimensions).`,
``,
`${pc.bold('Output data types (--output-dtype):')}`,
``,
` ${pc.cyan('float')} 32 bits/dim 4 bytes/dim Baseline (default)`,
` ${pc.cyan('int8')} 8 bits/dim 1 byte/dim ${pc.green('4× smaller')} Signed: -128 to 127`,
` ${pc.cyan('uint8')} 8 bits/dim 1 byte/dim ${pc.green('4× smaller')} Unsigned: 0 to 255`,
` ${pc.cyan('binary')} 1 bit/dim 1/8 byte/dim ${pc.green('32× smaller')} Bit-packed int8 (offset binary)`,
` ${pc.cyan('ubinary')} 1 bit/dim 1/8 byte/dim ${pc.green('32× smaller')} Bit-packed uint8`,
``,
`${pc.bold('Storage math for 1M documents at 1024 dims:')}`,
` float: ${pc.dim('1M × 1024 × 4B')} = ${pc.cyan('4.0 GB')}`,
` int8: ${pc.dim('1M × 1024 × 1B')} = ${pc.cyan('1.0 GB')} (4× savings)`,
` binary: ${pc.dim('1M × 1024 / 8B')} = ${pc.cyan('128 MB')} (32× savings)`,
` ${pc.dim('+ reduced dimensions:')} 256-dim binary = ${pc.cyan('32 MB')} (128× savings)`,
``,
`${pc.bold('How binary quantization works:')} Each float value is converted to a single bit:`,
`positive values become 1, zero/negative become 0. Eight bits are packed into`,
`one byte. ${pc.cyan('binary')} uses offset binary (subtract 128) for signed int8 output;`,
`${pc.cyan('ubinary')} stores the raw unsigned uint8 value.`,
``,
`${pc.bold('Quality impact:')} Quantization-aware training minimizes degradation:`,
` ${pc.dim('•')} ${pc.cyan('int8/uint8')} — Typically <1% retrieval quality loss vs float`,
` ${pc.dim('•')} ${pc.cyan('binary/ubinary')} — ~2-5% quality loss; best paired with a reranker`,
` ${pc.dim('•')} Combining lower dimensions + quantization compounds the quality loss`,
``,
`${pc.bold('Matryoshka dimensions:')} Voyage 4 models produce ${pc.cyan('nested embeddings')} — the`,
`first 256 entries of a 1024-dim vector are themselves a valid 256-dim embedding.`,
`You can embed once at full dimension and truncate later without re-embedding.`,
`Supported values: 256, 512, 1024 (default), 2048.`,
``,
`${pc.bold('Which vector databases support quantized storage?')}`,
` ${pc.dim('•')} MongoDB Atlas Vector Search — float and int8`,
` ${pc.dim('•')} Milvus, Qdrant, Weaviate, Elasticsearch, Vespa — float, int8, binary`,
``,
`${pc.bold('Decision framework:')}`,
` 1. Start with ${pc.cyan('float')} at default dimensions — measure your baseline`,
` 2. Try ${pc.cyan('int8')} — if quality holds, you get 4× storage savings for free`,
` 3. If storage is critical, try ${pc.cyan('binary')} + reranker for 32× savings`,
` 4. Reduce dimensions (1024→256) for another 4× on top of quantization`,
` 5. Use ${pc.cyan('vai benchmark quantization')} to measure the tradeoffs on your data`,
].join('\n'),
links: [
'https://docs.voyageai.com/docs/flexible-dimensions-and-quantization',
'https://docs.vaicli.com/models/text-embeddings/',
],
tryIt: [
'vai embed "hello world" --output-dtype int8',
'vai embed "hello world" --output-dtype binary --dimensions 256',
'vai benchmark quantization --model voyage-4-large',
],
},
benchmarking: {
title: 'Benchmarking & Model Selection',
summary: 'How to choose the right model for your use case',
content: [
`Choosing the right embedding or reranking model depends on your priorities:`,
`${pc.cyan('latency')}, ${pc.cyan('accuracy')}, ${pc.cyan('cost')}, or a balance of all three.`,
``,
`${pc.bold('vai benchmark embed')} — Compare embedding models head-to-head:`,
` Measures avg/p50/p95 latency, token usage, and cost per model.`,
` ${pc.dim('vai benchmark embed --models voyage-4-large,voyage-4,voyage-4-lite --rounds 5')}`,
``,
`${pc.bold('vai benchmark similarity')} — Test ranking quality on your data:`,
` Embeds a query + corpus with each model, shows side-by-side top-K rankings.`,
` If models agree on the top results, the cheaper one is likely sufficient.`,
` ${pc.dim('vai benchmark similarity --query "your query" --file corpus.txt')}`,
``,
`${pc.bold('vai benchmark rerank')} — Compare reranking models:`,
` Measures latency and shows how models order the same documents.`,
` ${pc.dim('vai benchmark rerank --query "your query" --documents-file docs.json')}`,
``,
`${pc.bold('vai benchmark cost')} — Project monthly costs at scale:`,
` Shows estimated cost for each model at different daily query volumes.`,
` ${pc.dim('vai benchmark cost --tokens 500 --volumes 100,1000,10000,100000')}`,
``,
`${pc.bold('vai benchmark batch')} — Find optimal batch size for ingestion:`,
` Measures throughput (texts/sec) at different batch sizes.`,
` ${pc.dim('vai benchmark batch --batch-sizes 1,5,10,25,50 --rounds 3')}`,
``,
`${pc.bold('vai benchmark quantization')} — Compare output dtypes for storage savings:`,
` Embeds the same corpus with float, int8, and binary, measures ranking quality`,
` degradation vs storage savings. Helps you decide if quantization works for your data.`,
` ${pc.dim('vai benchmark quantization --model voyage-4-large --dtypes float,int8,ubinary')}`,
``,
`${pc.bold('Decision framework:')}`,
` 1. Run ${pc.cyan('benchmark cost')} to eliminate models outside your budget`,
` 2. Run ${pc.cyan('benchmark embed')} to compare latency of affordable models`,
` 3. Run ${pc.cyan('benchmark similarity')} with your actual data to compare quality`,
` 4. Run ${pc.cyan('benchmark quantization')} to see if int8/binary preserves your ranking`,
` 5. If quality is similar, pick the cheaper/faster model + smallest viable dtype`,
` 6. Use ${pc.cyan('--save')} to track results over time as your data evolves`,
].join('\n'),
links: ['https://docs.vaicli.com/models/text-embeddings/'],
tryIt: [
'vai benchmark embed --rounds 3',
'vai benchmark cost',
'vai benchmark similarity --query "your search query" --file your-docs.txt',
],
},
'mixture-of-experts': {
title: 'Mixture-of-Experts (MoE) Architecture',
summary: 'How voyage-4-large achieves SOTA quality at 40% lower cost',
content: [
`${pc.cyan('Mixture-of-Experts (MoE)')} is a neural network architecture where multiple`,
`specialized sub-networks ("experts") share a single model. A learned ${pc.cyan('router')}`,
`selects which experts activate for each input — typically 2-4 out of 8-64 total.`,
``,
`${pc.bold('Why MoE matters for embeddings:')}`,
` ${pc.dim('•')} ${pc.cyan('Higher capacity, lower cost')} — the model has more total parameters`,
` (knowledge) but only activates a fraction per input, keeping inference fast`,
` ${pc.dim('•')} ${pc.cyan('Specialization')} — different experts learn different domains (code,`,
` legal, medical) without interfering with each other`,
` ${pc.dim('•')} ${pc.cyan('State-of-the-art quality')} — voyage-4-large beats all competitors on`,
` RTEB benchmarks while costing 40% less than comparable dense models`,
``,
`${pc.bold('voyage-4-large')} is the ${pc.cyan('first production-grade embedding model')} to use MoE.`,
`Previous MoE successes (Mixtral, Switch Transformer) were language models —`,
`applying MoE to embedding models required solving alignment across the shared`,
`embedding space, which is what makes the Voyage 4 family unique.`,
``,
`${pc.bold('Dense vs MoE:')}`,
` ${pc.dim('Dense (voyage-4, voyage-4-lite):')} Every parameter is used for every input.`,
` Simpler, predictable latency, lower total parameter count.`,
` ${pc.dim('MoE (voyage-4-large):')} Sparse activation — more total parameters, but each`,
` input only uses a subset. Higher quality ceiling, similar serving cost.`,
``,
`${pc.bold('In practice:')} You don't need to do anything special to use MoE — the API`,
`interface is identical. The architecture difference shows up in quality and cost:`,
` ${pc.dim('•')} voyage-4-large: $0.12/1M tokens, best quality via MoE architecture`,
` ${pc.dim('•')} 40% cheaper than comparable dense models at the same quality tier`,
].join('\n'),
links: [
'https://blog.voyageai.com/2026/01/15/voyage-4/',
'https://docs.vaicli.com/models/text-embeddings/',
],
tryIt: [
'vai embed "test MoE quality" --model voyage-4-large',
'vai benchmark embed --models voyage-4-large,voyage-4,voyage-4-lite',
'vai models --wide',
],
},
'shared-embedding-space': {
title: 'Shared Embedding Space',
summary: 'How Voyage 4 models produce compatible, interchangeable embeddings',
content: [
`The Voyage 4 series introduces an ${pc.cyan('industry-first capability')}: all four models`,
`(voyage-4-large, voyage-4, voyage-4-lite, voyage-4-nano) produce embeddings in`,
`the ${pc.cyan('same vector space')}. Embeddings from different models are directly comparable.`,
``,
`${pc.bold('What this means:')}`,
` ${pc.dim('•')} Embed documents with ${pc.cyan('voyage-4-large')} (best quality, one-time cost)`,
` ${pc.dim('•')} Query with ${pc.cyan('voyage-4-lite')} (low cost) or ${pc.cyan('voyage-4-nano')} (local, HuggingFace only)`,
` ${pc.dim('•')} Cosine similarity works across model boundaries`,
` ${pc.dim('•')} Upgrade query model later ${pc.cyan('without re-vectorizing documents')}`,
``,
`${pc.bold('Why this is new:')} Previously, embeddings from different models lived in`,
`incompatible vector spaces. Switching models meant re-embedding your entire`,
`corpus — expensive and slow. The shared space eliminates this constraint.`,
``,
`${pc.bold('Recommended workflow:')}`,
` ${pc.dim('1.')} Vectorize your document corpus once with ${pc.cyan('voyage-4-large')}`,
` ${pc.dim('2.')} Start with ${pc.cyan('voyage-4-lite')} for queries in development / early production`,
` ${pc.dim('3.')} Upgrade to ${pc.cyan('voyage-4')} or ${pc.cyan('voyage-4-large')} as accuracy needs grow`,
` ${pc.dim('4.')} No re-vectorization needed at any step`,
``,
`${pc.bold('Validate it yourself:')} Use ${pc.cyan('vai benchmark space')} to embed identical text`,
`with all Voyage 4 models and see the cross-model cosine similarities.`,
``,
`${pc.bold('Interactive proof:')} Try the ${pc.cyan('Shared Space Explorer')} at`,
`${pc.cyan('vaicli.com/shared-space')} — embed text with all three models simultaneously`,
`and see 0.95+ cross-model similarity in a live 3×3 matrix, scatter plot, and`,
`cost comparison. Share your results directly to LinkedIn.`,
].join('\n'),
links: [
'https://blog.voyageai.com/2026/01/15/voyage-4/',
'https://vaicli.com/shared-space',
],
tryIt: [
'vai benchmark space',
'vai benchmark asymmetric --query "your search" --file corpus.txt',
'vai estimate --docs 1M --queries 10M',
],
},
'rteb-benchmarks': {
title: 'RTEB Benchmark Scores',
summary: 'Retrieval quality scores across embedding providers',
content: [
`The ${pc.cyan('Retrieval Embedding Benchmark (RTEB)')} evaluates general-purpose retrieval`,
`quality across 29 diverse datasets. Scores are ${pc.cyan('NDCG@10')} (normalized discounted`,
`cumulative gain at top 10 results) — higher is better.`,
``,
`${pc.bold('Current standings (Jan 2026):')}`,
` ${pc.cyan('voyage-4-large')} ${pc.bold('71.41')} ${pc.dim('— SOTA, MoE architecture')}`,
` ${pc.cyan('voyage-4')} ${pc.bold('70.07')} ${pc.dim('— balanced quality/cost')}`,
` ${pc.cyan('Gemini Embedding 001')} ${pc.bold('68.66')} ${pc.dim('— Google')}`,
` ${pc.cyan('voyage-4-lite')} ${pc.bold('68.10')} ${pc.dim('— best budget option')}`,
` ${pc.cyan('Cohere Embed v4')} ${pc.bold('65.75')} ${pc.dim('— Cohere')}`,
` ${pc.cyan('OpenAI v3 Large')} ${pc.bold('62.57')} ${pc.dim('— OpenAI')}`,
``,
`${pc.bold('What the numbers mean:')}`,
` ${pc.dim('•')} voyage-4-large beats Gemini by ${pc.cyan('3.87%')}, Cohere by ${pc.cyan('8.20%')}, OpenAI by ${pc.cyan('14.05%')}`,
` ${pc.dim('•')} voyage-4 (mid-tier pricing) outperforms all non-Voyage models`,
` ${pc.dim('•')} Even voyage-4-lite ($0.02/1M) is competitive with Gemini Embedding`,
``,
`${pc.bold('Asymmetric retrieval bonus:')} When documents are embedded with voyage-4-large`,
`and queries with a smaller Voyage 4 model, retrieval quality ${pc.cyan('improves')} over`,
`using the smaller model alone — you get the benefit of the larger model's`,
`document representations.`,
``,
`${pc.bold('Note:')} These scores are from Voyage AI's evaluation. Independent benchmarks`,
`may differ. Always test on your own data with ${pc.cyan('vai benchmark similarity')}.`,
].join('\n'),
links: [
'https://blog.voyageai.com/2026/01/15/voyage-4/',
'https://docs.google.com/spreadsheets/d/1GfPkqCAjPKaGS9f66IDhMRxVpd2bMuqL2wXjj-kNS7E/',
],
tryIt: [
'vai models --benchmarks',
'vai benchmark similarity --query "your query" --file your-docs.txt',
'vai estimate --docs 1M --queries 10M',
],
},
'voyage-4-nano': {
title: 'voyage-4-nano -- Local Inference with the CLI',
summary: 'Zero-API-key embeddings via vai nano setup + --local flag',
content: [
`${pc.bold('What is voyage-4-nano?')}`,
`${pc.cyan('voyage-4-nano')} is Voyage AI's open-weight embedding`,
`model (Apache 2.0, 340M params). Run it locally with the`,
`CLI -- no API key, no network, no cost.`,
``,
`${pc.bold('CLI Workflow:')}`,
` ${pc.dim('1.')} ${pc.cyan('vai nano setup')} -- one-time env + model download`,
` ${pc.dim('2.')} ${pc.cyan('vai nano status')} -- verify readiness`,
` ${pc.dim('3.')} ${pc.cyan('vai embed "text" --local')} -- embed locally`,
``,
`${pc.bold('Architecture:')}`,
` Node.js spawns a Python subprocess (nano-bridge.py)`,
` that loads the model via sentence-transformers.`,
` Communication uses NDJSON over stdio. The bridge`,
` process stays warm for fast subsequent calls`,
` (~50-200ms per batch vs ~2s cold start).`,
``,
`${pc.bold('Key specs:')}`,
` ${pc.dim('\u2022')} Dimensions: 256, 512, 1024 (default), 2048 (MRL)`,
` ${pc.dim('\u2022')} Quantization: float32, int8, uint8, binary`,
` ${pc.dim('\u2022')} Context: 32K tokens`,
` ${pc.dim('\u2022')} Model size: ~700MB, cached at ~/.vai/nano-model/`,
` ${pc.dim('\u2022')} Venv: ~/.vai/nano-env/`,
``,
`${pc.bold('Shared embedding space:')}`,
` Nano embeddings are compatible with voyage-4,`,
` voyage-4-lite, and voyage-4-large. Embed docs`,
` locally, query via the API -- no re-indexing needed.`,
].join('\n'),
links: [
'https://huggingface.co/voyageai/voyage-4-nano',
],
tryIt: [
'vai nano setup',
'vai nano status',
'vai embed "hello world" --local',
'vai demo nano',
'vai explain shared-space',
],
},
'local-inference': {
title: 'Local Inference',
summary: 'Run voyage-4-nano locally, compare it to remote APIs, and understand the Python bridge',
content: [
`${pc.bold('What is local inference?')}`,
`${pc.cyan('Local inference')} means generating embeddings on your own machine instead of`,
`sending text to a hosted API. In vai, local inference uses ${pc.cyan('voyage-4-nano')},`,
`Voyage AI's open-weight embedding model, through ${pc.cyan('vai nano setup')} and the`,
`${pc.cyan('--local')} flag on commands like ${pc.cyan('vai embed')}, ${pc.cyan('vai ingest')}, and ${pc.cyan('vai chat')}.`,
``,
`${pc.bold('How it differs from remote Voyage models:')}`,
` ${pc.dim('•')} ${pc.cyan('Remote API models')} are hosted, managed, and accessed over HTTPS with an API key`,
` ${pc.dim('•')} ${pc.cyan('Local inference')} runs only ${pc.cyan('voyage-4-nano')} on your hardware with no network call`,
` ${pc.dim('•')} Remote models give you managed scaling and the full hosted model catalog`,
` ${pc.dim('•')} Local inference gives you offline development, zero per-call cost, and more data locality`,
` ${pc.dim('•')} Performance for local inference depends on your CPU/GPU, RAM, and whether the bridge is already warm`,
``,
`${pc.bold('Why the Python bridge exists:')}`,
` The CLI is written in ${pc.cyan('Node.js')}, but local embedding inference depends on`,
` the ${pc.cyan('Python ML stack')} -- sentence-transformers, PyTorch, and HuggingFace model`,
` loading. Instead of re-implementing model inference in JavaScript, vai spawns`,
` ${pc.cyan('nano-bridge.py')} as a persistent subprocess and talks to it over`,
` ${pc.cyan('NDJSON over stdio')}. This keeps the CLI fast and portable while using the`,
` ecosystem where local model inference is most mature.`,
``,
`${pc.bold('Why a persistent bridge matters:')}`,
` Loading the model is the expensive step. By keeping the Python process warm,`,
` subsequent embedding batches avoid repeated cold starts and feel much closer`,
` to normal CLI latency.`,
``,
`${pc.bold('When to use local vs remote:')}`,
` ${pc.dim('•')} Use ${pc.cyan('local inference')} for demos, offline work, prototyping, privacy-sensitive dev, and cost control`,
` ${pc.dim('•')} Use ${pc.cyan('remote API models')} for hosted scale, simpler ops, and production workloads that should not depend on local hardware`,
``,
`${pc.bold('Shared embedding space:')}`,
` ${pc.cyan('voyage-4-nano')} lives in the same embedding space as voyage-4,`,
` voyage-4-lite, and voyage-4-large. That means you can embed documents locally`,
` and still query or rerank with remote Voyage models later without rebuilding`,
` your whole index.`,
].join('\n'),
links: [
'https://huggingface.co/voyageai/voyage-4-nano',
'https://www.sbert.net/',
'https://blog.voyageai.com/2026/01/15/voyage-4/',
],
tryIt: [
'vai nano setup',
'vai nano status',
'vai embed "local embeddings are live" --local',
'vai explain voyage-4-nano',
'vai explain shared-space',
],
},
'multimodal-embeddings': {
title: 'Multimodal Embeddings',
summary: 'Embed images and text into the same vector space',
content: [
`${pc.bold('What are multimodal embeddings?')}`,
`${pc.cyan('Multimodal embeddings')} encode both text and images into the same vector space.`,
`Unlike text-only models, a multimodal model can process a photo, a slide deck,`,
`a PDF screenshot, or a mix of text and images — and produce a vector that lives`,
`in the ${pc.cyan('same space')} as pure text embeddings.`,
``,
`${pc.bold('Why this matters:')}`,
` ${pc.dim('•')} Search images with text queries ("sunset over mountains")`,
` ${pc.dim('•')} Search text with image queries (drop a photo, find matching descriptions)`,
` ${pc.dim('•')} Compare images to text directly with cosine similarity`,
` ${pc.dim('•')} Build RAG pipelines over documents with charts, tables, and figures`,
` ${pc.dim('•')} No OCR or complex document parsing needed — the model sees the visuals`,
``,
`${pc.bold('How Voyage AI is different:')}`,
`Most multimodal models (CLIP, Cohere, Amazon Titan) use ${pc.cyan('separate encoders')} for`,
`text and images — a "text tower" and a "vision tower." This creates a fundamental`,
`problem called the ${pc.cyan('modality gap')}: text vectors cluster with other text, and image`,
`vectors cluster with other images, regardless of semantic content.`,
``,
`Voyage's ${pc.cyan('voyage-multimodal-3.5')} processes both modalities through a ${pc.cyan('single unified')}`,
`${pc.cyan('transformer backbone')} — the same architecture used in modern vision-language models,`,
`but for vectorization. This eliminates the modality gap and enables true cross-modal`,
`search where a text query finds the most relevant image, not just the closest text.`,
``,
`${pc.bold('Interleaved inputs:')}`,
`Unlike CLIP-style models that accept a single text OR a single image, Voyage`,
`multimodal models accept ${pc.cyan('interleaved sequences')} of text and images. You can embed`,
`a slide that has a title, a chart, and bullet points as a single input — the`,
`model captures the spatial and contextual relationships between all elements.`,
``,
`${pc.bold('Supported formats:')} PNG, JPEG, WebP, GIF`,
`${pc.bold('Max image size:')} 16 million pixels, 20 MB`,
`${pc.bold('Token counting:')} Every 560 pixels ≈ 1 token`,
].join('\n'),
links: [
'https://blog.voyageai.com/2026/01/15/voyage-multimodal-3-5/',
'https://blog.voyageai.com/2024/11/12/voyage-multimodal-3/',
'https://docs.voyageai.com/docs/multimodal-embeddings',
],
tryIt: [
'vai embed --image photo.jpg --model voyage-multimodal-3.5',
'vai embed --image chart.png --text "Q4 revenue growth" --model voyage-multimodal-3.5',
],
},
'cross-modal-search': {
title: 'Cross-Modal Search',
summary: 'Search images with text and text with images',
content: [
`${pc.bold('What is cross-modal search?')}`,
`${pc.cyan('Cross-modal search')} means querying across different data types — for example,`,
`typing "a cat sitting on a windowsill" and getting back the most relevant`,
`${pc.cyan('photos')} from your collection, or uploading a product photo and finding the`,
`most relevant ${pc.cyan('text descriptions')} in your catalog.`,
``,
`${pc.bold('How it works:')}`,
` ${pc.dim('1.')} Embed all your content (images, text, or mixed) with a multimodal model`,
` ${pc.dim('2.')} Store the vectors in a database like MongoDB Atlas Vector Search`,
` ${pc.dim('3.')} At query time, embed the query (text or image) with the same model`,
` ${pc.dim('4.')} Find nearest neighbors — results can be any modality`,
``,
`${pc.bold('Use cases:')}`,
` ${pc.dim('•')} ${pc.cyan('E-commerce:')} "Show me red running shoes" → product photos`,
` ${pc.dim('•')} ${pc.cyan('Medical:')} Upload an X-ray → find similar cases in the database`,
` ${pc.dim('•')} ${pc.cyan('Legal:')} Search for relevant contract clauses across scanned PDFs`,
` ${pc.dim('•')} ${pc.cyan('Education:')} "Explain photosynthesis" → find diagrams and text explanations`,
` ${pc.dim('•')} ${pc.cyan('Content moderation:')} Check if uploaded images match banned content descriptions`,
``,
`${pc.bold('Try it in the Multimodal tab:')}`,
`The Cross-Modal Gallery lets you build a mini corpus of images and text, then`,
`search across both modalities. Results are ranked by cosine similarity regardless`,
`of whether they're images or text — demonstrating true unified search.`,
``,
`${pc.bold('The modality gap problem:')}`,
`CLIP-style models suffer from a ${pc.cyan('modality gap')} — text and image vectors occupy`,
`different regions of the embedding space. A text query like "sunset" will rank`,
`${pc.cyan('irrelevant texts')} higher than a ${pc.cyan('perfect sunset photo')}, because the text vectors`,
`are geometrically closer to each other regardless of meaning.`,
``,
`Voyage's unified backbone eliminates this bias. In benchmarks, voyage-multimodal-3`,
`outperforms CLIP by ${pc.cyan('41%')} on figure/table retrieval and ${pc.cyan('27%')} on document screenshots.`,
].join('\n'),
links: [
'https://docs.voyageai.com/docs/multimodal-embeddings',
'https://www.mongodb.com/docs/atlas/atlas-vector-search/vector-search-overview/',
],
tryIt: [
'vai embed --image product.jpg --model voyage-multimodal-3.5',
'vai embed "red running shoes" --model voyage-multimodal-3.5',
],
},
'modality-gap': {
title: 'The Modality Gap',
summary: 'Why CLIP-style models fail at mixed search — and how Voyage solves it',
content: [
`${pc.bold('What is the modality gap?')}`,
`The ${pc.cyan('modality gap')} is a well-documented phenomenon in CLIP-style multimodal models`,
`where text and image embeddings occupy ${pc.cyan('separate regions')} of the vector space, even`,
`when they represent the same concept.`,
``,
`${pc.bold('Example:')} Given the text "I address you, members of Congress..." and a screenshot`,
`of that exact text, a CLIP model will place the screenshot's vector ${pc.cyan('closer to')}`,
`${pc.cyan('other random images')} than to the text it literally contains. The vectors cluster`,
`by modality (text vs. image), not by meaning.`,
``,
`${pc.bold('Why it happens:')}`,
`CLIP-style architectures have two ${pc.cyan('independent encoders')} — a text transformer and`,
`a vision transformer — that are trained with contrastive learning to align their`,
`outputs. But because the encoders are separate, they develop different internal`,
`representations that never fully converge. The result is a geometric gap between`,
`the two modality clusters.`,
``,
`${pc.bold('Consequences for search:')}`,
` ${pc.dim('•')} Text queries retrieve ${pc.cyan('irrelevant text')} over ${pc.cyan('relevant images')}`,
` ${pc.dim('•')} Image queries retrieve ${pc.cyan('irrelevant images')} over ${pc.cyan('relevant text')}`,
` ${pc.dim('•')} You essentially get two separate search systems, not one unified one`,
` ${pc.dim('•')} Cosine similarity across modalities is not meaningful`,
``,
`${pc.bold('How Voyage AI solves this:')}`,
`${pc.cyan('voyage-multimodal-3.5')} uses a ${pc.cyan('single transformer backbone')} that processes both`,
`text and images through the same network — similar to modern vision-language`,
`models (GPT-4V, Claude), but optimized for vectorization instead of generation.`,
``,
`