UNPKG

@juspay/neurolink

Version:

The pipe layer of an AI nervous system: one interface connecting provider neurons to your application, across three inference types — generate, stream and decide. `decide` returns typed, calibrated judgements from a non-generative model (~400ms, ~$0.00002

42 lines (41 loc) • 1.3 kB
/** * Semantic Chunker * * LLM-powered semantic chunking that groups related content together. * Uses embedding similarity to determine natural breakpoints. * Best for complex documents where meaning should drive segmentation. */ import type { BaseChunkerConfig, Chunk, Chunker, ChunkerValidationResult, SemanticChunkerConfig } from "../../types/index.js"; /** * Semantic chunker implementation * Uses embedding similarity to find natural content boundaries */ export declare class SemanticChunker implements Chunker { readonly strategy: "semantic"; chunk(text: string, config?: SemanticChunkerConfig): Promise<Chunk[]>; /** * Split text into initial segments for embedding */ private splitIntoSegments; /** * Get embeddings for segments */ private getEmbeddings; /** * Find semantic breakpoints using cosine similarity */ private findSemanticBreakpoints; /** * Group segments based on breakpoints and size limits */ private groupSegments; /** * Calculate cosine similarity between two vectors */ private cosineSimilarity; /** * Fallback to simple chunking when embeddings fail */ private fallbackChunk; validateConfig(config: BaseChunkerConfig): ChunkerValidationResult; }