create-agent-chat-app/templates/research-agent/sample_docs.json

Create a LangGraph chat app with one command

[
  {
    "pageContent": "All runnables expose input and output **schemas** to inspect the inputs and outputs:\n\n- `input_schema`: an input Pydantic model auto-generated from the structure of the Runnable\n\n- `output_schema`: an output Pydantic model auto-generated from the structure of the Runnable\n\n## Components\u200b\n\nLangChain provides standard, extendable interfaces and external integrations for various components useful for building with LLMs.\nSome components LangChain implements, some components we rely on third-party integrations for, and others are a mix.\n\n### Chat models\u200b\n\nLanguage models that use a sequence of messages as inputs and return chat messages as outputs (as opposed to using plain text).\nThese are traditionally newer models (older models are generally `LLMs`, see below).\nChat models support the assignment of distinct roles to conversation messages, helping to distinguish messages from the AI, users, and instructions such as system messages.\n\nAlthough the underlying models are messages in, message out, the LangChain wrappers also allow these models to take a string as input. This means you can easily use chat models in place of LLMs.\n\nWhen a string is passed in as input, it is converted to a `HumanMessage` and then passed to the underlying model.\n\nLangChain does not host any Chat Models, rather we rely on third party integrations.\n\nWe have some standardized parameters when constructing ChatModels:\n\n- `model`: the name of the model\n\n- `temperature`: the sampling temperature\n\n- `timeout`: request timeout\n\n- `max_tokens`: max tokens to generate\n\n- `stop`: default stop sequences\n\n- `max_retries`: max number of times to retry requests\n\n- `api_key`: API key for the model provider\n\n- `base_url`: endpoint to send requests to\n\nSome important things to note:\n\n- standard params only apply to model providers that expose parameters with the intended functionality. For example, some providers do not expose a configuration for maximum output tokens, so max_tokens can't be supported on these.\n\n- standard params are currently only enforced on integrations that have their own integration packages (e.g. `langchain-openai`, `langchain-anthropic`, etc.), they're not enforced on models in `langchain-community`.\n\nChatModels also accept other parameters that are specific to that integration. To find all the parameters supported by a ChatModel head to the API reference for that model.\n\ninfoSome chat models have been fine-tuned for **tool calling** and provide a dedicated API for it.\nGenerally, such models are better at tool calling than non-fine-tuned models, and are recommended for use cases that require tool calling.\nPlease see the [tool calling section](/v0.2/docs/concepts/#functiontool-calling) for more information.\n\nFor specifics on how to use chat models, see the [relevant how-to guides here](/v0.2/docs/how_to/#chat-models).\n\n#### Multimodality\u200b\n\nSome chat models are multimodal, accepting images, audio and even video as inputs. These are still less common, meaning model providers haven't standardized on the \"best\" way to define the API. Multimodal **outputs** are even less common. As such, we've kept our multimodal abstractions fairly light weight and plan to further solidify the multimodal APIs and interaction patterns as the field matures.\n\nIn LangChain, most chat models that support multimodal inputs also accept those values in OpenAI's content blocks format. So far this is restricted to image inputs. For models like Gemini which support video and other bytes input, the APIs also support the native, model-specific representations.\n\nFor specifics on how to use multimodal models, see the [relevant how-to guides here](/v0.2/docs/how_to/#multimodal).\n\nFor a full list of LangChain model providers with multimodal models, [check out this table](/v0.2/docs/integrations/chat/#advanced-features).\n\n### LLMs\u200b",
    "metadata": { "source": "https://python.langchain.com/v0.2/docs/concepts/" }
  },
  {
    "pageContent": "### Prompt templates\u200b\n\nPrompt templates help to translate user input and parameters into instructions for a language model.\nThis can be used to guide a model's response, helping it understand the context and generate relevant and coherent language-based output.\n\nPrompt Templates take as input a dictionary, where each key represents a variable in the prompt template to fill in.\n\nPrompt Templates output a PromptValue. This PromptValue can be passed to an LLM or a ChatModel, and can also be cast to a string or a list of messages.\nThe reason this PromptValue exists is to make it easy to switch between strings and messages.\n\nThere are a few different types of prompt templates:\n\n#### String PromptTemplates\u200b\n\nThese prompt templates are used to format a single string, and generally are used for simpler inputs.\nFor example, a common way to construct and use a PromptTemplate is as follows:\n\n```python\nfrom langchain_core.prompts import PromptTemplate\n\nprompt_template = PromptTemplate.from_template(\"Tell me a joke about {topic}\")\n\nprompt_template.invoke({\"topic\": \"cats\"})\n```\n\n**API Reference:**[PromptTemplate](https://python.langchain.com/v0.2/api_reference/core/prompts/langchain_core.prompts.prompt.PromptTemplate.html)#### ChatPromptTemplates\u200b\n\nThese prompt templates are used to format a list of messages. These \"templates\" consist of a list of templates themselves.\nFor example, a common way to construct and use a ChatPromptTemplate is as follows:\n\n```python\nfrom langchain_core.prompts import ChatPromptTemplate\n\nprompt_template = ChatPromptTemplate.from_messages([\n    (\"system\", \"You are a helpful assistant\"),\n    (\"user\", \"Tell me a joke about {topic}\")\n])\n\nprompt_template.invoke({\"topic\": \"cats\"})\n```\n\n**API Reference:**[ChatPromptTemplate](https://python.langchain.com/v0.2/api_reference/core/prompts/langchain_core.prompts.chat.ChatPromptTemplate.html)In the above example, this ChatPromptTemplate will construct two messages when called.\nThe first is a system message, that has no variables to format.\nThe second is a HumanMessage, and will be formatted by the `topic` variable the user passes in.\n\n#### MessagesPlaceholder\u200b\n\nThis prompt template is responsible for adding a list of messages in a particular place.\nIn the above ChatPromptTemplate, we saw how we could format two messages, each one a string.\nBut what if we wanted the user to pass in a list of messages that we would slot into a particular spot?\nThis is how you use MessagesPlaceholder.\n\n```python\nfrom langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder\nfrom langchain_core.messages import HumanMessage\n\nprompt_template = ChatPromptTemplate.from_messages([\n    (\"system\", \"You are a helpful assistant\"),\n    MessagesPlaceholder(\"msgs\")\n])\n\nprompt_template.invoke({\"msgs\": [HumanMessage(content=\"hi!\")]})\n```\n\n**API Reference:**[ChatPromptTemplate](https://python.langchain.com/v0.2/api_reference/core/prompts/langchain_core.prompts.chat.ChatPromptTemplate.html) | [MessagesPlaceholder](https://python.langchain.com/v0.2/api_reference/core/prompts/langchain_core.prompts.chat.MessagesPlaceholder.html) | [HumanMessage](https://python.langchain.com/v0.2/api_reference/core/messages/langchain_core.messages.human.HumanMessage.html)This will produce a list of two messages, the first one being a system message, and the second one being the HumanMessage we passed in.\nIf we had passed in 5 messages, then it would have produced 6 messages in total (the system message plus the 5 passed in).\nThis is useful for letting a list of messages be slotted into a particular spot.\n\nAn alternative way to accomplish the same thing without using the `MessagesPlaceholder` class explicitly is:\n\n```python\nprompt_template = ChatPromptTemplate.from_messages([\n    (\"system\", \"You are a helpful assistant\"),\n    (\"placeholder\", \"{msgs}\") # <-- This is the changed part\n])\n```\n\nFor specifics on how to use prompt templates, see the [relevant how-to guides here](/v0.2/docs/how_to/#prompt-templates).",
    "metadata": { "source": "https://python.langchain.com/v0.2/docs/concepts/" }
  },
  {
    "pageContent": "model = ChatAnthropic(model=\"claude-3-sonnet-20240229\")\n\nfor chunk in model.stream(\"what color is the sky?\"):\n    print(chunk.content, end=\"|\", flush=True)\n```\n\n**API Reference:**[ChatAnthropic](https://python.langchain.com/v0.2/api_reference/anthropic/chat_models/langchain_anthropic.chat_models.ChatAnthropic.html)For models (or other components) that don't support streaming natively, this iterator would just yield a single chunk, but\nyou could still use the same general pattern when calling them. Using `.stream()` will also automatically call the model in streaming mode\nwithout the need to provide additional config.\n\nThe type of each outputted chunk depends on the type of component - for example, chat models yield [AIMessageChunks](https://python.langchain.com/v0.2/api_reference/core/messages/langchain_core.messages.ai.AIMessageChunk.html).\nBecause this method is part of [LangChain Expression Language](/v0.2/docs/concepts/#langchain-expression-language-lcel),\nyou can handle formatting differences from different outputs using an [output parser](/v0.2/docs/concepts/#output-parsers) to transform\neach yielded chunk.\n\nYou can check out [this guide](/v0.2/docs/how_to/streaming/#using-stream) for more detail on how to use `.stream()`.\n\n#### .astream_events()\u200b\n\nWhile the `.stream()` method is intuitive, it can only return the final generated value of your chain. This is fine for single LLM calls,\nbut as you build more complex chains of several LLM calls together, you may want to use the intermediate values of\nthe chain alongside the final output - for example, returning sources alongside the final generation when building a chat\nover documents app.\n\nThere are ways to do this [using callbacks](/v0.2/docs/concepts/#callbacks-1), or by constructing your chain in such a way that it passes intermediate\nvalues to the end with something like chained [.assign()](/v0.2/docs/how_to/passthrough/) calls, but LangChain also includes an\n`.astream_events()` method that combines the flexibility of callbacks with the ergonomics of `.stream()`. When called, it returns an iterator\nwhich yields [various types of events](/v0.2/docs/how_to/streaming/#event-reference) that you can filter and process according\nto the needs of your project.\n\nHere's one small example that prints just events containing streamed chat model output:\n\n```python\nfrom langchain_core.output_parsers import StrOutputParser\nfrom langchain_core.prompts import ChatPromptTemplate\nfrom langchain_anthropic import ChatAnthropic\n\nmodel = ChatAnthropic(model=\"claude-3-sonnet-20240229\")\n\nprompt = ChatPromptTemplate.from_template(\"tell me a joke about {topic}\")\nparser = StrOutputParser()\nchain = prompt | model | parser\n\nasync for event in chain.astream_events({\"topic\": \"parrot\"}, version=\"v2\"):\n    kind = event[\"event\"]\n    if kind == \"on_chat_model_stream\":\n        print(event, end=\"|\", flush=True)\n```\n\n**API Reference:**[StrOutputParser](https://python.langchain.com/v0.2/api_reference/core/output_parsers/langchain_core.output_parsers.string.StrOutputParser.html) | [ChatPromptTemplate](https://python.langchain.com/v0.2/api_reference/core/prompts/langchain_core.prompts.chat.ChatPromptTemplate.html) | [ChatAnthropic](https://python.langchain.com/v0.2/api_reference/anthropic/chat_models/langchain_anthropic.chat_models.ChatAnthropic.html)You can roughly think of it as an iterator over callback events (though the format differs) - and you can use it on almost all LangChain components!\n\nSee [this guide](/v0.2/docs/how_to/streaming/#using-stream-events) for more detailed information on how to use `.astream_events()`,\nincluding a table listing available events.\n\n#### Callbacks\u200b",
    "metadata": { "source": "https://python.langchain.com/v0.2/docs/concepts/" }
  },
  {
    "pageContent": "You MUST compile your graph before you can use it.\nState\u00b6\nThe first thing you do when you define a graph is define the State of the graph. The State consists of the schema of the graph as well as reducer functions which specify how to apply updates to the state. The schema of the State will be the input schema to all Nodes and Edges in the graph, and can be either a TypedDict or a Pydantic model. All Nodes will emit updates to the State which are then applied using the specified reducer function.\nSchema\u00b6\nThe main documented way to specify the schema of a graph is by using TypedDict. However, we also support using a Pydantic BaseModel as your graph state to add default values and additional data validation.\nBy default, the graph will have the same input and output schemas. If you want to change this, you can also specify explicit input and output schemas directly. This is useful when you have a lot of keys, and some are explicitly for input and others for output. See the notebook here for how to use.\nMultiple schemas\u00b6\nTypically, all graph nodes communicate with a single schema. This means that they will read and write to the same state channels. But, there are cases where we want more control over this:\n\nInternal nodes can pass information that is not required in the graph's input / output.\nWe may also want to use different input / output schemas for the graph. The output might, for example, only contain a single relevant output key.\n\nIt is possible to have nodes write to private state channels inside the graph for internal node communication. We can simply define a private schema, PrivateState. See this notebook for more detail.  \nIt is also possible to define explicit input and output schemas for a graph. In these cases, we define an \"internal\" schema that contains all keys relevant to graph operations. But, we also define input and output schemas that are sub-sets of the \"internal\" schema to constrain the input and output of the graph. See this notebook for more detail.\nLet's look at an example:\nclass InputState(TypedDict):\n    user_input: str\n\nclass OutputState(TypedDict):\n    graph_output: str\n\nclass OverallState(TypedDict):\n    foo: str\n    user_input: str\n    graph_output: str\n\nclass PrivateState(TypedDict):\n    bar: str\n\ndef node_1(state: InputState) -> OverallState:\n    # Write to OverallState\n    return {\"foo\": state[\"user_input\"] + \" name\"}\n\ndef node_2(state: OverallState) -> PrivateState:\n    # Read from OverallState, write to PrivateState\n    return {\"bar\": state[\"foo\"] + \" is\"}\n\ndef node_3(state: PrivateState) -> OutputState:\n    # Read from PrivateState, write to OutputState\n    return {\"graph_output\": state[\"bar\"] + \" Lance\"}\n\nbuilder = StateGraph(OverallState,input=InputState,output=OutputState)\nbuilder.add_node(\"node_1\", node_1)\nbuilder.add_node(\"node_2\", node_2)\nbuilder.add_node(\"node_3\", node_3)\nbuilder.add_edge(START, \"node_1\")\nbuilder.add_edge(\"node_1\", \"node_2\")\nbuilder.add_edge(\"node_2\", \"node_3\")\nbuilder.add_edge(\"node_3\", END)\n\ngraph = builder.compile()\ngraph.invoke({\"user_input\":\"My\"})\n{'graph_output': 'My name is Lance'}\n\nThere are two subtle and important points to note here:\n\nWe pass state: InputState as the input schema to node_1. But, we write out to foo, a channel in OverallState. How can we write out to a state channel that is not included in the input schema? This is because a node can write to any state channel in the graph state. The graph state is the union of of the state channels defined at initialization, which includes OverallState and the filters InputState and OutputState.",
    "metadata": {
      "source": "https://langchain-ai.github.io/langgraph/concepts/low_level/"
    }
  },
  {
    "pageContent": "| Name | Index Type | Uses an LLM | When to Use | Description |\n| ---- | ---- | ---- | ---- | ---- |\n| Vector store | Vector store | No | If you are just getting started and looking for something quick and easy. | This is the simplest method and the one that is easiest to get started with. It involves creating embeddings for each piece of text. |\n| ParentDocument | Vector store + Document Store | No | If your pages have lots of smaller pieces of distinct information that are best indexed by themselves, but best retrieved all together. | This involves indexing multiple chunks for each document. Then you find the chunks that are most similar in embedding space, but you retrieve the whole parent document and return that (rather than individual chunks). |\n| Multi Vector | Vector store + Document Store | Sometimes during indexing | If you are able to extract information from documents that you think is more relevant to index than the text itself. | This involves creating multiple vectors for each document. Each vector could be created in a myriad of ways - examples include summaries of the text and hypothetical questions. |\n| Time-Weighted Vector store | Vector store | No | If you have timestamps associated with your documents, and you want to retrieve the most recent ones | This fetches documents based on a combination of semantic similarity (as in normal vector retrieval) and recency (looking at timestamps of indexed documents) |\n\ntip- See our RAG from Scratch video on [indexing fundamentals](https://youtu.be/bjb_EMsTDKI?feature=shared)\n\n- See our RAG from Scratch video on [multi vector retriever](https://youtu.be/gTCU9I6QqCE?feature=shared)\n\nFifth, consider ways to improve the quality of your similarity search itself. Embedding models compress text into fixed-length (vector) representations that capture the semantic content of the document. This compression is useful for search / retrieval, but puts a heavy burden on that single vector representation to capture the semantic nuance / detail of the document. In some cases, irrelevant or redundant content can dilute the semantic usefulness of the embedding.\n\n[ColBERT](https://docs.google.com/presentation/d/1IRhAdGjIevrrotdplHNcc4aXgIYyKamUKTWtB3m3aMU/edit?usp=sharing) is an interesting approach to address this with a higher granularity embeddings: (1) produce a contextually influenced embedding for each token in the document and query, (2) score similarity between each query token and all document tokens, (3) take the max, (4) do this for all query tokens, and (5) take the sum of the max scores (in step 3) for all query tokens to get a query-document similarity score; this token-wise scoring can yield strong results. \n\n![](/v0.2/assets/images/colbert-0bf5bd7485724d0005a2f5bdadbdaedb.png)\n\nThere are some additional tricks to improve the quality of your retrieval. Embeddings excel at capturing semantic information, but may struggle with keyword-based queries. Many [vector stores](/v0.2/docs/integrations/retrievers/pinecone_hybrid_search/) offer built-in [hybrid-search](https://docs.pinecone.io/guides/data/understanding-hybrid-search) to combine keyword and semantic similarity, which marries the benefits of both approaches. Furthermore, many vector stores have [maximal marginal relevance](https://python.langchain.com/v0.1/docs/modules/model_io/prompts/example_selectors/mmr/), which attempts to diversify the results of a search to avoid returning similar and redundant documents.",
    "metadata": { "source": "https://python.langchain.com/v0.2/docs/concepts/" }
  },
  {
    "pageContent": "graph = StateGraph(State, config_schema=ConfigSchema)\n\nYou can then pass this configuration into the graph using the configurable config field.\nconfig = {\"configurable\": {\"llm\": \"anthropic\"}}\n\ngraph.invoke(inputs, config=config)\n\nYou can then access and use this configuration inside a node:\ndef node_a(state, config):\n    llm_type = config.get(\"configurable\", {}).get(\"llm\", \"openai\")\n    llm = get_llm(llm_type)\n    ...\n\nSee this guide for a full breakdown on configuration.\nRecursion Limit\u00b6\nThe recursion limit sets the maximum number of super-steps the graph can execute during a single execution. Once the limit is reached, LangGraph will raise GraphRecursionError. By default this value is set to 25 steps. The recursion limit can be set on any graph at runtime, and is passed to .invoke/.stream via the config dictionary. Importantly, recursion_limit is a standalone config key and should not be passed inside the configurable key as all other user-defined configuration. See the example below:\ngraph.invoke(inputs, config={\"recursion_limit\": 5, \"configurable\":{\"llm\": \"anthropic\"}})\n\nRead this how-to to learn more about how the recursion limit works.\nBreakpoints\u00b6\nIt can often be useful to set breakpoints before or after certain nodes execute. This can be used to wait for human approval before continuing. These can be set when you \"compile\" a graph. You can set breakpoints either before a node executes (using interrupt_before) or after a node executes (using interrupt_after.)\nYou MUST use a checkpoiner when using breakpoints. This is because your graph needs to be able to resume execution.\nIn order to resume execution, you can just invoke your graph with None as the input.\n# Initial run of graph\ngraph.invoke(inputs, config=config)\n\n# Let's assume it hit a breakpoint somewhere, you can then resume by passing in None\ngraph.invoke(None, config=config)\n\nSee this guide for a full walkthrough of how to add breakpoints.\nDynamic Breakpoints\u00b6\nIt may be helpful to dynamically interrupt the graph from inside a given node based on some condition. In LangGraph you can do so by using NodeInterrupt -- a special exception that can be raised from inside a node.\ndef my_node(state: State) -> State:\n    if len(state['input']) > 5:\n        raise NodeInterrupt(f\"Received input that is longer than 5 characters: {state['input']}\")\n\n    return state\n\nVisualization\u00b6\nIt's often nice to be able to visualize graphs, especially as they get more complex. LangGraph comes with several built-in ways to visualize graphs. See this how-to guide for more info.\nStreaming\u00b6\nLangGraph is built with first class support for streaming, including streaming updates from graph nodes during the execution, streaming tokens from LLM calls and more. See this conceptual guide for more information.\nComments",
    "metadata": {
      "source": "https://langchain-ai.github.io/langgraph/concepts/low_level/"
    }
  },
  {
    "pageContent": "See [this guide](/v0.2/docs/how_to/streaming/#using-stream-events) for more detailed information on how to use `.astream_events()`,\nincluding a table listing available events.\n\n#### Callbacks\u200b\n\nThe lowest level way to stream outputs from LLMs in LangChain is via the [callbacks](/v0.2/docs/concepts/#callbacks) system. You can pass a\ncallback handler that handles the [on_llm_new_token](https://python.langchain.com/v0.2/api_reference/langchain/callbacks/langchain.callbacks.streaming_aiter.AsyncIteratorCallbackHandler.html#langchain.callbacks.streaming_aiter.AsyncIteratorCallbackHandler.on_llm_new_token) event into LangChain components. When that component is invoked, any\n[LLM](/v0.2/docs/concepts/#llms) or [chat model](/v0.2/docs/concepts/#chat-models) contained in the component calls\nthe callback with the generated token. Within the callback, you could pipe the tokens into some other destination, e.g. a HTTP response.\nYou can also handle the [on_llm_end](https://python.langchain.com/v0.2/api_reference/langchain/callbacks/langchain.callbacks.streaming_aiter.AsyncIteratorCallbackHandler.html#langchain.callbacks.streaming_aiter.AsyncIteratorCallbackHandler.on_llm_end) event to perform any necessary cleanup.\n\nYou can see [this how-to section](/v0.2/docs/how_to/#callbacks) for more specifics on using callbacks.\n\nCallbacks were the first technique for streaming introduced in LangChain. While powerful and generalizable,\nthey can be unwieldy for developers. For example:\n\n- You need to explicitly initialize and manage some aggregator or other stream to collect results.\n\n- The execution order isn't explicitly guaranteed, and you could theoretically have a callback run after the `.invoke()` method finishes.\n\n- Providers would often make you pass an additional parameter to stream outputs instead of returning them all at once.\n\n- You would often ignore the result of the actual model call in favor of callback results.\n\n#### Tokens\u200b\n\nThe unit that most model providers use to measure input and output is via a unit called a **token**.\nTokens are the basic units that language models read and generate when processing or producing text.\nThe exact definition of a token can vary depending on the specific way the model was trained -\nfor instance, in English, a token could be a single word like \"apple\", or a part of a word like \"app\".\n\nWhen you send a model a prompt, the words and characters in the prompt are encoded into tokens using a **tokenizer**.\nThe model then streams back generated output tokens, which the tokenizer decodes into human-readable text.\nThe below example shows how OpenAI models tokenize `LangChain is cool!`:\n\n![](/v0.2/assets/images/tokenization-10f566ab6774724e63dd99646f69655c.png)\n\nYou can see that it gets split into 5 different tokens, and that the boundaries between tokens are not exactly the same as word boundaries.\n\nThe reason language models use tokens rather than something more immediately intuitive like \"characters\"\nhas to do with how they process and understand text. At a high-level, language models iteratively predict their next generated output based on\nthe initial input and their previous generations. Training the model using tokens language models to handle linguistic\nunits (like words or subwords) that carry meaning, rather than individual characters, which makes it easier for the model\nto learn and understand the structure of the language, including grammar and context.\nFurthermore, using tokens can also improve efficiency, since the model processes fewer units of text compared to character-level processing.\n\n### Function/tool calling\u200b\n\ninfoWe use the term `tool calling` interchangeably with `function calling`. Although\nfunction calling is sometimes meant to refer to invocations of a single function,\nwe treat all models as though they can return multiple tool or function calls in\neach message.",
    "metadata": { "source": "https://python.langchain.com/v0.2/docs/concepts/" }
  },
  {
    "pageContent": "tip- See our RAG from Scratch [code](https://github.com/langchain-ai/rag-from-scratch) and [video series](https://youtube.com/playlist?list=PLfaIDFEXuae2LXbO1_PKyVJiQ23ZztA0x&feature=shared).\n\n- For a high-level guide on retrieval, see this [tutorial on RAG](/v0.2/docs/tutorials/rag/).\n\nRAG is only as good as the retrieved documents\u2019 relevance and quality. Fortunately, an emerging set of techniques can be employed to design and improve RAG systems. We've focused on taxonomizing and summarizing many of these techniques (see below figure) and will share some high-level strategic guidance in the following sections.\nYou can and should experiment with using different pieces together. You might also find [this LangSmith guide](https://docs.smith.langchain.com/how_to_guides/evaluation/evaluate_llm_application) useful for showing how to evaluate different iterations of your app.\n\n![](/v0.2/assets/images/rag_landscape-627f1d0fd46b92bc2db0af8f99ec3724.png)\n\n#### Query Translation\u200b\n\nFirst, consider the user input(s) to your RAG system. Ideally, a RAG system can handle a wide range of inputs, from poorly worded questions to complex multi-part queries.\n**Using an LLM to review and optionally modify the input is the central idea behind query translation.** This serves as a general buffer, optimizing raw user inputs for your retrieval system.\nFor example, this can be as simple as extracting keywords or as complex as generating multiple sub-questions for a complex query.\n\n| Name | When to use | Description |\n| ---- | ---- | ---- |\n| Multi-query | When you need to cover multiple perspectives of a question. | Rewrite the user question from multiple perspectives, retrieve documents for each rewritten question, return the unique documents for all queries. |\n| Decomposition | When a question can be broken down into smaller subproblems. | Decompose a question into a set of subproblems / questions, which can either be solved sequentially (use the answer from first + retrieval to answer the second) or in parallel (consolidate each answer into final answer). |\n| Step-back | When a higher-level conceptual understanding is required. | First prompt the LLM to ask a generic step-back question about higher-level concepts or principles, and retrieve relevant facts about them. Use this grounding to help answer the user question.Paper. |\n| HyDE | If you have challenges retrieving relevant documents using the raw user inputs. | Use an LLM to convert questions into hypothetical documents that answer the question. Use the embedded hypothetical documents to retrieve real documents with the premise that doc-doc similarity search can produce more relevant matches.Paper. |\n\ntipSee our RAG from Scratch videos for a few different specific approaches:\n\n- [Multi-query](https://youtu.be/JChPi0CRnDY?feature=shared)\n\n- [Decomposition](https://youtu.be/h0OPWlEOank?feature=shared)\n\n- [Step-back](https://youtu.be/xn1jEjRyJ2U?feature=shared)\n\n- [HyDE](https://youtu.be/SaDzIVkYqyY?feature=shared)\n\n#### Routing\u200b\n\nSecond, consider the data sources available to your RAG system. You want to query across more than one database or across structured and unstructured data sources. **Using an LLM to review the input and route it to the appropriate data source is a simple and effective approach for querying across sources.**\n\n| Name | When to use | Description |\n| ---- | ---- | ---- |\n| Logical routing | When you can prompt an LLM with rules to decide where to route the input. | Logical routing can use an LLM to reason about the query and choose which datastore is most appropriate. |\n| Semantic routing | When semantic similarity is an effective way to determine where to route the input. | Semantic routing embeds both query and, typically a set of prompts. It then chooses the appropriate prompt based upon similarity. |\n\ntipSee our RAG from Scratch video on [routing](https://youtu.be/pfpIndq7Fi8?feature=shared).  \n\n#### Query Construction\u200b",
    "metadata": { "source": "https://python.langchain.com/v0.2/docs/concepts/" }
  },
  {
    "pageContent": "tipSee our RAG from Scratch video on [routing](https://youtu.be/pfpIndq7Fi8?feature=shared).  \n\n#### Query Construction\u200b\n\nThird, consider whether any of your data sources require specific query formats. Many structured databases use SQL. Vector stores often have specific syntax for applying keyword filters to document metadata. **Using an LLM to convert a natural language query into a query syntax is a popular and powerful approach.**\nIn particular, [text-to-SQL](/v0.2/docs/tutorials/sql_qa/), [text-to-Cypher](/v0.2/docs/tutorials/graph/), and [query analysis for metadata filters](/v0.2/docs/tutorials/query_analysis/#query-analysis) are useful ways to interact with structured, graph, and vector databases respectively. \n\n| Name | When to Use | Description |\n| ---- | ---- | ---- |\n| Text to SQL | If users are asking questions that require information housed in a relational database, accessible via SQL. | This uses an LLM to transform user input into a SQL query. |\n| Text-to-Cypher | If users are asking questions that require information housed in a graph database, accessible via Cypher. | This uses an LLM to transform user input into a Cypher query. |\n| Self Query | If users are asking questions that are better answered by fetching documents based on metadata rather than similarity with the text. | This uses an LLM to transform user input into two things: (1) a string to look up semantically, (2) a metadata filter to go along with it. This is useful because oftentimes questions are about the METADATA of documents (not the content itself). |\n\ntipSee our [blog post overview](https://blog.langchain.dev/query-construction/) and RAG from Scratch video on [query construction](https://youtu.be/kl6NwWYxvbM?feature=shared), the process of text-to-DSL where DSL is a domain specific language required to interact with a given database. This converts user questions into structured queries. \n\n#### Indexing\u200b\n\nFourth, consider the design of your document index. A simple and powerful idea is to **decouple the documents that you index for retrieval from the documents that you pass to the LLM for generation.** Indexing frequently uses embedding models with vector stores, which [compress the semantic information in documents to fixed-size vectors](/v0.2/docs/concepts/#embedding-models).\n\nMany RAG approaches focus on splitting documents into chunks and retrieving some number based on similarity to an input question for the LLM. But chunk size and chunk number can be difficult to set and affect results if they do not provide full context for the LLM to answer a question. Furthermore, LLMs are increasingly capable of processing millions of tokens. \n\nTwo approaches can address this tension: (1) [Multi Vector](/v0.2/docs/how_to/multi_vector/) retriever using an LLM to translate documents into any form (e.g., often into a summary) that is well-suited for indexing, but returns full documents to the LLM for generation. (2) [ParentDocument](/v0.2/docs/how_to/parent_document_retriever/) retriever embeds document chunks, but also returns full documents. The idea is to get the best of both worlds: use concise representations (summaries or chunks) for retrieval, but use the full documents for answer generation.",
    "metadata": { "source": "https://python.langchain.com/v0.2/docs/concepts/" }
  },
  {
    "pageContent": "```python\nfrom langchain_community.document_loaders.csv_loader import CSVLoader\n\nloader = CSVLoader(\n    ...  # <-- Integration specific parameters here\n)\ndata = loader.load()\n```\n\n**API Reference:**[CSVLoader](https://python.langchain.com/v0.2/api_reference/community/document_loaders/langchain_community.document_loaders.csv_loader.CSVLoader.html)For specifics on how to use document loaders, see the [relevant how-to guides here](/v0.2/docs/how_to/#document-loaders).\n\n### Text splitters\u200b\n\nOnce you've loaded documents, you'll often want to transform them to better suit your application. The simplest example is you may want to split a long document into smaller chunks that can fit into your model's context window. LangChain has a number of built-in document transformers that make it easy to split, combine, filter, and otherwise manipulate documents.\n\nWhen you want to deal with long pieces of text, it is necessary to split up that text into chunks. As simple as this sounds, there is a lot of potential complexity here. Ideally, you want to keep the semantically related pieces of text together. What \"semantically related\" means could depend on the type of text. This notebook showcases several ways to do that.\n\nAt a high level, text splitters work as following:\n\n1. Split the text up into small, semantically meaningful chunks (often sentences).\n\n2. Start combining these small chunks into a larger chunk until you reach a certain size (as measured by some function).\n\n3. Once you reach that size, make that chunk its own piece of text and then start creating a new chunk of text with some overlap (to keep context between chunks).\n\nThat means there are two different axes along which you can customize your text splitter:\n\n1. How the text is split\n\n2. How the chunk size is measured\n\nFor specifics on how to use text splitters, see the [relevant how-to guides here](/v0.2/docs/how_to/#text-splitters).\n\n### Embedding models\u200b\n\nEmbedding models create a vector representation of a piece of text. You can think of a vector as an array of numbers that captures the semantic meaning of the text.\nBy representing the text in this way, you can perform mathematical operations that allow you to do things like search for other pieces of text that are most similar in meaning.\nThese natural language search capabilities underpin many types of [context retrieval](/v0.2/docs/concepts/#retrieval),\nwhere we provide an LLM with the relevant data it needs to effectively respond to a query.\n\n![](/v0.2/assets/images/embeddings-9c2616450a3b4f497a2d95a696b5f1a7.png)\n\nThe `Embeddings` class is a class designed for interfacing with text embedding models. There are many different embedding model providers (OpenAI, Cohere, Hugging Face, etc) and local models, and this class is designed to provide a standard interface for all of them.\n\nThe base Embeddings class in LangChain provides two methods: one for embedding documents and one for embedding a query. The former takes as input multiple texts, while the latter takes a single text. The reason for having these as two separate methods is that some embedding providers have different embedding methods for documents (to be searched over) vs queries (the search query itself).\n\nFor specifics on how to use embedding models, see the [relevant how-to guides here](/v0.2/docs/how_to/#embedding-models).\n\n### Vector stores\u200b\n\nOne of the most common ways to store and search over unstructured data is to embed it and store the resulting embedding vectors,\nand then at query time to embed the unstructured query and retrieve the embedding vectors that are 'most similar' to the embedded query.\nA vector store takes care of storing embedded data and performing vector search for you.\n\nMost vector stores can also store metadata about embedded vectors and support filtering on that metadata before\nsimilarity search, allowing you more control over returned documents.\n\nVector stores can be converted to the retriever interface by doing:",
    "metadata": { "source": "https://python.langchain.com/v0.2/docs/concepts/" }
  },
  {
    "pageContent": "LangGraph Glossary\n\nLangGraph Glossary\u00b6\nGraphs\u00b6\nAt its core, LangGraph models agent workflows as graphs. You define the behavior of your agents using three key components:\n\nState: A shared data structure that represents the current snapshot of your application. It can be any Python type, but is typically a TypedDict or Pydantic BaseModel.\n\nNodes: Python functions that encode the logic of your agents. They receive the current State as input, perform some computation or side-effect, and return an updated State.\n\nEdges: Python functions that determine which Node to execute next based on the current State. They can be conditional branches or fixed transitions.\n\nBy composing Nodes and Edges, you can create complex, looping workflows that evolve the State over time. The real power, though, comes from how LangGraph manages that State. To emphasize: Nodes and Edges are nothing more than Python functions - they can contain an LLM or just good ol' Python code.\nIn short: nodes do the work. edges tell what to do next.\nLangGraph's underlying graph algorithm uses message passing to define a general program. When a Node completes its operation, it sends messages along one or more edges to other node(s). These recipient nodes then execute their functions, pass the resulting messages to the next set of nodes, and the process continues. Inspired by Google's Pregel system, the program proceeds in discrete \"super-steps.\"\nA super-step can be considered a single iteration over the graph nodes. Nodes that run in parallel are part of the same super-step, while nodes that run sequentially belong to separate super-steps. At the start of graph execution, all nodes begin in an inactive state. A node becomes active when it receives a new message (state) on any of its incoming edges (or \"channels\"). The active node then runs its function and responds with updates. At the end of each super-step, nodes with no incoming messages vote to halt by marking themselves as inactive. The graph execution terminates when all nodes are inactive and no messages are in transit.\nStateGraph\u00b6\nThe StateGraph class is the main graph class to uses. This is parameterized by a user defined State object.\nMessageGraph\u00b6\nThe MessageGraph class is a special type of graph. The State of a MessageGraph is ONLY a list of messages. This class is rarely used except for chatbots, as most applications require the State to be more complex than a list of messages.\nCompiling your graph\u00b6\nTo build your graph, you first define the state, you then add nodes and edges, and then you compile it. What exactly is compiling your graph and why is it needed?\nCompiling is a pretty simple step. It provides a few basic checks on the structure of your graph (no orphaned nodes, etc). It is also where you can specify runtime args like checkpointers and breakpoints. You compile your graph by just calling the .compile method:\ngraph = graph_builder.compile(...)",
    "metadata": {
      "source": "https://langchain-ai.github.io/langgraph/concepts/low_level/"
    }
  },
  {
    "pageContent": "- Async callback handlers implement the [AsyncCallbackHandler](https://python.langchain.com/v0.2/api_reference/core/callbacks/langchain_core.callbacks.base.AsyncCallbackHandler.html) interface.\n\nDuring run-time LangChain configures an appropriate callback manager (e.g., [CallbackManager](https://python.langchain.com/v0.2/api_reference/core/callbacks/langchain_core.callbacks.manager.CallbackManager.html) or [AsyncCallbackManager](https://python.langchain.com/v0.2/api_reference/core/callbacks/langchain_core.callbacks.manager.AsyncCallbackManager.html) which will be responsible for calling the appropriate method on each \"registered\" callback handler when the event is triggered.\n\n#### Passing callbacks\u200b\n\nThe `callbacks` property is available on most objects throughout the API (Models, Tools, Agents, etc.) in two different places:\n\nThe callbacks are available on most objects throughout the API (Models, Tools, Agents, etc.) in two different places:\n\n- **Request time callbacks**: Passed at the time of the request in addition to the input data.\nAvailable on all standard `Runnable` objects. These callbacks are INHERITED by all children\nof the object they are defined on. For example, `chain.invoke({\"number\": 25}, {\"callbacks\": [handler]})`.\n\n- **Constructor callbacks**: `chain = TheNameOfSomeChain(callbacks=[handler])`. These callbacks\nare passed as arguments to the constructor of the object. The callbacks are scoped\nonly to the object they are defined on, and are **not** inherited by any children of the object.\n\ndangerConstructor callbacks are scoped only to the object they are defined on. They are **not** inherited by children\nof the object.\n\nIf you're creating a custom chain or runnable, you need to remember to propagate request time\ncallbacks to any child objects.\n\nAsync in Python<=3.10Any `RunnableLambda`, a `RunnableGenerator`, or `Tool` that invokes other runnables\nand is running `async` in python<=3.10, will have to propagate callbacks to child\nobjects manually. This is because LangChain cannot automatically propagate\ncallbacks to child objects in this case.\n\nThis is a common reason why you may fail to see events being emitted from custom\nrunnables or tools.\n\nFor specifics on how to use callbacks, see the [relevant how-to guides here](/v0.2/docs/how_to/#callbacks).\n\n## Techniques\u200b\n\n### Streaming\u200b\n\nIndividual LLM calls often run for much longer than traditional resource requests.\nThis compounds when you build more complex chains or agents that require multiple reasoning steps.\n\nFortunately, LLMs generate output iteratively, which means it's possible to show sensible intermediate results\nbefore the final response is ready. Consuming output as soon as it becomes available has therefore become a vital part of the UX\naround building apps with LLMs to help alleviate latency issues, and LangChain aims to have first-class support for streaming.\n\nBelow, we'll discuss some concepts and considerations around streaming in LangChain.\n\n#### .stream() and .astream()\u200b\n\nMost modules in LangChain include the `.stream()` method (and the equivalent `.astream()` method for [async](https://docs.python.org/3/library/asyncio.html) environments) as an ergonomic streaming interface.\n`.stream()` returns an iterator, which you can consume with a simple `for` loop. Here's an example with a chat model:\n\n```python\nfrom langchain_anthropic import ChatAnthropic\n\nmodel = ChatAnthropic(model=\"claude-3-sonnet-20240229\")\n\nfor chunk in model.stream(\"what color is the sky?\"):\n    print(chunk.content, end=\"|\", flush=True)\n```",
    "metadata": { "source": "https://python.langchain.com/v0.2/docs/concepts/" }
  },
  {
    "pageContent": "If you are still using AgentExecutor, do not fear: we still have a guide on [how to use AgentExecutor](/v0.2/docs/how_to/agent_executor/).\nIt is recommended, however, that you start to transition to LangGraph.\nIn order to assist in this, we have put together a [transition guide on how to do so](/v0.2/docs/how_to/migrate_agent/).\n\n#### ReAct agents\u200b\n\nOne popular architecture for building agents is [ReAct](https://arxiv.org/abs/2210.03629).\nReAct combines reasoning and acting in an iterative process - in fact the name \"ReAct\" stands for \"Reason\" and \"Act\".\n\nThe general flow looks like this:\n\n- The model will \"think\" about what step to take in response to an input and any previous observations.\n\n- The model will then choose an action from available tools (or choose to respond to the user).\n\n- The model will generate arguments to that tool.\n\n- The agent runtime (executor) will parse out the chosen tool and call it with the generated arguments.\n\n- The executor will return the results of the tool call back to the model as an observation.\n\n- This process repeats until the agent chooses to respond.\n\nThere are general prompting based implementations that do not require any model-specific features, but the most\nreliable implementations use features like [tool calling](/v0.2/docs/how_to/tool_calling/) to reliably format outputs\nand reduce variance.\n\nPlease see the [LangGraph documentation](https://langchain-ai.github.io/langgraph/) for more information,\nor [this how-to guide](/v0.2/docs/how_to/migrate_agent/) for specific information on migrating to LangGraph.\n\n### Callbacks\u200b\n\nLangChain provides a callbacks system that allows you to hook into the various stages of your LLM application. This is useful for logging, monitoring, streaming, and other tasks.\n\nYou can subscribe to these events by using the `callbacks` argument available throughout the API. This argument is list of handler objects, which are expected to implement one or more of the methods described below in more detail.\n\n#### Callback Events\u200b\n\n| Event | Event Trigger | Associated Method |\n| ---- | ---- | ---- |\n| Chat model start | When a chat model starts | on_chat_model_start |\n| LLM start | When a llm starts | on_llm_start |\n| LLM new token | When an llm OR chat model emits a new token | on_llm_new_token |\n| LLM ends | When an llm OR chat model ends | on_llm_end |\n| LLM errors | When an llm OR chat model errors | on_llm_error |\n| Chain start | When a chain starts running | on_chain_start |\n| Chain end | When a chain ends | on_chain_end |\n| Chain error | When a chain errors | on_chain_error |\n| Tool start | When a tool starts running | on_tool_start |\n| Tool end | When a tool ends | on_tool_end |\n| Tool error | When a tool errors | on_tool_error |\n| Agent action | When an agent takes an action | on_agent_action |\n| Agent finish | When an agent ends | on_agent_finish |\n| Retriever start | When a retriever starts | on_retriever_start |\n| Retriever end | When a retriever ends | on_retriever_end |\n| Retriever error | When a retriever errors | on_retriever_error |\n| Text | When arbitrary text is run | on_text |\n| Retry | When a retry event is run | on_retry |\n\n#### Callback handlers\u200b\n\nCallback handlers can either be `sync` or `async`:\n\n- Sync callback handlers implement the [BaseCallbackHandler](https://python.langchain.com/v0.2/api_reference/core/callbacks/langchain_core.callbacks.base.BaseCallbackHandler.html) interface.\n\n- Async callback handlers implement the [AsyncCallbackHandler](https://python.langchain.com/v0.2/api_reference/core/callbacks/langchain_core.callbacks.base.AsyncCallbackHandler.html) interface.",
    "metadata": { "source": "https://python.langchain.com/v0.2/docs/concepts/" }
  },
  {
    "pageContent": "We initialize the graph with StateGraph(OverallState,input=InputState,output=OutputState). So, how can we write to PrivateState in node_2? How does the graph gain access to this schema if it was not passed in the StateGraph initialization? We can do this because nodes can also declare additional state channels as long as the state schema definition exists. In this case, the PrivateState schema is defined, so we can add bar as a new state channel in the graph and write to it.\n\nReducers\u00b6\nReducers are key to understanding how updates from nodes are applied to the State. Each key in the State has its own independent reducer function. If no reducer function is explicitly specified then it is assumed that all updates to that key should override it. There are a few different types of reducers, starting with the default type of reducer:\nDefault Reducer\u00b6\nThese two examples show how to use the default reducer:\nExample A:\nfro