LlamaIndex, LangChain, DSPy, Instructor, and the Vercel AI SDK are the five frameworks developers most often compare when building LLM applications, although each framework solves a different engineering problem. LlamaIndex connects models to application data; LangChain coordinates multi-step applications across a broad integration catalog; DSPy optimizes prompts based on input and output specifications; Instructor validates structured model responses; and the Vercel AI SDK supports streaming applications in TypeScript.
This guide explains the primary use case and scope of each framework, helping you choose based on your application's engineering requirements. Whichever framework you pick, response quality still has to be measured separately. All five frameworks have native Braintrust SDK integrations, so teams can track runs and evaluate model responses consistently as they switch frameworks or combine several in a single application.
What an LLM application framework does
An LLM application can call a model directly through a provider API or local inference endpoint. A framework reduces the additional code required when the application needs to retrieve external data, coordinate several model calls, optimize prompts, validate outputs, or stream responses to a frontend.
Most LLM application frameworks handle one or more of five responsibilities.
Connecting a model to your data: Retrieval-augmented generation (RAG) retrieves relevant passages from documents and adds them to the model's input. RAG frameworks can load source data, split documents into chunks, generate embeddings, build indexes, and retrieve passages for each query.
Orchestrating multiple steps: Applications that depend on several model calls need logic to sequence calls, invoke tools, handle conditional branches, retry failed operations, and pass data between steps.
Optimizing prompts: Prompt-optimization frameworks generate and test candidate instructions and examples against a scoring function. Developers can then compare the results and select prompts that meet defined quality criteria without relying entirely on manual experimentation.
Enforcing structured output: Applications that write model responses to a database or send them to another service require predictable fields and data types. Structured-output libraries define schemas, validate responses, and retry model calls when validation fails.
Building TypeScript applications: TypeScript frameworks support response streaming, UI state management, and model provider switching in JavaScript applications.
How LLM app frameworks differ
Use the five dimensions below to compare frameworks according to your application's engineering requirements. A framework's position on any dimension does not indicate its overall quality.
Primary job: Start with the part of the application that requires the most support from the framework. Although a framework may cover several responsibilities, its main focus determines the APIs, defaults, and documentation developers use most often.
Abstraction level: Compare how much code the framework places between the application and the provider SDK. Higher-level abstractions reduce setup and orchestration code, but they can make exact requests and intermediate execution states harder to inspect. Lower-level libraries require developers to manage more application logic while keeping model calls easier to follow.
Language support: Choose a framework that works with the application's existing stack. Python has extensive support for retrieval tooling, model integrations, and research libraries. TypeScript keeps API code, model logic, interface components, and shared types within the same language when the application runs on Next.js, Node.js, or another JavaScript runtime.
Ecosystem maturity: Review release history, API stability, maintainer activity, integration coverage, and documentation. Stable APIs limit migration work, maintained integrations reduce the need for custom adapters, and complete documentation makes implementation and debugging easier.
Tracing and evaluation support: Check whether the framework exposes the model calls and intermediate operations required to reproduce failures. Retrieval pipelines, prompt optimization runs, and streaming routes produce different trace structures, but teams still need consistent criteria for evaluating output quality. Braintrust supports tracing LLM apps and evaluating their outputs against shared quality standards.
The five LLM app frameworks worth knowing
LlamaIndex for data and retrieval workflows
LlamaIndex is an MIT-licensed Python framework for building applications that use private or application-specific data. Its components ingest source content, organize it into indexes, retrieve relevant passages, and use the retrieved context to generate responses.
Primary function: LlamaIndex connects models to external data. Components for document parsing, chunking, embeddings, indexing, hybrid retrieval, reranking, and response synthesis reduce the amount of custom code required to build a RAG pipeline.
Known for: Developers can assemble each stage of a retrieval pipeline from open-source components, while LlamaCloud provides managed services for document parsing, extraction, indexing, and retrieval.
Common applications: Typical uses include question answering across contracts, financial filings, support documentation, and internal knowledge bases. LlamaIndex can also supply retrieval capabilities to applications that coordinate model calls and tools through another orchestration library.
Braintrust integration: Braintrust's LlamaIndex integration captures model calls, embeddings, parsing, retrieval, and query-engine operations. Because query-engine traces retain the source nodes used for each response, teams can determine whether an incorrect answer originated in retrieval or generation and evaluate the answer against the supplied context.
LangChain for broad orchestration and fast prototyping
Available for Python and JavaScript under the MIT license, LangChain provides shared interfaces for combining models, tools, retrievers, and application state. Developers can use those interfaces to coordinate multi-step LLM applications across different providers and external services.
Primary function: LangChain orchestrates model calls, retrieval operations, tool execution, conditional branches, and stateful interactions. Its integration catalog reduces the amount of provider-specific connection code required when an application depends on multiple models, data sources, or services.
Known for: Broad integration coverage allows developers to assemble and test an application before every architectural choice has been finalized. LangChain also provides middleware for controlling agent execution, including model selection, tool access, retries, and human approval steps.
Common applications: Teams build chat applications, RAG pipelines, and tool-using agents that coordinate multiple data sources or external services within a single execution sequence.
Braintrust integration: The Braintrust LangChain integration traces chains, model calls, retrieval operations, and tool execution within each run. Teams can then apply the same evaluation criteria to development experiments and production traces.
DSPy for declarative prompt optimization
DSPy is an MIT-licensed Python framework for defining model behavior through signatures and modules. Signatures specify the expected inputs and outputs, while modules define how the application processes those inputs or combines several reasoning steps.
Primary function: DSPy's optimizers search for instructions and examples that improve a defined metric on representative data, providing teams with a repeatable process for updating prompts as models or application logic change.
Known for: DSPy ties prompt generation to measurable output quality. Teams can compare optimized programs across datasets and models while the required input and output behavior stays fixed.
Common applications: Teams reach for DSPy on classification, extraction, RAG, and multi-step reasoning tasks when they have representative examples and a metric that can distinguish better outputs from worse ones.
Braintrust integration: The Braintrust DSPy integration traces execution of the module, prompt formatting, model calls, parsing, and tool use. Optimized programs can be evaluated against Braintrust datasets, with scores retained alongside each experiment for comparison.
Instructor for structured, validated output
Instructor is an MIT-licensed library that converts model responses into validated objects. Its Python implementation uses Pydantic models to define the required fields and data types, then returns a typed object to the application.
import instructor
from openai import OpenAI
from pydantic import BaseModel
class UserInfo(BaseModel):
name: str
age: int
client = instructor.from_openai(OpenAI(), mode=instructor.Mode.RESPONSES_TOOLS)
user_info = client.responses.create(
model="gpt-5-mini",
input="John Doe is 30 years old.",
response_model=UserInfo,
)
Primary function: Instructor validates structured model output before a database, API, or downstream service receives it. When generated data fails schema validation, Instructor can return the validation error to the model and retry the request.
Known for: Instructor adds typed response validation directly to provider clients while leaving the rest of the application architecture unchanged. Implementations are available across Python, TypeScript, Go, Ruby, Elixir, and Rust, with support for multiple model providers.
Common applications: Data extraction, classification, and database-bound responses fit well, particularly when schema compliance is the main requirement, and the provider SDK handles the remaining application logic.
Braintrust integration: The Braintrust Instructor integration records the model request and validated response in the same trace. Teams can evaluate individual fields or compare the complete object with an expected result.
Vercel AI SDK for TypeScript and streaming apps
The Vercel AI SDK is an open-source TypeScript toolkit for adding AI features and agents to web applications. A shared provider interface handles model calls across vendors, while UI packages manage streamed responses and client-side state across supported frontend frameworks.
Primary function: The Vercel AI SDK keeps server-side generation, structured output, tool calling, agent execution, and client-side streaming within a TypeScript application. Full-stack teams can build an AI feature without introducing a separate Python service.
Known for: Consistent provider APIs and frontend integrations connect model execution with the interface that presents each response. Provider changes require fewer application-level changes because the surrounding code uses the same SDK interface.
Common applications: Teams use the Vercel AI SDK for chat interfaces, streaming assistants, generative interfaces, and agent features across Next.js, React, Svelte, Vue, and Angular.
Braintrust integration: Braintrust's Vercel AI SDK integration traces generation, streaming, tool calls, and agent execution. Teams can evaluate the responses delivered through the interface and score production traces against the same criteria used during development.
Also worth knowing
Firebase Genkit: Google's open-source framework supports AI application development across TypeScript, Go, and Python. Its local CLI and Developer UI help developers inspect prompts, flows, tools, and model responses during development. The Braintrust Genkit integration captures generation, embedding, flow, and tool operations for tracing and evaluation.
Agno: Agno is an Apache 2.0 Python framework and runtime for building agent applications. Its components manage agents, multi-agent teams, workflows, sessions, memory, knowledge, and persistent storage. Braintrust's Agno integration records agent, workflow, model, and tool operations within a single trace for evaluation.
LLM app framework comparison (2026)
| Dimension | LlamaIndex | LangChain | DSPy | Instructor | Vercel AI SDK |
|---|---|---|---|---|---|
| Primary function | Data ingestion, indexing, retrieval, and context augmentation | Agent and multi-step application orchestration across models, tools, and data sources | Optimizing prompts and model programs against a defined metric | Returning schema-validated, typed model outputs | Building TypeScript AI applications with server-side generation and UI streaming |
| Abstraction level | High for retrieval. APIs cover ingestion through response synthesis, with lower-level components available for customization. | High for orchestration. Standard interfaces manage models, tools, retrievers, state, and middleware. | High for prompt logic. Signatures and modules are compiled into prompts and examples. | Low. Instructor extends provider clients with schema validation and automatic retries. | Medium. Unified model and UI APIs standardize providers and streaming while application logic remains in developer code. |
| Language support | Python and TypeScript | Python and TypeScript/JavaScript | Python | Python, TypeScript, Go, Ruby, Elixir, and Rust | TypeScript and JavaScript |
| Ecosystem maturity | Connectors cover models, embeddings, retrievers, and vector stores, with managed parsing and retrieval available through LlamaCloud. | Stable v1 packages and integrations for models, embeddings, tools, document loaders, and vector stores | Maintained by Stanford NLP, with documented modules, optimizers, and model access through LiteLLM | Implementations across six languages and support for more than 15 model providers | Provider packages and UI integrations for React, Next.js, Svelte, Vue, Angular, and Node.js |
| Braintrust tracing and evaluation support | ✅ Native integration captures model calls, embeddings, node parsing, and query-engine runs. | ✅ Native integration captures models, chains, agents, tools, and retrievers. | ✅ Native integration captures modules, adapters, model calls, and tool execution. | ✅ Native integration captures structured-output generation and underlying provider calls. | ✅ Native integration captures generation, embeddings, reranking, streaming, tool calls, and agent execution. |
Matching LLM app frameworks to use cases
Retrieval, orchestration, prompt optimization, schema validation, and frontend streaming create different requirements, and a framework with a broader scope can introduce abstractions the application does not need.
For applications that depend on private documents or proprietary data: LlamaIndex covers ingestion, parsing, indexing, retrieval, reranking, and response synthesis. It is a strong starting point when answer quality depends heavily on retrieving the correct information from contracts, support content, research, or internal knowledge.
For applications that coordinate models, tools, and external services: LangChain manages multi-step execution, state, conditional branches, retries, and integrations through shared interfaces. Its broader scope supports applications that span several providers, data sources, or services.
For applications with measurable prompt-quality targets: DSPy uses representative examples and a scoring metric to optimize instructions and demonstrations. It becomes useful when teams can define what a better output looks like and need a reproducible process for improving prompts.
For applications that require schema-valid responses: Instructor adds typed validation and automatic retries to provider clients. Its narrower scope works well for extraction, classification, and database-bound responses when the application does not require broader orchestration.
For TypeScript applications with streaming interfaces: The Vercel AI SDK keeps model calls, tool execution, response streaming, and interface state within the same TypeScript codebase. Common applications include chat interfaces, assistants, and interactive AI features built with modern JavaScript frameworks.
Applications may combine frameworks when retrieval, orchestration, and structured output all require dedicated support. LlamaIndex can manage retrieval, LangChain can coordinate execution, and Instructor can validate responses before downstream services receive them. Framework selection determines how the application is assembled, but release decisions still require evidence that outputs meet defined quality standards. Braintrust provides a consistent evaluation layer across all five options, including applications that use several frameworks. Start evaluating for free with Braintrust →
FAQs: Best LLM app frameworks (2026)
Do I need a framework to build an LLM app, or is the provider SDK enough?
A provider SDK is enough when an application makes a small number of direct model calls and its request, response, and error-handling logic remains easy to test. A framework becomes useful once teams repeatedly write custom code for state, retries, retrieval, tool coordination, or provider adapters. Waiting until a concrete maintenance problem appears keeps unused abstractions and dependencies out of the codebase.
LlamaIndex vs LangChain: which one is better for connecting to my data?
LlamaIndex is generally the more direct choice when most engineering work involves parsing documents, building indexes, tuning retrieval, or returning citations. LangChain becomes more relevant when data access is one stage in a larger sequence of model calls, tools, and service integrations. Choose according to whether retrieval quality or application orchestration consumes more engineering time.
Do I need a framework just to get structured output?
A framework is usually unnecessary if the provider SDK can return JSON that conforms to your schema. A dedicated structured-output library becomes useful when you need typed validation, automatic retries after validation errors, or consistent handling across multiple providers. Structured output alone rarely justifies adopting a broader LLM application framework.
Can I use more than one LLM framework in the same app?
Multiple frameworks can share a single application, but each should have a clearly defined responsibility. Avoid letting two libraries control the same state, retry logic, or tool-execution path because overlapping control makes errors difficult to reproduce and upgrades harder to isolate. A shared tracing and evaluation workflow should cover the complete request even when separate libraries handle individual stages.
How do I evaluate an LLM app's output?
Define success according to the task before selecting a scorer. Extraction calls for field accuracy and schema compliance, RAG for retrieval relevance and groundedness, and agents for task completion and correct tool use. Evaluate each candidate change against normal requests, edge cases, and known failures, then inspect individual regressions that an average score may hide. Braintrust runs offline evaluations against versioned datasets and applies the same criteria to production traces, giving teams comparable evidence for release decisions.