Skip to main content

Structured Output with Pydantic

You've built chains. You've seen LLMs generate text. Now you need those responses to be predictable, typed, and validated — not free-form strings that your downstream code has to parse.

Structured output is the bridge between the LLM's creative generation and your application's need for reliable data. This guide covers the with_structured_output() API, Pydantic model design, advanced patterns, error handling, and testing strategies.

If you've read Getting Started and Best Practices, you already know the basics. This document goes deep on the pattern that makes production LLM applications possible.

Why Structured Output​

By default, LLMs return free-form text. That works for chat interfaces. It does not work for applications that need to route data, store results, or make decisions based on model output.

The Problem With Free-Form Text​

Without structured output, your code looks like this:

# Anti-pattern: parsing free-form LLM text
response = model.invoke("Extract the product name and price from: 'I want the Pro plan at $29/month'")
# You get back something like:
# "The product name is 'Pro plan' and the price is $29/month."

# Now you must parse it manually:
import re
match = re.search(r"('[^']+)',.*?\$(\d+)", response.content)
if match:
    name = match.group(1)
    price = int(match.group(2))
else:
    # What do you do here? The LLM formatted differently.
    pass

This is fragile. The LLM might:

  • Use different formatting on different calls.
  • Include extra commentary around the data.
  • Hallucinate fields that do not exist in your schema.
  • Omit fields when it "decides" they are not relevant.

What Structured Output Guarantees​

With with_structured_output(), you get:

  • Guaranteed shape — the response matches your Pydantic model or raises a clear error.
  • Automatic validation — type checking, constraints, and field validation happen before your code runs.
  • Type safety — your IDE knows the return type. Your type checker catches mistakes at build time.
  • No manual parsing — no regex, no string splitting, no guesswork.
from pydantic import BaseModel, Field
from langchain.chat_models import init_chat_model


class ProductInfo(BaseModel):
    name: str = Field(description="The product or plan name")
    price_usd: float = Field(description="Monthly price in US dollars")


model = init_chat_model("gpt-4o")
structured = model.with_structured_output(ProductInfo)

result = structured.invoke("I want the Pro plan at $29/month")
# ProductInfo(name='Pro plan', price_usd=29.0)

# Type-safe access:
print(result.name)       # "Pro plan"
print(result.price_usd)  # 29.0

Free-Form vs. Structured Output​

AspectFree-Form TextStructured Output
Return typestr — you parse it yourselfPydantic model instance
Shape guaranteeNone — the LLM decidesEnforced by schema validation
Type safetyNo — everything is a stringYes — fields have explicit types
Error handlingRegex fails silently or crashesClear OutputParserException
Downstream codeDefensive parsing everywhereClean, typed access
TestingHard — non-deterministic stringsEasy — assert on model fields
IDE supportNoneFull autocomplete on fields
tip

Structured output is not just convenience — it is a reliability requirement for production systems. Every chain that feeds data into a database, an API, or a business decision should use structured output.

How It Works​

The with_structured_output() method transforms a chat model into a typed output producer. Here is what happens under the hood.

The Flow​

User Input
    ↓
Prompt Template (fills variables)
    ↓
Chat Model + JSON Schema (injected into system prompt)
    ↓
LLM generates JSON matching the schema
    ↓
Pydantic validation (type checking, constraints)
    ↓
Typed Pydantic model instance (or raises OutputParserException)

Step by Step​

  1. Schema generation — LangChain converts your Pydantic model into a JSON Schema. This schema describes every field, its type, constraints, and the description you provided via Field().

  2. Prompt injection — The JSON Schema is injected into the system prompt alongside your instructions. The model sees the exact structure it must produce.

  3. Model generation — The LLM generates a response formatted as JSON that matches the schema. Modern models (GPT-4o, Claude Sonnet, etc.) are highly reliable at this when the schema is clear.

  4. Parsing and validation — LangChain parses the JSON response and passes it to Pydantic's model_validate(). Pydantic checks types, runs validators, and constructs a typed model instance.

  5. Return or error — If validation succeeds, you get a typed object. If it fails, you get an OutputParserException with details about what went wrong.

What the Model Actually Sees​

When you call model.with_structured_output(MyModel), LangChain constructs a system message that includes the JSON Schema. For a model like ProductInfo, the LLM receives something equivalent to:

{
  "name": "For your response, produce a JSON object matching this schema:",
  "schema": {
    "type": "object",
    "properties": {
      "name": {
        "type": "string",
        "description": "The product or plan name"
      },
      "price_usd": {
        "type": "number",
        "description": "Monthly price in US dollars"
      }
    },
    "required": ["name", "price_usd"],
    "additionalProperties": false
  }
}

The description on each field is critical — it is the LLM's instruction for what to put in that field.

info

Different model providers implement structured output differently. OpenAI uses native JSON mode. Anthropic uses tool calling. LangChain abstracts these differences so your code is provider-agnostic.

Basic Usage​

Let's build structured output chains from scratch.

Your First Structured Output​

from pydantic import BaseModel, Field
from langchain.chat_models import init_chat_model


# 1. Define your output schema
class SentimentAnalysis(BaseModel):
    sentiment: str = Field(description="One of: positive, negative, neutral")
    confidence: float = Field(description="Confidence score between 0.0 and 1.0")
    reasoning: str = Field(description="Brief explanation for the classification")


# 2. Create the model with structured output
model = init_chat_model("gpt-4o")
structured_model = model.with_structured_output(SentimentAnalysis)

# 3. Invoke with your input
result = structured_model.invoke(
    "Analyze the sentiment of this review: "
    "The food was amazing but the service was painfully slow."
)

# 4. You get a typed object back
print(result.sentiment)    # "mixed" or "neutral"
print(result.confidence)   # 0.85
print(result.reasoning)    # "Positive food experience offset by negative service..."

Combining with Prompt Templates​

For more control, combine structured output with a ChatPromptTemplate:

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    (
        "system",
        "You are a product review analyst. Analyze each review carefully "
        "and return your findings as structured data.",
    ),
    ("human", "Review: {review_text}"),
])

# Chain: prompt → structured model
chain = prompt | model.with_structured_output(SentimentAnalysis)

result = chain.invoke({"review_text": "Battery life is terrible. Screen is great though."})
print(result.sentiment)  # "mixed"

Do and Don't​

Do this:

from pydantic import BaseModel, Field
from typing import Literal


class TicketClassification(BaseModel):
    category: Literal["billing", "technical", "account", "general"]
    urgency: Literal["low", "medium", "high", "critical"]
    summary: str = Field(description="One-sentence summary of the issue")
    requires_escalation: bool = Field(description="True if a human agent should handle this")


structured = model.with_structured_output(TicketClassification)
result = structured.invoke("Classify: My credit card was charged twice for the same subscription.")
# TicketClassification(category='billing', urgency='high',
#                      summary='Customer reports duplicate charge for subscription.',
#                      requires_escalation=True)

Don't do this:

# Anti-pattern: vague schema with no guidance
class BadSchema(BaseModel):
    stuff: str       # What is "stuff"? The LLM will guess.
    data: list       # What kind of list? Of what?
    other: dict      # Dict with no structure is useless.
warning

Every field needs a description. The description is the instruction the LLM follows to fill that field. Without it, you get inconsistent or hallucinated values.

Pydantic Model Design for LLMs​

Designing Pydantic models for LLMs is different from designing them for APIs. The consumer is not a programmer — it is a language model that reads instructions and generates text.

Use Field(description=...) Religiously​

The description parameter is the LLM's instruction for each field. Be specific:

from pydantic import BaseModel, Field
from typing import Literal


class EntityExtraction(BaseModel):
    # GOOD: specific, actionable descriptions
    person_names: list[str] = Field(
        description="Full names of people mentioned in the text. Use exact spelling."
    )
    locations: list[str] = Field(
        description="Geographic locations mentioned. Use city, country format."
    )
    date_references: list[str] = Field(
        description="Dates or time periods mentioned. Use ISO 8601 format when possible."
    )
    sentiment: Literal["positive", "negative", "neutral", "mixed"] = Field(
        description="Overall sentiment of the text. Use 'mixed' if both positive and negative elements exist."
    )

Keep Schemas Simple​

Complex nested models confuse LLMs. If your schema has more than three levels of nesting, flatten it:

# GOOD: flat structure
class SupportAnalysis(BaseModel):
    issue_type: str = Field(description="Primary issue category")
    sub_issues: list[str] = Field(description="Related sub-issues, comma-separated concepts")
    customer_emotion: str = Field(description="Emotional tone: frustrated, calm, urgent, satisfied")
    suggested_action: str = Field(description="Recommended next step for the support agent")
    estimated_resolution_time: str = Field(description="Expected resolution: immediate, same_day, within_week, needs_escalation")


# BAD: deeply nested — LLMs struggle with this
class NestedAnalysis(BaseModel):
    customer: dict       # Too vague
    issue: dict          # Too vague
    resolution: dict     # Too vague
    metadata: dict       # Too vague

Use Literal for Constrained Choices​

When a field should only take specific values, use Literal. This constrains the LLM and eliminates hallucination:

from typing import Literal


class CodeReview(BaseModel):
    severity: Literal["critical", "major", "minor", "style"] = Field(
        description="Severity level of the issue found"
    )
    category: Literal["bug", "security", "performance", "readability", "architecture"] = Field(
        description="Category of the code issue"
    )
    fix_available: bool = Field(
        description="True if a concrete fix can be suggested"
    )
    suggestion: str = Field(
        description="Specific recommendation or code snippet to fix the issue"
    )

Provide Examples in the Prompt, Not the Schema​

Do not embed examples inside Field(description=...). Put them in the prompt template:

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    (
        "system",
        "You are a code reviewer. Analyze the code snippet and return findings.\n\n"
        "Examples:\n"
        "- A function with no type hints → severity: 'style', category: 'readability'\n"
        "- SQL concatenation with user input → severity: 'critical', category: 'security'\n"
        "- Nested loops over large lists → severity: 'major', category: 'performance'",
    ),
    ("human", "Review this code:\n{code_snippet}"),
])

chain = prompt | model.with_structured_output(CodeReview)

Schema Design Checklist​

GuidelineReason
Every field has a descriptionThe LLM needs instructions
Use Literal for fixed choicesPrevents hallucinated values
Keep nesting to 1–2 levelsDeep nesting confuses models
Prefer list[str] over nested modelsSimpler types are more reliable
Use bool for binary decisionsEasier for LLMs than string enums
Avoid Optional unless truly neededLLMs sometimes skip optional fields entirely
Name fields descriptivelycustomer_emotion beats emotion
tip

When in doubt, test your schema against 5–10 inputs before integrating it into a chain. Use FakeListChatModel in tests to verify the shape without calling real APIs. See Testing Structured Output.

Advanced Patterns​

Multi-Step Chains With Intermediate Structured Outputs​

Break complex tasks into stages, each with its own structured output:

from pydantic import BaseModel, Field
from langchain.chat_models import init_chat_model
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough


# Stage 1: Extract key information
class ExtractedInfo(BaseModel):
    entities: list[str] = Field(description="Named entities found in the text")
    topics: list[str] = Field(description="Main topics discussed")
    tone: str = Field(description="Overall tone: formal, informal, urgent, casual")


# Stage 2: Analyze the extracted information
class AnalysisResult(BaseModel):
    risk_level: str = Field(description="Risk assessment: low, medium, high")
    key_findings: list[str] = Field(description="2-3 most important findings")
    recommended_action: str = Field(description="What should be done next")


model = init_chat_model("gpt-4o")

# Build each stage
extract_prompt = ChatPromptTemplate.from_messages([
    ("system", "Extract key information from the following text."),
    ("human", "{text}"),
])

analyze_prompt = ChatPromptTemplate.from_messages([
    ("system", "Analyze the extracted information and assess risk."),
    ("human", "Entities: {entities}\nTopics: {topics}\nTone: {tone}"),
])

extract_chain = extract_prompt | model.with_structured_output(ExtractedInfo)
analyze_chain = analyze_prompt | model.with_structured_output(AnalysisResult)

# Wire them together
full_chain = (
    extract_chain
    | RunnablePassthrough.assign(
        entities=lambda x: ", ".join(x.entities),
        topics=lambda x: ", ".join(x.topics),
        tone=lambda x: x.tone,
    )
    | analyze_chain
)

result = full_chain.invoke({"text": "URGENT: Multiple users reporting data loss after the latest deployment. "
                                    "The engineering team at Acme Corp is investigating but no ETA yet."})
print(result.risk_level)           # "high"
print(result.recommended_action)   # "Immediate rollback and user communication..."

Tool Calling With Structured Output​

Combine structured output with tools for agents that return typed data:

from pydantic import BaseModel, Field
from langchain.tools import tool
from langchain.chat_models import init_chat_model
from langchain.agents import create_agent


# Tool that returns structured data
class WeatherInfo(BaseModel):
    temperature_c: float = Field(description="Current temperature in Celsius")
    condition: str = Field(description="Weather condition: sunny, cloudy, rainy, stormy, snowy")
    humidity_pct: int = Field(description="Humidity percentage")


@tool("get_weather", returning_type=WeatherInfo)
def get_weather(city: str) -> WeatherInfo:
    """Get current weather for a city. Returns structured weather data."""
    # In production, this calls a weather API
    return WeatherInfo(temperature_c=22.5, condition="sunny", humidity_pct=45)


model = init_chat_model("gpt-4o")
agent = create_agent(
    model=model,
    tools=[get_weather],
    prompt="You are a helpful assistant that uses weather data to answer questions.",
)

result = agent.invoke({"messages": [("user", "What's the weather in Lisbon?")]})

Parallel Structured Outputs​

Run independent structured output calls in parallel using RunnableParallel:

from pydantic import BaseModel, Field
from langchain.chat_models import init_chat_model
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnableParallel


class Entities(BaseModel):
    people: list[str] = Field(description="People mentioned")
    organizations: list[str] = Field(description="Organizations mentioned")
    locations: list[str] = Field(description="Locations mentioned")


class Sentiment(BaseModel):
    overall: str = Field(description="Overall sentiment: positive, negative, neutral")
    confidence: float = Field(description="Confidence between 0.0 and 1.0")


class Summary(BaseModel):
    headline: str = Field(description="One-line headline summarizing the article")
    key_points: list[str] = Field(description="2-3 key takeaways")


model = init_chat_model("gpt-4o")

# Build independent chains
entity_chain = ChatPromptTemplate.from_messages([
    ("system", "Extract named entities from the text."),
    ("human", "{text}"),
]) | model.with_structured_output(Entities)

sentiment_chain = ChatPromptTemplate.from_messages([
    ("system", "Classify the sentiment of the text."),
    ("human", "{text}"),
]) | model.with_structured_output(Sentiment)

summary_chain = ChatPromptTemplate.from_messages([
    ("system", "Summarize the article concisely."),
    ("human", "{text}"),
]) | model.with_structured_output(Summary)

# Run all three in parallel
parallel_chain = RunnableParallel({
    "entities": entity_chain,
    "sentiment": sentiment_chain,
    "summary": summary_chain,
})

article = """
OpenAI announced GPT-5 today at their San Francisco headquarters.
The new model shows significant improvements in reasoning capabilities.
Industry analysts are cautiously optimistic about the release.
"""

results = parallel_chain.invoke({"text": article})
print(results["entities"].people)       # []
print(results["entities"].organizations)  # ["OpenAI"]
print(results["sentiment"].overall)      # "positive"
print(results["summary"].headline)       # "OpenAI Announces GPT-5..."

Streaming With Structured Output​

Structured output supports streaming. The model generates the full JSON before returning, but you get progress events:

from pydantic import BaseModel, Field
from langchain.chat_models import init_chat_model


class CodeSuggestion(BaseModel):
    issue: str = Field(description="The problem found in the code")
    severity: str = Field(description="Severity: low, medium, high")
    fix: str = Field(description="The corrected code snippet")


model = init_chat_model("gpt-4o")
structured = model.with_structured_output(CodeSuggestion)

# Stream events to see progress
for event in structured.stream_events(
    "Review this Python code: def get_user(id): return db.query('SELECT * FROM users WHERE id=' + id)",
    include_namespaces=True,
):
    event_type = event["event"]
    if event_type == "on_chat_model_stream":
        chunk = event["data"]["chunk"]
        if chunk.content:
            print(chunk.content, end="", flush=True)
    elif event_type == "on_chain_end":
        result = event["data"]["output"]
        print(f"\n\n--- Parsed result ---")
        print(f"Issue: {result.issue}")
        print(f"Severity: {result.severity}")
        print(f"Fix: {result.fix}")
note

Streaming with structured output does not give you partial Pydantic objects. The model must complete the full JSON response before validation runs. Streaming is useful for UX (showing progress) but the typed result arrives atomically.

Error Handling​

Structured output fails when the LLM generates JSON that does not match the schema. You must handle these cases.

Understanding OutputParserException​

When Pydantic validation fails, LangChain raises an OutputParserException:

from langchain_core.output_parsers import OutputParserException
from pydantic import BaseModel, Field, ValidationError
from langchain.chat_models import init_chat_model


class StrictAnalysis(BaseModel):
    score: int = Field(description="A score from 1 to 10", ge=1, le=10)
    label: str = Field(description="Classification label")


model = init_chat_model("gpt-4o")
structured = model.with_structured_output(StrictAnalysis)

try:
    result = structured.invoke("Rate this: something mediocre")
    print(result.score)
except OutputParserException as e:
    print(f"Structured output failed: {e}")
    # Log the raw response for debugging
    if hasattr(e, "llm_output"):
        print(f"Raw LLM output: {e.llm_output}")

What Triggers Validation Failure​

CauseExample
Wrong typeLLM returns "score": "high" instead of "score": 7
Missing required fieldLLM omits label from the JSON
Constraint violationLLM returns "score": 15 when le=10
Malformed JSONLLM wraps JSON in markdown or adds commentary
Hallucinated fieldLLM adds a field not in the schema (usually fine — ignored)

Retry Strategies With Fallback Schemas​

When the primary schema is too strict, fall back to a relaxed version:

from pydantic import BaseModel, Field
from langchain_core.output_parsers import OutputParserException
from langchain.chat_models import init_chat_model


class StrictReview(BaseModel):
    rating: int = Field(description="Rating from 1 to 5", ge=1, le=5)
    sentiment: str = Field(description="positive, negative, or neutral")
    summary: str = Field(description="One-sentence summary")


class RelaxedReview(BaseModel):
    rating: str = Field(description="A numeric rating if available, otherwise 'unknown'")
    sentiment: str = Field(description="positive, negative, neutral, or unknown")
    summary: str = Field(description="Summary of the review, or 'no summary available'")


model = init_chat_model("gpt-4o")


def analyze_review(text: str) -> StrictReview | RelaxedReview:
    """Try strict schema first, fall back to relaxed schema."""
    try:
        strict_model = model.with_structured_output(StrictReview)
        return strict_model.invoke(f"Analyze this review: {text}")
    except OutputParserException:
        relaxed_model = model.with_structured_output(RelaxedReview)
        return relaxed_model.invoke(f"Analyze this review (best effort): {text}")


result = analyze_review("This product is okay I guess, nothing special really.")
print(type(result).__name__)  # "StrictReview" or "RelaxedReview"

Graceful Degradation Pattern​

For production systems, design chains that degrade gracefully rather than crash:

from pydantic import BaseModel, Field
from langchain_core.output_parsers import OutputParserException
from langchain.chat_models import init_chat_model
import logging

logger = logging.getLogger(__name__)


class ArticleAnalysis(BaseModel):
    topic: str = Field(description="Main topic of the article")
    entities: list[str] = Field(description="Key entities mentioned")
    sentiment: str = Field(description="positive, negative, or neutral")


def analyze_with_fallback(text: str) -> dict:
    """Analyze text with graceful degradation."""
    model = init_chat_model("gpt-4o")
    structured = model.with_structured_output(ArticleAnalysis)

    try:
        result = structured.invoke(f"Analyze: {text}")
        return {
            "status": "success",
            "topic": result.topic,
            "entities": result.entities,
            "sentiment": result.sentiment,
        }
    except OutputParserException as e:
        logger.warning("Structured output failed, falling back to free-form", exc_info=e)
        # Fallback: get free-text and return partial data
        free_form = model.invoke(f"Tell me the main topic of: {text}")
        return {
            "status": "degraded",
            "topic": free_form.content[:100],  # Best effort
            "entities": [],
            "sentiment": "unknown",
        }

Logging Malformed Responses for Langfuse​

Log failed structured output attempts so you can analyze patterns:

from langchain_core.output_parsers import OutputParserException
import json
import logging

logger = logging.getLogger(__name__)


def structured_invoke_with_logging(structured_model, input_text: str, run_id: str):
    """Invoke structured output with detailed logging on failure."""
    try:
        return structured_model.invoke(input_text)
    except OutputParserException as e:
        # Log the failure with context for Langfuse or your observability tool
        failure_data = {
            "run_id": run_id,
            "error_type": "structured_output_validation",
            "error_message": str(e),
            "input_preview": input_text[:200],
        }
        logger.error("Structured output validation failed", extra=failure_data)

        # If using Langfuse, you can also log to the trace:
        # langfuse_span.event(
        #     name="structured_output_failure",
        #     data=failure_data,
        #     level="ERROR",
        # )
        raise
warning

Always log the raw LLM response when structured output fails. Without it, you cannot diagnose whether the problem is your schema, your prompt, or the model's capability.

Structured Output in LangGraph​

When building agents with LangGraph, structured output becomes a core pattern for node returns and state management.

Using Structured Output in Agent Nodes​

Each node in a LangGraph agent can return structured data:

from pydantic import BaseModel, Field
from langchain.chat_models import init_chat_model
from langchain_core.prompts import ChatPromptTemplate
from langgraph.graph import StateGraph, START, END
from typing import TypedDict


class ExtractionResult(BaseModel):
    entities: list[str] = Field(description="Named entities extracted")
    claims: list[str] = Field(description="Factual claims made in the text")
    uncertainties: list[str] = Field(description="Statements that need verification")


class AgentState(TypedDict):
    text: str
    extraction: ExtractionResult
    analysis: str


model = init_chat_model("gpt-4o")


def extract_node(state: AgentState) -> dict:
    """Extract structured information from the input text."""
    prompt = ChatPromptTemplate.from_messages([
        ("system", "Extract entities, claims, and uncertainties from the text."),
        ("human", "{text}"),
    ])
    chain = prompt | model.with_structured_output(ExtractionResult)
    result = chain.invoke({"text": state["text"]})
    return {"extraction": result}


def analyze_node(state: AgentState) -> dict:
    """Analyze the extracted information."""
    extraction = state["extraction"]
    prompt = ChatPromptTemplate.from_messages([
        ("system", "Analyze the extracted data and provide an assessment."),
        (
            "human",
            f"Entities: {extraction.entities}\n"
            f"Claims: {extraction.claims}\n"
            f"Uncertainties: {extraction.uncertainties}",
        ),
    ])
    response = model.invoke(prompt.format_messages())
    return {"analysis": response.content}


# Build the graph
graph = StateGraph(AgentState)
graph.add_node("extract", extract_node)
graph.add_node("analyze", analyze_node)
graph.add_edge(START, "extract")
graph.add_edge("extract", "analyze")
graph.add_edge("analyze", END)

app = graph.compile()

result = app.invoke({
    "text": "Apple announced a new chip designed in Cupertino that outperforms "
            "Intel's latest offering, according to benchmark tests."
})
print(result["extraction"].entities)  # ["Apple", "Cupertino", "Intel"]

Structured Tool Returns​

Tools in LangGraph agents can return structured data that flows through the graph state:

from pydantic import BaseModel, Field
from langchain.tools import tool


class SearchResults(BaseModel):
    query: str = Field(description="The search query used")
    results: list[str] = Field(description="Top search result titles")
    relevance_scores: list[float] = Field(description="Relevance score for each result, 0.0 to 1.0")
    total_found: int = Field(description="Total number of matching documents")


@tool("search_docs", returning_type=SearchResults)
def search_documents(query: str) -> SearchResults:
    """Search the documentation knowledge base."""
    # In production: call your vector search or Azure AI Search
    return SearchResults(
        query=query,
        results=["Getting Started with LangChain", "Structured Output Guide"],
        relevance_scores=[0.95, 0.82],
        total_found=147,
    )

State Schema With Pydantic Models​

Use Pydantic models as your LangGraph state schema for type safety:

from pydantic import BaseModel, Field
from typing import Annotated
from langgraph.graph.message import add_messages


class ProcessingState(BaseModel):
    """State schema for the document processing pipeline."""
    input_text: str = Field(description="The raw input document")
    extracted_entities: list[str] = Field(default_factory=list)
    analysis_complete: bool = Field(default=False)
    messages: Annotated[list, add_messages] = Field(default_factory=list)
    error: str | None = Field(default=None, description="Error message if processing failed")
tip

See LangGraph Getting Started for a full walkthrough of building stateful agents. Structured output fits naturally into the node-return pattern.

Testing Structured Output​

Untested structured output is a production risk. Here is how to test it properly.

Mock LLM Responses With FakeListChatModel​

Never call real models in tests. Use FakeListChatModel to return controlled JSON:

from langchain_community.chat_models import FakeListChatModel
from pydantic import BaseModel, Field


class SentimentResult(BaseModel):
    sentiment: str = Field(description="positive, negative, or neutral")
    score: float = Field(description="Score from 0.0 to 1.0")


def test_structured_output_with_valid_json():
    """Test that valid JSON from the LLM is parsed correctly."""
    fake_model = FakeListChatModel(
        responses=['{"sentiment": "positive", "score": 0.92}']
    )
    structured = fake_model.with_structured_output(SentimentResult)

    result = structured.invoke("Analyze: This product is amazing!")

    assert isinstance(result, SentimentResult)
    assert result.sentiment == "positive"
    assert result.score == 0.92


def test_structured_output_with_invalid_json():
    """Test that malformed JSON raises OutputParserException."""
    from langchain_core.output_parsers import OutputParserException

    fake_model = FakeListChatModel(
        responses=["This is not JSON at all"]
    )
    structured = fake_model.with_structured_output(SentimentResult)

    try:
        structured.invoke("Analyze: Something")
        assert False, "Should have raised OutputParserException"
    except OutputParserException:
        pass  # Expected

Testing Schema Validation​

Test that your Pydantic models enforce the constraints you expect:

from pydantic import BaseModel, Field, ValidationError


class Rating(BaseModel):
    value: int = Field(description="Rating from 1 to 5", ge=1, le=5)


def test_rating_enforces_bounds():
    """Test that the schema rejects out-of-range values."""
    assert Rating(value=3).value == 3

    try:
        Rating(value=0)
        assert False, "Should reject value below minimum"
    except ValidationError:
        pass

    try:
        Rating(value=6)
        assert False, "Should reject value above maximum"
    except ValidationError:
        pass


def test_rating_rejects_wrong_type():
    """Test that the schema rejects non-integer values."""
    try:
        Rating(value="high")
        assert False, "Should reject string values"
    except ValidationError:
        pass

Testing Error Recovery​

Test your fallback and retry logic:

from langchain_community.chat_models import FakeListChatModel
from langchain_core.output_parsers import OutputParserException
from pydantic import BaseModel, Field


class StrictModel(BaseModel):
    score: int = Field(description="Integer score", ge=1, le=10)


class RelaxedModel(BaseModel):
    score: str = Field(description="Score as string")


def test_fallback_on_validation_failure():
    """Test that fallback schema is used when strict schema fails."""
    # First model returns invalid data for strict schema
    bad_model = FakeListChatModel(responses=['{"score": "not_a_number"}'])
    # Second model returns valid data for relaxed schema
    good_model = FakeListChatModel(responses=['{"score": "7"}'])

    # Simulate retry logic
    strict_structured = bad_model.with_structured_output(StrictModel)
    try:
        strict_structured.invoke("Rate this")
        assert False, "Should fail"
    except OutputParserException:
        # Fall back to relaxed schema
        relaxed_structured = good_model.with_structured_output(RelaxedModel)
        result = relaxed_structured.invoke("Rate this")
        assert result.score == "7"

Golden Dataset Integration​

Test structured output against a golden dataset of inputs and expected outputs:

import pytest
from langchain_community.chat_models import FakeListChatModel
from pydantic import BaseModel, Field


class Classification(BaseModel):
    category: str = Field(description="One of: billing, technical, account, general")
    urgency: str = Field(description="One of: low, medium, high, critical")


GOLDEN_TEST_CASES = [
    {
        "input": "My credit card was charged twice",
        "expected_category": "billing",
        "expected_urgency": "high",
        "mock_response": '{"category": "billing", "urgency": "high"}',
    },
    {
        "input": "How do I reset my password?",
        "expected_category": "account",
        "expected_urgency": "medium",
        "mock_response": '{"category": "account", "urgency": "medium"}',
    },
    {
        "input": "The app crashes when I open settings",
        "expected_category": "technical",
        "expected_urgency": "high",
        "mock_response": '{"category": "technical", "urgency": "high"}',
    },
]


@pytest.mark.parametrize("test_case", GOLDEN_TEST_CASES)
def test_classification_against_golden_dataset(test_case):
    """Test structured output against golden test cases."""
    fake_model = FakeListChatModel(responses=[test_case["mock_response"]])
    structured = fake_model.with_structured_output(Classification)

    result = structured.invoke(f"Classify: {test_case['input']}")

    assert result.category == test_case["expected_category"]
    assert result.urgency == test_case["expected_urgency"]
info

For golden dataset testing in production, integrate with Langfuse to continuously evaluate structured output quality against labeled examples.

Cost and Performance​

Structured output has real tradeoffs. Understand them before reaching for it.

Token Overhead of Schema Injection​

Every structured output call injects the JSON Schema into the system prompt. This adds tokens:

Schema ComplexityApprox. Token OverheadExample
Simple (3–5 fields)~50–100 tokensSentimentAnalysis with sentiment, score, reasoning
Medium (6–10 fields)~100–200 tokensTicketClassification with category, urgency, summary, tags
Complex (10+ fields, nested)~200–500+ tokensMulti-entity extraction with nested lists

For a model like GPT-4o at $2.50/1M input tokens, a medium schema adds roughly $0.00025 per call. Negligible for most applications, but noticeable at scale.

Retry Cost When Validation Fails​

When structured output fails and you retry, you pay for the failed call plus the retry:

First call (fails validation):    $X
Retry call (succeeds):            $X
Total cost:                       $2X  (double)

Minimize retry cost by:

  • Using clear Field(description=...) values.
  • Testing schemas before production deployment.
  • Using Literal types for constrained fields.

When to Skip Structured Output​

Structured output is not always the right tool:

ScenarioRecommendation
Human-facing output (chat responses, email drafts)Free-form text. Structured output adds overhead with no benefit.
Simple classification (one-word answer)Free-form text with a clear system prompt.
Creative generation (stories, marketing copy)Free-form text. Schemas constrain creativity.
Downstream processing requiredStructured output. Guaranteed shape saves parsing code.
Data extraction (entities, fields, records)Structured output. Type safety prevents runtime errors.
Agent tool returnsStructured output. Typed returns make state management cleaner.

Performance Comparison​

PatternLatencyCostReliabilityBest For
Free-form + regex parsingFastLowestLow — regex breaks oftenHuman-facing output
Structured outputSame as free-form+50–200 tokensHigh — validated by PydanticDownstream processing
Structured output + retry2x on failure2x on failureHighest — retry covers edge casesCritical pipelines
Two-stage (extract then validate)2x always2x alwaysModerate — complex to maintainRarely justified
note

The latency of structured output is the same as free-form text — the model generates the same amount of tokens. The only difference is that the response goes through Pydantic validation, which adds microseconds of CPU time.

Key Takeaways​

PrincipleAction
Always use descriptionsEvery Field() gets a description= that tells the LLM what to put there.
Keep schemas simpleFlat structures, 1–2 levels of nesting max. Use Literal for constrained choices.
Handle failuresCatch OutputParserException, implement fallback schemas, log raw responses.
Test with mocksFakeListChatModel for controlled responses. Golden datasets for regression testing.
Measure overheadSchema injection adds tokens. At high volume, profile the cost.
Skip when unnecessaryHuman-facing output does not need structured output.

Structured output with Pydantic is the foundation for reliable LLM applications. It transforms creative text generation into predictable, type-safe data pipelines.

Further Reading​