Building JSON Chat Agents in LangChain

The create_json_chat_agent function in LangChain provides a powerful way to create agents that use JSON formatting for their decision-making process. These agents are specifically built to work with chat models and can interact with various tools while maintaining a structured conversation flow.

What is a JSON Chat Agent?

A JSON Chat Agent is a specialized type of LangChain agent that:

• Uses JSON formatting to structure its responses and actions
• Works with chat-based language models
• Can use multiple tools to accomplish tasks
• Maintains conversation context through chat history
• Makes decisions based on a prompt template and tool outputs

Reference

How to Use JSON Chat Agents

Basic Setup

Here's how to create a simple JSON chat agent:

from langchain import hub
from langchain_community.chat_models import ChatOpenAI
from langchain.agents import AgentExecutor, create_json_chat_agent

# Initialize the chat model
model = ChatOpenAI()

# Get the default JSON chat prompt
prompt = hub.pull("hwchase17/react-chat-json")

# Define tools the agent can use
tools = [...] # Add your tools here

# Create the agent
agent = create_json_chat_agent(model, tools, prompt)

# Create an executor to run the agent
agent_executor = AgentExecutor(agent=agent, tools=tools)

Simple Interactions

You can interact with the agent using simple inputs:

# Simple query
response = agent_executor.invoke({
    "input": "hi"
})

Working with Chat History

The agent can maintain context through chat history:

from langchain_core.messages import AIMessage, HumanMessage

# Query with chat history
response = agent_executor.invoke({
    "input": "what's my name?",
    "chat_history": [
        HumanMessage(content="hi! my name is bob"),
        AIMessage(content="Hello Bob! How can I assist you today?"),
    ],
})

Creating Custom Prompts

You can create custom prompts for your agent:

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder

system_message = """You are an AI assistant that helps users by taking actions 
and providing responses in JSON format."""

human_template = """TOOLS
------
Available tools:
{tools}

RESPONSE FORMAT
-------------
Respond with a JSON blob in this format:
\`\`\`json
{
    "action": string,  // Either a tool name or "Final Answer"
    "action_input": string  // The input for the tool or your final response
}
\`\`\`

USER'S INPUT
-----------
{input}"""

prompt = ChatPromptTemplate.from_messages([
    ("system", system_message),
    MessagesPlaceholder("chat_history", optional=True),
    ("human", human_template),
    MessagesPlaceholder("agent_scratchpad"),
])

# Create agent with custom prompt
agent = create_json_chat_agent(model, tools, prompt)

Error Handling

When creating agents, make sure to handle potential errors:

# Create agent executor with error handling
agent_executor = AgentExecutor(
    agent=agent, 
    tools=tools,
    verbose=True,
    handle_parsing_errors=True  # Handles JSON parsing errors gracefully
)

The JSON Chat Agent pattern is particularly useful when you need structured responses from your agent or when integrating with systems that expect JSON formatted data. The agent's responses will always be in a predictable JSON format, making it easier to process and handle the outputs programmatically.