10 Alternatives to LangChain for Calling LLMs in Python

Posted: Nov 1, 2024.

While LangChain can be useful for prototyping projects, it often becomes overkill for long-term production applications. Its extensive abstractions and complex architecture can make debugging challenging, especially when dealing with edge cases or trying to understand what's happening under the hood.

Many developers find that a lighter-weight solution would suffice. The complexity of LangChain's architecture can often get in the way rather than help.

In this article, we'll explore ten simpler alternatives to LangChain that might better suit your needs while keeping your code maintainable and easy to reason about.

1. Vanilla Python

By using Vanilla Python developers can interact with LLM APIs like OpenAI, Cohere, or Anthropic without relying on external frameworks. This method offers maximum control over the codebase and is perfect for developers who value flexibility over convenience.

Key feature:s

Full control over your codebase.
No dependencies, which reduces overhead.
Flexibility in designing workflows and precise tailoring.

import requests

# Set up your API key and endpoint
api_key = "YOUR_OPENAI_API_KEY"  # replace with your actual API key
url = "https://api.openai.com/v1/chat/completions"

# Set up the request headers and data
headers = {
    "Authorization": f"Bearer {api_key}",
    "Content-Type": "application/json"
}

data = {
    "model": "gpt-3.5-turbo",  # specify the model you'd like to use
    "messages": [
        {"role": "user", "content": "Tell me a joke."}
    ],
    "temperature": 0.7  # controls randomness in the output; lower values make it more focused
}

# Make the API request
response = requests.post(url, headers=headers, json=data)

# Process the response
if response.status_code == 200:
    response_json = response.json()
    print("AI Response:", response_json['choices'][0]['message']['content'])
else:
    print("Request failed:", response.status_code, response.text)

You get a simple and streamlined approach without the need to worry about breaking updates from a third-party library. This method is ideal for those who prefer a minimalistic approach and want to understand how LLM interactions work under the hood.

While this may require more upfront work but it allows you to build exactly what you need without unnecessary features or restrictions.

2. LiteLLM

Star

LiteLLM is a Python SDK and proxy server that simplifies interactions with various LLM APIs such as OpenAI, Hugging Face and Cohere. With LiteLLM you can easily switch between different LLM providers without having to change the core code.

Key features:

Unified API for multiple LLM providers.
Automatic retry and fallback logic to handle API errors.
Support for streaming and asynchronous calls.

pip install litellm

from litellm import completion
api_key = "YOUR_OPENAI_API_KEY"  # replace with your actual API key
# Define the prompt
prompt = "Explain the theory of relativity in simple terms."

# Call the completion function
response = completion(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": prompt}],
    api_key=api_key,
)

# Print the response
print(response.choices[0].message.content)

LiteLLM helps to transition from one LLM provider to another which is useful when cost or performance considerations necessitate a change. Its built-in retry mechanisms ensure that your applications remain resilient when encountering rate limits or network issues.

3. LlamaIndex

Star

LlamaIndex is a data framework designed to integrate LLMs with external data sources effectively. It is known for efficient document indexing and retrieval for large-scale data processing.

Key features:

Supports complex data queries and indexing.
Seamless integration with multiple vector stores.
Useful for applications needing complex data processing pipelines.

pip install llama-index

import os
import openai

os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY"
openai.api_key = os.environ["OPENAI_API_KEY"]

from llama_index.llms.openai import OpenAI

llm = OpenAI(model="gpt-3.5-turbo")

response = llm.complete("Hi, write a short story")
print(response)

LlamaIndex is beneficial for creating knowledge-based systems such as customer support bots or research assistants.

4. Promptify

Star

Promptify is a specialized library that focuses on enhancing prompt engineering for LLMs. With tools to create, test and optimize prompts, Promptify helps you achieve more accurate and relevant outputs.

Key features:

Advanced prompt engineering tools.
Support for prompt chaining to create multi-step LLM interactions.
Compatibility with multiple LLMs, including GPT-3 and GPT-4.

pip3 install promptify

from promptify import Prompter,OpenAI, Pipeline

api_key = "YOUR_OPENAI_API_KEY"
sentence     =  """The patient is a 93-year-old female with a medical  				 
                history of chronic right hip pain, osteoporosis,					
                hypertension, depression, and chronic atrial						
                fibrillation admitted for evaluation and management				
                of severe nausea and vomiting and urinary tract				
                infection"""

model        = OpenAI(api_key=api_key) # or `HubModel()` for Huggingface-based inference or 'Azure' etc
prompter     = Prompter('ner.jinja') # select a template or provide custom template
pipe         = Pipeline(prompter , model)


result = pipe.fit(sentence, domain="medical", labels=None)

Promptify's emphasis on prompt engineering can save time by ensuring that prompts are structured to yield the best responses.

5. Langroid

Star

Langroid is an intuitive and lightweight framework from researchers at CMU and UW-Madison. It is designed to easily build LLM-powered applications using Agents that can solve problems collaboratively.

Key features:

Extensible multi-agent framework for solving tasks.
Integration with vector stores and optional components like tools and functions.
Inspired by the Actor Framework but easy to understand for newcomers.

pip install langroid

import os
os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY"

import langroid.language_models as lm

# set up LLM
llm_cfg = lm.OpenAIGPTConfig(
  chat_model=lm.OpenAIChatModel.GPT4_TURBO, # or, e.g., "ollama/mistral"
)
# use LLM directly
mdl = lm.OpenAIGPT(llm_cfg)
response = mdl.chat("What is the capital of Ontario?", max_tokens=10)

Langroid brings an approachable framework for distributed problem-solving making it a good fit for AI-driven applications like collaborative learning environments, automated assistants and multi-step decision-making systems.

6. HayStack

Star

HayStack is an open-source framework aimed at building search and question-answering systems. It allows developers to create pipelines for document retrieval, question answering and summarization.

Key features:

Flexible pipeline creation for complex LLM-based applications.
Integration with multiple backends such as Elasticsearch and FAISS.
Customizable components for specialized tasks.

pip install haystack-ai

import os

from haystack import Pipeline, PredefinedPipeline
import urllib.request

os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY"
urllib.request.urlretrieve(
    "https://archive.org/stream/leonardodavinci00brocrich/leonardodavinci00brocrich_djvu.txt",
    "davinci.txt",
)

indexing_pipeline = Pipeline.from_template(PredefinedPipeline.INDEXING)
indexing_pipeline.run(data={"sources": ["davinci.txt"]})

rag_pipeline = Pipeline.from_template(PredefinedPipeline.RAG)

query = "How old was he when he died?"
result = rag_pipeline.run(
    data={"prompt_builder": {"query": query}, "text_embedder": {"text": query}}
)
print(result["llm"]["replies"][0])

The framework’s ability to integrate with multiple backends makes it extremely flexible for developers dealing with disparate data sources or needing to scale their search solutions.

7. AutoChain

Star

AutoChain offers a lightweight framework that helps developers build generative agents quickly. It focuses on simplicity and visualization for rapid iteration and debugging of prompt interactions.

Key features:

Emphasizes ease of prompt updates and visualization.
Supports multiple LLM providers and integrates easily with external tools.
Active community with resources and tutorials.

pip install autochain

import os
# Set environment variables
os.environ['OPENAI_API_KEY'] = 'YOUR_OPENAI_API_KEY'

from autochain.chain.chain import Chain
from autochain.memory.buffer_memory import BufferMemory
from autochain.models.chat_openai import ChatOpenAI
from autochain.agent.conversational_agent.conversational_agent import ConversationalAgent

llm = ChatOpenAI(temperature=0)
memory = BufferMemory()
agent = ConversationalAgent.from_llm_and_tools(llm=llm)
chain = Chain(agent=agent, memory=memory)

print(chain.run("Write me a poem about AI")['message'])

Autochain has the ability to quickly see how changes to a prompt impact the output and it helps in developing LLM-driven products without unnecessary complexities.

8. Griptape AI

Star

Griptape AI is a versatile framework designed for building AI-powered applications focussing on balancing predictability and creativity. It provides advanced capabilities such as pipelines, workflows and memory management.

Key features:

Offers predictable task sequencing using directed acyclic graphs (DAGs) and memory structures.
Safely connects LLMs to external APIs and data stores with tools for image generation, SQL queries and web scraping.
Enforces schema validation and activity-level permissions while handling big data off-prompt for secure and efficient processing.

pip install "griptape[all]"

import os
os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY"

from griptape.structures import Agent
agent = Agent()

# Run the agent
agent.run("Give me a haiku about skateboarding")

9. Microsoft AutoGen

Star

Microsoft AutoGen introduces a multi-agent conversation framework that builds LLM workflows involving agents that interact with each other, external tools and even humans. This model makes it easier to orchestrate complex workflows.

Key features:

Multi-agent collaboration with teachability and personalization features.
Ideal for creating systems that adapt to user preferences over time.
Active support community.

pip install autogen-agentchat

from autogen import ConversableAgent

agent = ConversableAgent(
    "chatbot",
    llm_config={
        "config_list": [
            {
                "model": "gpt-4",
                "api_key": "YOUR_OPENAI_API_KEY",
            }
        ]
    },
    code_execution_config=False,  # Turn off code execution, by default it is off.
    function_map=None,  # No registered functions, by default it is None.
    human_input_mode="NEVER",  # Never ask for human input.
)

reply = agent.generate_reply(messages=[{"content": "Tell me a joke.", "role": "user"}])
print(reply)

Autogen is useful if your application demands interaction between multiple components or if teachability is a major concern.

10. Mirascope

Star

Mirascope is a powerful and flexible library offering a unified interface to interact with numerous providers including OpenAI, Anthropic, Mistral, Gemini and many others.

Key features:

Compatible with multiple LLM providers through a unified API.
User-friendly abstractions for developing LLM-driven applications.
Strong community support with various resources for learning.

pip install "mirascope[openai]"

import os
# Set environment variables
os.environ['OPENAI_API_KEY'] = 'YOUR_OPENAI_API_KEY'

from mirascope.core import openai
from pydantic import BaseModel

class Book(BaseModel):
    title: str
    author: str

@openai.call("gpt-4o-mini", response_model=Book)
def extract_book(text: str) -> str:
    return f"Extract {text}"

book = extract_book("The Name of the Wind by Patrick Rothfuss")
assert isinstance(book, Book)
print(book)

Its user-friendly abstractions make it easier to implement tasks like text generation or building conversational bots.

Building an AI chatbot?

Open-source GenAI monitoring, prompt management, and magic.

Learn More

Join 10,000+ subscribers

Every 2 weeks, latest model releases and industry news.

Building an AI chatbot?

Open-source GenAI monitoring, prompt management, and magic.

Learn More