9 Open Source tools to take your AI app to the next level

The AI ecosystem is booming.

If you've started building AI-powered apps, you might have missed some of those awesome open-source projects that can help you make your LLM queries, more relevant and improve the general quality of your chatbots and AI apps.

Here are 9 projects to take your app to the next level in 2023.

Lunary (sponsored)

Lunary is an all-in-one open-source toolkit for AI devs ready to take their app to production, with features such as:

• 💵 Cost & latency analytics
• 👪 Users tracking
• 🐛 Traces to debug easily
• 🔍 Inspect & replay AI requests
• 🏷️ Label and export fine-tuning datasets
• 🖲️ Collect feedback from users
• 🧪 Evaluate prompts against tests

Once you're ready for users to try your app, using an observability solution is essential.

We invite you to give Lunary a shot (it's completely free up to 1000 events / day).

⭐ Check out on GitHub

Guidance

Guidance is a new language format released by Microsoft allowing to create complex agent flows. It looks like this:

experts = guidance('''
{{#system~}}
You are a helpful and terse assistant.
{{~/system}}

{{#user~}}
I want a response to the following question:
{{query}}
Name 3 world-class experts (past or present) who would be great at answering this?
Don't answer the question yet.
{{~/user}}

{{#assistant~}}
{{gen 'expert_names' temperature=0 max_tokens=300}}
{{~/assistant}}

{{#user~}}
Great, now please answer the question as if these experts had collaborated in writing a joint anonymous answer.
{{~/user}}

{{#assistant~}}
{{gen 'answer' temperature=0 max_tokens=500}}
{{~/assistant}}
''', llm=gpt4)

This interleaving of generation and prompting allows for precise output structure that might help produce clear and parsable results.

⭐ Check out on GitHub

LiteLLM

Call any LLM API using the OpenAI format (Bedrock, Huggingface, Cohere, TogetherAI, Azure, OpenAI, etc.)

For example, integrating Anthropic will look like this:

from litellm import completion
import os

## set ENV variables
os.environ["ANTHROPIC_API_KEY"] = "your-api-key"

response = completion(
  model="claude-2", 
  messages=[{ "content": "Hello, how are you?","role": "user"}]
)

This way, you can integrate different models in your app without learning and intergrating new APIs.

⭐ Check out on GitHub

Zep

Zep allows you to summarize, embed, and enhance chat histories and documents asynchronously. It ensures that these operations do not impact the chat experience of your users.

With Zep, chatbot histories are persisted to a database, enabling you to easily scale out as your user base grows.

As a drop-in replacement for popular LangChain components, Zep allows you to get your application into production within minutes, without the need to rewrite your existing code.

⭐ Check out on GitHub

LangChain

Who hasn't heard of LangChain by the now? LangChain is the most popular AI framework allowing you to plug together models into chains, with vector stores and more to build powerful AI apps.

⭐ Check out on GitHub

DeepEval

DeepEval is an evaluation framework designed for LLM apps that is straightforward to use and available open-source.

It functions similarly to Pytest, but with a specialization in testing LLM applications. DeepEval assesses using metrics like factual consistency, accuracy, and answer relevancy utilizing LLMs along with various other NLP models.

⭐ Check out on GitHub

pgVector

pgVector is a Postgres extension to store your embeddings and perform operations such as similarity search.

If you're using Supabase, pgVector is already available.

You could use pgVector instead of specialized vector databases Pinecone to simplify your stack.

⭐ Check out on GitHub

promptfoo

With promptfoo, similarly to deepEval, you can test your prompts & models against predefined test cases.

Evaluate quality and catch regressions by comparing LLM outputs side-by-side, score outputs automatically by defining test cases.

⭐ Check out on GitHub

Model Fusion

Model Fusion is a TypeScript library designed for building AI applications, chatbots, and agents.

It offers support for a wide range of models, including text generation, image generation, text-to-speech, speech-to-text, and embedding models.

Features:

• Multimodal: combine different modalities such as text, images, and speech.
• Streaming: Model Fusion supports streaming for many generation models, including text streaming, structure streaming, and full duplex speech streaming.
• Utility: Model Fusion provides a set of utility functions for tools and tool usage, vector indices, and guard functions.
• Type inference and validation: Model Fusion leverages TypeScript to infer types and validate model responses.

Useful if you're prefer Typescript to Python.

Model fusion is a quite new but very promising project.

⭐ Check out on GitHub

End

Thank you for reading!

A star on our GitHub project would mean the world 🙏

Click on the cat to make him happy 👇