Roadmap

Here’s a preview of the key features and improvements coming soon to Lunary.
Check out our recent updates on the changelog.
Got a feature request or found a bug? Contact us via live chat.
June 2024
SDK methods to pull all prompts at once(in progress)
Analytics v2(in progress)

The new version of Analytics will contain many new charts and insights, the ability to filter data by tags, metadata or users, as well as a new hourly breakdown of historic data.

LLamaIndex Python support

Our most requested integration, LLamaIndex, is being worked on.

Separate user feedback and operator feedback

In the details of a run, it should be possible to see the difference between feedback from operators and feedback from end users.

Dynamic Datasets: Import Prompts into datasets

That will make it easier to run evaluations on Prompts directly. Tie Prompts and Evaluations together.

We will allow using prompts in Datasets that are always up to date with their latest version. We'll also add a button "Add to dataset" in the Prompt directory

Switch homemade Evaluators (such as PII) to more scalable infrastructure

To keep up with growth, we need to switch PII scanning and sentiment analysis evaluators into a more scalable infrastructure.

Visualize results of evaluations ran via SDK in dashboard

We need to be able to see the results of custom evaluators ran from SDK in the dashboard.

Saved Views

Save combination of filters into reusable "Views".

July 2024
Show latest changelog items on dashboard

That will help make it easier for users to keep up with the latest changes.

Allow manually changing theme from UI

Toggle between light and dark theme manually, separately from the system theme.

Allow custom inference API keys for LLM evaluators
Prompts: add custom tags to versions

By adding custom tag to prompt versions, we can enable use cases such as A/B testing different variations of a prompt and serving different versions of a prompt to different users.

Add date range filter for logs
Add all Evaluation methods to JS SDK

We will bring the JS SDK to feature parity with the Python SDK.

Simplification of Evaluation SDK methods

We will deprecate the concept of Checklists, replaced by a new "Evaluators" concept. Assertions will now be done in the code instead of the dashboard, allowing for more flexibility of use-cases.

Support for LlamaIndex TS
Automated email digests

With all the numbers of new analytics: new users, total logs, costs split by model, total chat threads, average latency, …

Documentation for all API resources endpoints

We will publish the full documentation for the Logs, Analytics, Users, Prompts, Evaluations and Datasets endpoints.

Improve performances of search
August 2024
Alerts v2

Create fully custom email and webhook alerts on views with match thresholds.

Upgrade to PostgreSQL 16
Integration with OpenAI's Assistants API
Automatic instrumentation of OpenAI embeddings