Changelog

Run model-based evaluations on traces in Langfuse to scale your evaluation workflows. Start with one of our battletested templates or use your own custom templates.

New editor, metadata on all objects, tables that render i/o, dataset runs on traces, push traces to datasets, and more.

The decorator now supports all Langfuse features and is the recommended way to use Langfuse in Python.

Test and iterate on your prompts with the new LLM playground directly in Langfuse.

We teamed up with PostHog to integrate LLM-related product metrics into your existing PostHog dashboards.

Langfuse now provides a simple to adopt wrapper for the OpenAI JS/TS SDK for you to seemlessly trace your OpenAI calls.

From now on, we will publish Docker images on Docker Hub and the GitHub Container Registry.

Langfuse's Information Security program has been certified to the ISO 27001:2022 standard.

The OpenAI SDK integration for Langfuse Tracing now supports capturing the used prompt (version) from Prompt Management.

Introducing the "chat" prompt type in Langfuse Prompt Management to version control and release ChatML (role/message) prompts.

With this release, Langfuse Tracing integrates natively with all LangChain interfaces on Python and JS.

Use @observe() decorator to trace Python functions, fully compatible with the native integrations with LangChain, LlamaIndex, and the OpenAI SDK.

Extending the "Pretty ✨" mode to additional json content included in ChatML messages such as functions and tools. Previously, Langfuse rendered these messages in plain json.

Langfuse automatically tracks token counts and USD spend for Anthropic Claude 3 models

Langfuse adds >20 open source evaluators through cookbook integration with open source project and fellow Y Combinator startup UpTrain.ai

Previously, the tables loaded very slowly depending on the project size and the kind of ingested data. I used the leap day to tweak queries and indexes to fix this.

Automatically capture detailed traces and metrics for every request of your LlamaIndex application with the new Langfuse Integration.

Version control model names, model parameters, and additional json data in Langfuse Prompt Management.

Retrieve aggregated daily usage metrics via a unified endpoint. Filters for application type, user, or tags are available for tailored data retrieval.

The latest release of the Python and JS/TS Langfuse SDK includes a new prompt caching feature that improves the reliability and performance of your applications.

The new major version allows to track usage and costs for many more models and for the first time also allows to add custom models and prices.

Categorize and manage your traces with tags in Langfuse.

Manage, version and deploy prompts from within Langfuse.

Finally, you can choose how to sort your trace table to find the slowest, fastest, or oldest traces with ease.

We are excited to announce the release of v2.0.0 for Python and JS/TS SDKs. Langfuse has come a long way this year since initially launching v1 of the SDKs and it was time to finally group some necessary breaking changes into one release. The release includes simpler interfaces in Python, better defaults for the Langchain integration, and a performance upgrade for the JS/TS SDK. Read on for more details.

Group multiple traces into a session and replay interactions between users and your LLM based application.

In the project settings you can now rename and transfer your project.

All tables in Langfuse include lots of columns. You can now customize them to your workflow by hiding the ones you don't need.

Removed cron job used for telemetry to increase stability of the docker image.

OpenAI has released a major SDK update. The latest Langfuse integration now supports both the new and old versions, ensuring optimal backwards compatibility with other packages.

The JSON viewer is flexible but not super simple to read. Starting with OpenAI Chat Messages you can now view payloads in a pretty format to read and debug even faster.

Langfuse now uses semantic versioning to increase the quality of the releases.

LangChain Expression Language, or LCEL, is a declarative way to easily compose chains together. Langfuse natively supports the new expression language.

Accurate token counts and costs for GPT-4 Turbo

Optional SSO enforcement on an organizational level for advanced security.

You can now upgrade to the latest version of Langfuse by pulling the latest container. Database migrations are applied automatically on startup. Optionally, one-click deploy via Railway.

RAGAS is an open-source package focused on evaluating RAG pipelines. With this new cookbook, you can use RAGAS evals on your production data in Langfuse.

Drop-in replacement for OpenAI's Python SDK to get full visibility by changing only one line of code.

The Python SDK uses Pydantic to validate the data you send to Langfuse at run time before it is asynchronously sent to the Langfuse API. Previously, the Python SDK used Pydantic v1 leading to incompatibilities with projects using Pydantic v2.

We've removed a performance bottleneck in the API authentication which affected all API requests.

New core dashboard with analytics on token usage, latencies, and scores/metrics.

If you pass JSON as a completion, it is rendered in the Langfuse UI.

Browse traces and users super-⚡️. Jump between traces with keyboard shortcuts (k & j).

Use the new filter builder to filter across all sorts of attributes. State is persisted in the URL, so you can share your filters with others.

Inputs and outputs of spans and generations can be complex, now it is much easier to read them. Click to copy any value.

Collect sets of inputs and expected outputs in Langfuse to evaluate your LLM app. Use evaluations to benchmark different experiments.

Filter generations and export them as CSV, JSON, or OpenAI-JSONL

Run model-based evaluations on your production data in Langfuse using the Python SDK.

You can now make your traces public to discuss them with your team or share them more publicly.

All users on cloud.langfuse.com have access to analytics dashboards. More analytics to come soon in core Langfuse project.

In addition to token counts, usage can now be displayed in USD. Available for many popular models.

Compare metrics across releases of your application or version of your chain/prompt. Releases are automatically tracked for major hosting providers.

Now the Langchain integration is also available for Javascript.

Self-hosting is now easier than ever with our prebuilt Docker images.

We've built the Q&A chatbot for the Langfuse docs, now everyone can access its production traces. Create an account on Langfuse Cloud and play with the Q&A chatbot to see your own traces.

Get usage data from Langfuse API for use in your own application and dashboards. Optionally broken down by user or use case.

Do 'human-in-the-loop' evaluation by adding manual scores right in the Langfuse UI.

LLM applications/chains/agents running on Langchain can now be traces via the native integration.

Share projects with coworkers and manage permissions.

New user interface to browse complex traces in a nested way.

For early-users on Langfuse Cloud, we are releasing an alpha version of our analytics dashboards.

For OpenAI and Anthropic models token counts are now automatically calculated. This is helpful for streaming endpoints that do not return usage data.

After working on the initial version for 2 months, we are excited to launch Langfuse today!