A model request is one step in an agent task. Choose a platform that helps your team connect model decisions, tool execution, retries, and the outcome the user experienced.
Teams shipping multi-step AI agents
Trace the entire task
Preserve the parent task and its model, retrieval, and tool operations, including failures.
Judge useful outcomes
A successful HTTP response can still produce an unsuccessful user task. Record both.
Trial Lunary
Use SDK or OpenTelemetry instrumentation to inspect a representative agent path with your team.
At a glance
| Platform | Reason to include it | Agent-specific test |
|---|---|---|
| Lunary | Application runs plus conversation context | Follow a user issue through nested tool calls |
| Langfuse | Tracing and evaluation workflows | Turn a failed observation into a repeatable test |
| LangSmith | Tracing, experiments, and prompt work | Inspect a nested failure and compare a candidate |
| Helicone | Gateway plus request observability | Verify provider behavior and surrounding tool context |
What agent observability needs to explain
When an agent returns a wrong result, the failure may come from the plan, a retrieved document, a tool argument, or a downstream response. The trace should let a reviewer follow that chain without guessing. Record meaningful operation names and stable parent relationships. Keep the distinction between the model requesting a tool and the tool actually executing; they can have different timings and failure modes.
- Task input and final user-visible result.
- Model calls, tool executions, retries, and timeouts.
- Release and prompt context, plus safe session identifiers.
Build a shortlist around the architecture
Lunary is a candidate for application traces, conversations, and prompt investigation. Langfuse and LangSmith both document tracing and evaluation workflows worth testing with complex agents. Helicone belongs on the shortlist when a provider gateway is also needed. Treat this as a workflow-based shortlist, not a measured performance ranking: the best choice depends on the stack and the investigation your team needs to perform.
Use a deliberately difficult trial
Create a representative task where retrieval returns incomplete context, a tool times out, and the agent retries before presenting a result. Verify that every attempted operation appears under the correct parent. Then ask a teammate to explain whether the retry helped and why the final answer was acceptable or wrong. Repeat with a streamed response and a user cancellation so the happy path does not hide incomplete end events.
Track cost and latency per completed task
Per-request metrics can miss expensive loops. Define a task identifier and count the attempts that belong to it. Calculate cost per successful task only after you have an explicit success signal and a clear policy for failed or cancelled tasks. Inspect tail latency as well as averages, and compare releases within the same kind of workload. A cheaper model is not an improvement if it causes enough retries or failed tasks to raise total cost.
Set up a useful first workflow in Lunary
Start with one agent and a bounded environment. Lunary documents manual runs for agents, tools, model calls, and chains, including parentRunId for linking operations. Its OpenTelemetry integration provides another path for existing instrumentation. Capture only the fields needed to diagnose the task, inspect them in the application, and add feedback or a reviewer label for meaningful failures. Use those failures to define the next prompt or behavior test.
- Acceptance: one failed tool is visible under the correct agent task.
- Acceptance: the final result and recorded failure agree.
- Acceptance: a second reviewer can reconstruct the task without reading server logs.
Common questions
Is tracing model calls enough for an agent?
Usually not. You also need the actual tools, retrieval steps, retries, and final task outcome to explain many failures.
What is a useful success metric?
Define success for the user task, such as a correctly completed action or a reviewer-approved answer. Report that alongside latency and cost, and do not infer it from a 200 response.
Does this page claim Lunary wins a benchmark?
No. The shortlist is based on documented capabilities and an original evaluation protocol, not measured comparative results.
Sources & methodology
Lunary publishes this guide. We compare documented workflows and explain where each approach fits; this is not an independent benchmark or a hands-on product rating. Features, limits, and commercial terms can change. Check the linked vendor documentation before deciding.