Bring model calls, tool execution, and the user-visible result into the same investigation. Spend less of the debugging session reconstructing what happened.
AI engineers building and maintaining production applications
Connect a real workflow
Start with the SDK or your existing OpenTelemetry instrumentation and one representative application path.
Inspect the cause
Follow parent and child operations through model decisions, tools, retries, and the final response.
Review the fix
Capture a useful failure example, test the change, and retain the configuration that produced the result.
At a glance
| Engineering question | Evidence to retain |
|---|---|
| Why did this fail? | Inputs, outputs, operations, and errors |
| Who was affected? | Safe user/session context and environment |
| What changed? | Release, prompt version, model, and parameters |
| Is the fix better? | Baseline and candidate on relevant examples |
| Can we trust the view? | Runtime completion and ingestion health checks |
Start with a task, not every possible event
Choose a user task that your team regularly supports. Add a parent run and the model and tool operations needed to explain it. Lunary documents manual instrumentation for agent, tool, LLM, and chain runs, plus SDK integrations and OpenTelemetry ingestion. Verify one complete successful task and one failure before widening the rollout. A small trace that explains the application is more useful than many disconnected requests.
Separate bad model decisions from failed execution
When the answer is wrong, inspect what the model received, what it returned, and what the application actually did. A valid tool request can still fail during execution; a successful tool can return unhelpful data. Preserve those distinctions in your operation names and error records. Link the run to the relevant conversation and release context so the investigation ends with a specific hypothesis rather than a guess about the model.
Make prompt changes reproducible
Lunary's prompt templates and playground provide a place to work on prompt behavior. Bring the failing example and a few neighboring successes into the review. Record the variables, model, and parameters so you can distinguish a prompt change from an unrelated configuration difference. Inspect regressions alongside improvements. Keep the observed failed response separate from any reference answer you write for the test.
Verify telemetry in the runtime that serves users
Serverless shutdown, streaming, cancellations, and asynchronous workers can reveal gaps that a local happy-path call misses. Validate the completion and error events in the environment you intend to use. Test that redaction preserves enough structure to diagnose the task. Track ingestion health independently from provider health, because a quiet dashboard can mean either low traffic or missing telemetry.
A practical first-week acceptance check
Choose one production issue that already cost the team time. Reproduce it safely, instrument the relevant path, and ask another engineer to explain the cause using the recorded evidence. Make a candidate fix and compare it against the original case and nearby successes. The first milestone is a useful investigation and a defensible change; expanding coverage should follow the next unresolved question.
- One nested tool failure appears under the correct parent run.
- The final user-visible response matches the recorded task outcome.
- The prompt or code change has a clear before-and-after case.
- A teammate can understand the investigation without the original author.
Common questions
Can I bring existing OpenTelemetry instrumentation?
Lunary documents OpenTelemetry ingestion. Verify the attribute mapping, parent relationships, and supported payloads for your current instrumentation.
Must every provider use the same SDK?
No single SDK is required for every architecture. Choose a documented integration or explicit instrumentation path for each workload and validate the result.
Does a successful local trace prove production coverage?
No. Verify streaming, errors, shutdown behavior, and the deployed runtime before claiming production coverage.
Sources & methodology
Lunary publishes this guide. We compare documented workflows and explain where each approach fits; this is not an independent benchmark or a hands-on product rating. Features, limits, and commercial terms can change. Check the linked vendor documentation before deciding.