Decide which payloads and metadata you need before exporting. Then reconcile the archive and keep data migration separate from any change to your gateway.
Engineers extracting Helicone request data
Use the export tooling
Helicone documents an export package supporting common file formats, filtering, retries, and region selection.
Include bodies deliberately
Request metadata alone may not contain the inputs and outputs needed for debugging or evaluation.
Keep routing stable
An archive operation does not require changing the production model endpoint or gateway behavior.
At a glance
| Preserve | Why it matters |
|---|---|
| Request and session identifiers | Separate individual attempts from user tasks |
| Request and response bodies | Retain examples usable for debugging or evaluation |
| Custom properties | Preserve environment, release, and application context |
| Model, tokens, cost, and timestamps | Compare equivalent behavior and cost units |
| Feedback | Keep observed ratings separate from approved expected answers |
1. Define a request-level archive
Record the account region, project or application filters, date window, and the data you need. Helicone's export documentation distinguishes metadata, model information, request and response bodies, performance, costs, and feedback where available. Decide whether payloads are necessary for this archive and the policy governing their storage. Keep custom properties and session identifiers if you will later relate requests to user tasks.
2. Use the documented export route
Helicone's ETL guide links an official export package and a REST alternative. For the package route, configure HELICONE_API_KEY in the environment and use the current command options in that guide. Explicitly request bodies when needed, choose your region, and save to a new output file. The tool's checkpoints and retries help extraction, but you still need to inspect the completed result and reconcile its scope.
- Choose JSONL when you want to process records incrementally.
- Verify that the selected region matches the data you intend to export.
- Record the tool version and the date/property filters for reproducibility.
3. Query a small sample with the REST API
The documented ClickHouse query endpoint provides a programmatic route. The example below retrieves one page from the US API and is intended to inspect the response schema. It is not a complete export: add your date and application filters, paginate deliberately, and use the documented regional endpoint for your account. Custom-property filters belong under request_response_rmt in Helicone's filter structure.
curl --fail-with-body --request POST \
--url https://api.helicone.ai/v1/request/query-clickhouse \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $HELICONE_API_KEY" \
--data '{"filter":"all","limit":100,"offset":0}' \
--output helicone-first-page.json4. Validate completeness and meaning
Count unique request IDs, inspect the time range, and check a request with a body, an error, and a session. Do not assume that a metadata response also contains the full payload; inspect the chosen export mode. Preserve cost units and distinguish provider cost from platform fees. For retries, keep the individual requests and the session context so later analysis does not confuse attempts with completed user tasks.
5. Map evaluation examples into Lunary
Select the requests or complete sessions that represent the behavior you want to test. Define how the original input becomes a dataset input and how an observed response differs from a reviewer-approved expected answer. Lunary's documented dataset import API accepts CSV or JSONL content. Keep the raw export and a source-ID map outside the transformed dataset so provenance is not lost. Inspect the inserted count and representative examples after import.
6. Treat a gateway cutover as another release
If Helicone routes your production model requests, exporting its data does not remove that dependency. First validate Lunary instrumentation on a bounded application workflow. If you also intend to change the gateway, separately test provider credentials, model identifiers, streaming, failure handling, and rollback. A successful dataset import is not evidence that a new request path behaves correctly.
Common questions
Why are inputs or outputs missing from my export?
Check the selected extraction mode. Helicone's export tooling documents an explicit include-body option; metadata-only extraction is a different result.
Is the REST example a full archive script?
No. It fetches one page to inspect the schema. A complete archive needs explicit filters, pagination, error handling, and count reconciliation.
Does moving observability require replacing Helicone's gateway?
Not as a general rule. Keep gateway dependencies explicit and evaluate any request-path change separately.
Sources & methodology
Lunary publishes this guide. We compare documented workflows and explain where each approach fits; this is not an independent benchmark or a hands-on product rating. Features, limits, and commercial terms can change. Check the linked vendor documentation before deciding.