Traces

A trace is one request’s worth of work inside an instrumented application: the LLM calls, retrievals and tool invocations it made to produce a response, each recorded as a span with its own duration, status and model. A test result says what came back and what the metrics made of it. A trace says why.

Traces are written by instrumentation, not through the API. See Tracing for recording them; this page is about reading them back.

A trace has two ids

They are not interchangeable, and confusing them is the one thing that silently produces wrong results.

db_id is the root span’s row id. The SDK resolves it from trace_id for you, so annotate and get_annotations take neither — they work it out. You only need the distinction when an id reaches you from somewhere else, such as annotation.context.trace_db_id, which is the row id.

Properties

A Span carries id (its row id), span_id (the OpenTelemetry hex), span_name, span_kind, start_time, end_time, duration_ms, status_code, status_message, model_name, cost_usd, attributes, events, trace_metrics and children.

Unpriced is not free

A cost of None means the call was not priced: the platform holds no price for that model, or the span reported no model name at all. 0.0 means it was priced and the model really is free. The two never share a figure, so the distinction is readable rather than guessed at.

This matters when adding costs up. Treating None as zero under-reports a total, and cost == 0 reads a model nobody has a price for as one that costs nothing:

priced = [t for t in traces if t.total_cost_usd is not None]
total = sum(t.total_cost_usd for t in priced)

# Say what the total covers, rather than implying it covers everything
unpriced = len(traces) - len(priced)

A trace keeps its tokens and model names either way, so “no price for these models” stays distinguishable from “no LLM calls at all”.

Finding traces

from rhesis.sdk.entities import Traces

# The traces one run produced, one per test execution
Traces.for_test_run(test_run_id)

# Or from the entity you already have
test_run.get_traces()
test_result.get_traces()

Traces.for_test_result(test_result_id)
Traces.for_endpoint(endpoint_id)
Traces.for_conversation(conversation_id)   # every turn of one conversation
Traces.with_errors(test_run_id=test_run_id)
Traces.slower_than(5000, endpoint_id=endpoint_id)

query takes every filter the platform supports, named, so a wrong one fails at the call rather than being ignored:

Traces.query(
    test_run_id=test_run_id,
    status_code="ERROR",
    duration_min_ms=2000,
    start_time_after="2026-09-01",
    sort_by="duration_ms",
    limit=20,
)

# Every span as its own row, rather than one row per trace
Traces.query(test_run_id=test_run_id, root_spans_only=False, status_code="ERROR")

Only the arguments you pass are sent, so the platform’s own defaults stay in charge: one row per trace, newest first. limit is the most traces to return in total; omit it and every page is read, which on a busy project is a lot, so filter first.

status_code and has_errors both describe the root span, on a listing and on a detail alike. A trace whose inner LLM call failed under a root that returned OK reads as status_code="OK", has_errors=False — which is correct, and is why there are two other ways to find the failure:

# Every failing span as its own row, whichever trace it belongs to
Traces.query(test_run_id=test_run_id, root_spans_only=False, status_code="ERROR")

# Or, on a trace you have already read, the count across the whole tree
trace.error_count
[span for span in trace.spans() if span.status_code == "ERROR"]

error_count is the any-span figure and comes from the detail call only.

This route takes named filters rather than OData, so Traces.all(filter=...) refuses the argument instead of quietly ignoring it.

Scope

With no project_id and no project on your API token, the platform returns only traces that belong to no project. An unexpectedly empty list usually means that rather than an absence of traces:

Traces.query(project_id=project.id, test_run_id=test_run_id)

Providers

provider matches a trace where any priced call used one of the named providers. Ask what those are rather than guessing a name, since an unmatched one returns an empty page rather than an error:

Traces.providers()                          # e.g. ["openai", "anthropic", "unknown"]
Traces.query(provider=["openai"], test_run_id=test_run_id)

A trace whose costs have not been priced yet matches no provider, so this filter can hide recent traces. "unknown" is a real value: those are traces whose provider neither they nor their model name identify.

Reading one trace

Reading a trace by id needs the project as well, which is unusual for this SDK and is what the platform’s route requires:

trace = Traces.pull(trace_id, project_id=project.id)

trace.spans()                    # every span, depth first, roots included
trace.span("ai.llm.invoke")      # the first span with that name, or None
trace.spans(name="ai.llm.invoke")

slowest = max(trace.spans(), key=lambda span: span.duration_ms or 0)
print(slowest.span_name, slowest.duration_ms, slowest.status_message)

Set RHESIS_PROJECT_ID and project_id becomes optional. A trace that came from a listing already carries its project, so trace.spans() on a listed trace needs nothing extra — it fetches the detail once, behind the property.

Note: the detail response is the large one. Every span carries its full attributes and events, which hold up to 8000 characters of prompt and completion apiece and 10000 each of conversation input and output on the root, and nothing truncates it. Read span_count from the listing before walking the spans of many traces, and prefer root_spans_only=False when all you need is which operation was slow or failed.

Annotating a trace

A Pass or Fail annotation on a trace overrides its automated outcome, as on a test result:

trace.annotate("fail", "Answered from the wrong document.")

# One trace metric, leaving the others on their automated verdicts
trace.annotate("pass", "Grounded after all.", metric="Groundedness")

# One turn of a multi-turn conversation
trace.annotate("fail", "Went off script here.", turn=2)

# Or judge the one operation that went wrong
trace.span("ai.llm.invoke").annotate("fail", "Ignored the retrieved context.")

trace.get_annotations()

Both take the row id rather than the hex, which is why they resolve it themselves. See Annotations for verdicts, targets and resolving.

Spans on their own

A span’s row id is what the platform records against a trace: in an annotation’s context.trace_db_id, and in a /traces/… link. Spans is the way back from one:

from rhesis.sdk.entities import Spans

# From an annotation back to what it judged
trace = Spans.trace_for(annotation.context.trace_db_id)

span = Spans.pull(span_row_id)       # resolves the trace, picks the span out of its tree
Spans.lookup(span_row_id)            # {"trace_id": ..., "project_id": ..., "span_id": ...}
Spans.exists(span_row_id)

span.get_files()                     # files attached to the span, such as an audio input

There is no route that returns a span alone, so Spans.pull resolves the trace and finds it in the tree. Spans.all() therefore raises, pointing at Traces.query(root_spans_only=False) for a list of spans as rows.

Traces are read-only

The only write is ingestion. push() and delete() raise, naming the instrumentation instead:

trace.push()      # NotImplementedError: traces are produced by instrumentation
trace.delete()    # ... see rhesis.sdk.telemetry

What you can do to an existing trace is annotate it.