Skip to Content
DocsTracesOverview

Tracing

Tracing gives your LLM applications OpenTelemetry-based observability: it captures traces of LLM calls, tool invocations, and retrieval operations through a semantic layer built for AI workloads. It runs in two modes — test mode (linked to test runs) and production mode (live monitoring) — supports auto-instrumentation for LangChain, LangGraph, Microsoft Agent Framework, Pydantic AI, Google ADK and Haystack, and provides convenience decorators for common AI operations.

Core Concepts

What is a Trace?

A trace represents the complete journey of a single request through your application. It captures everything that happens from when a user sends a message to when they receive a response.

What is a Span?

A span represents a single operation within a trace. Each function call, LLM invocation, or tool execution creates a span with:

  • Name - What operation occurred (e.g., ai.llm.invoke, function.chat)
  • Duration - How long it took
  • Attributes - Metadata like model name, token counts, or tool parameters
  • Status - Success or error

Trace Hierarchy

Spans are organized in a parent-child hierarchy. The root span represents the entry point, with child spans for each nested operation:

In this example:

  • The trace captures a complete chat interaction
  • The root span (function.chat) is the entry point
  • Child spans show each nested operation with timing

Traces Dashboard

View all traces from your application in the Rhesis dashboard. Each row shows the operation, its input, the linked endpoint, when it started, duration, span count, tokens, cost, model, evaluation status and environment. Input tokens, Output tokens, Input cost and Output cost are hidden by default; turn them on from the column menu. Every column except Input, Endpoint and Evaluation sorts, including tokens, cost and model.

Four tiles above the list sum up what it holds: Traces, Spans (with errors), Cost and Models. Some filters cannot narrow the tiles; when one of those is active, a caption says the totals cover the whole project for the selected environment and time range.

The filter drawer’s Provider section narrows the list to traces that used a given LLM provider. A trace not yet priced matches no provider, and unknown covers traces whose provider could not be identified. How cost and provider are worked out is on Token Costs.

The dashboard shows traces for the active project. If the list looks empty or stale, use the organization menu to switch projects and reload the trace list under the correct project scope.

Traces dashboard — trace list with operation, input, duration, span count and evaluation status

Conversation Traces

For multi-turn tests and chat sessions, traces group into conversation threads with turn-by-turn navigation across the tree, sequence, and graph views. See Conversation Tracing for the full walkthrough.

Reading traces back

The dashboard is not the only way in. Recorded traces are also readable from code and from an agent:

  • Python SDK — query traces, walk their spans, and annotate them. See Traces & Spans.
  • MCP — list_traces, get_trace, get_trace_metrics, lookup_span and list_trace_providers, so an agent in Cursor or Claude Code can answer why a run failed or what it cost. See Agent Skill.

Quick Start

app.py
from rhesis.sdk import RhesisClient, observe

# Initialize client (required for tracing)
client = RhesisClient(
    api_key="your-api-key",
    project_id="your-project-id",
    environment="development",
)

# Use convenience decorators for common operations
@observe.llm(provider="openai", model="gpt-4")
def call_llm(prompt: str) -> str:
    return openai.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": prompt}]
    )

# Traces are automatically sent to Rhesis
response = call_llm("What is machine learning?")

Endpoints Are Automatically Traced

Use @endpoint for functions you want Rhesis to call from the product (for example when a test run executes your code via the Connector), not only for tracing. Those functions are automatically traced when they run. Use @observe for code you only need to observe inside your app; it is not registered for remote invocation. See Observe vs endpoint for the full comparison.

app.py
from rhesis.sdk import endpoint

@endpoint()
def chat(input: str, session_id: str = None) -> dict:
    # Registered with Rhesis; traced when run locally or triggered from the platform
    return {"output": process_message(input), "session_id": session_id}

How It Works

Traces are sent via HTTP (not WebSocket), batched and exported every 5 seconds.

Operating Modes

Test Mode

Traces originate from test runs triggered through the platform or SDK. Linked data is preserved:

  • The endpoint being tested
  • The test run that initiated the trace
  • The specific test case being executed

Click any trace to view its span hierarchy, timing breakdown, and linked test results:

Trace detail — span hierarchy grouped by turn, with per-span timing and the span details panel

Production Mode

Traces originate from normal application operation. They capture live behavior for monitoring and performance analysis.

Next steps

TasksGetting Started