Skip to Content
GuidesConfiguring Telemetry for Your AI ApplicationTelemetry: Raw OpenTelemetry

Configure Telemetry: Raw OpenTelemetry

For cases where you can’t decorate functions (third-party code, dynamic dispatch) or need fine-grained control over span names, attributes, and events, use the OpenTelemetry tracer directly with Rhesis’s semantic constants.

Prerequisites

If you initialize RhesisClient(), the export pipeline below is already wired up for you: skip to Creating Spans Manually. The manual pipeline setup is only needed if you’re bypassing RhesisClient() entirely.

The Export Pipeline

RhesisClient() calls get_tracer_provider(), which creates a singleton TracerProvider with a BatchSpanProcessor wrapping a RhesisOTLPExporter, and registers it as the global OpenTelemetry provider. After that, any trace.get_tracer(...) call returns a tracer that feeds into this pipeline:

Setting up the pipeline manually

If you skip RhesisClient() and wire things yourself:

telemetry_setup.py
from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from rhesis.telemetry.exporter import RhesisOTLPExporter

# 1. Create the exporter (authenticated JSON-over-HTTP to your backend)
exporter = RhesisOTLPExporter(
    api_key="your-api-key",
    base_url="https://api.rhesis.ai",  # or your self-hosted URL
    project_id="your-project-id",
    environment="production",
)

# 2. Create a TracerProvider with resource metadata
resource = Resource.create({
    "service.name": "my-app",
    "service.namespace": "rhesis",
    "deployment.environment": "production",
})
provider = TracerProvider(resource=resource)

# 3. Wire the exporter through a BatchSpanProcessor
provider.add_span_processor(
    BatchSpanProcessor(
        exporter,
        max_queue_size=2048,        # spans buffered in memory
        max_export_batch_size=512,  # spans per HTTP request
        schedule_delay_millis=5000, # flush interval (5 seconds)
    )
)

# 4. Register as global provider
trace.set_tracer_provider(provider)

# Now trace.get_tracer("my-app") works
tracer = trace.get_tracer("my-app")

RhesisOTLPExporter handles authentication, retry with exponential backoff jitter (on 408/429/5xx and connection errors), chunking for batches over max_chunk_size (default 100), conversation context propagation, and span-name validation.

At the end of your process, flush pending spans:

teardown.py
from rhesis.telemetry.provider import shutdown_tracer_provider

shutdown_tracer_provider()  # flushes buffered spans and shuts down

The Semantic Layer

All spans must be named ai.<domain>.<action> or function.<name>. Names like chain, workflow, or pipeline are rejected by the backend with HTTP 422.

constants.py
from rhesis.sdk.telemetry import AIAttributes, AIEvents
from rhesis.telemetry.schemas import AIOperationType

# Span names (use as span_name)
AIOperationType.LLM_INVOKE          # "ai.llm.invoke"
AIOperationType.TOOL_INVOKE         # "ai.tool.invoke"
AIOperationType.RETRIEVAL           # "ai.retrieval"
AIOperationType.EMBEDDING_GENERATE  # "ai.embedding.generate"
AIOperationType.RERANK              # "ai.rerank"
AIOperationType.EVALUATION          # "ai.evaluation"
AIOperationType.GUARDRAIL           # "ai.guardrail"
AIOperationType.TRANSFORM           # "ai.transform"
AIOperationType.AGENT_INVOKE        # "ai.agent.invoke"
AIOperationType.AGENT_HANDOFF       # "ai.agent.handoff"

# Attribute keys
AIAttributes.MODEL_PROVIDER        # "ai.model.provider"
AIAttributes.MODEL_NAME            # "ai.model.name"
AIAttributes.LLM_TOKENS_INPUT      # "ai.llm.tokens.input"
AIAttributes.LLM_TOKENS_OUTPUT     # "ai.llm.tokens.output"
AIAttributes.TOOL_NAME             # "ai.tool.name"
AIAttributes.TOOL_TYPE             # "ai.tool.type"
AIAttributes.AGENT_NAME            # "ai.agent.name"

# Event names (for prompt/completion capture)
AIEvents.PROMPT       # "ai.prompt"
AIEvents.COMPLETION   # "ai.completion"
AIEvents.TOOL_INPUT   # "ai.tool.input"
AIEvents.TOOL_OUTPUT  # "ai.tool.output"

See the Semantic Conventions Reference for the complete list.

Creating Spans Manually

Trace an LLM call

llm_span.py
from opentelemetry import trace
from rhesis.sdk.telemetry import AIAttributes, AIEvents
from rhesis.telemetry.schemas import AIOperationType

tracer = trace.get_tracer("my-app")

def call_llm(prompt: str) -> str:
    with tracer.start_as_current_span(AIOperationType.LLM_INVOKE) as span:
        span.set_attribute(AIAttributes.MODEL_PROVIDER, "openai")
        span.set_attribute(AIAttributes.MODEL_NAME, "gpt-4o")

        span.add_event(AIEvents.PROMPT, attributes={
            AIAttributes.PROMPT_ROLE: "user",
            AIAttributes.PROMPT_CONTENT: prompt,
        })

        response = openai.chat.completions.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": prompt}],
        )
        result = response.choices[0].message.content

        span.add_event(AIEvents.COMPLETION, attributes={
            AIAttributes.COMPLETION_CONTENT: result,
        })
        span.set_attribute(AIAttributes.LLM_TOKENS_INPUT, response.usage.prompt_tokens)
        span.set_attribute(AIAttributes.LLM_TOKENS_OUTPUT, response.usage.completion_tokens)

        return result

Trace a tool call

tool_span.py
def execute_tool(tool_name: str, tool_input: dict) -> dict:
    with tracer.start_as_current_span(AIOperationType.TOOL_INVOKE) as span:
        span.set_attribute(AIAttributes.TOOL_NAME, tool_name)
        span.set_attribute(AIAttributes.TOOL_TYPE, "function")

        span.add_event(AIEvents.TOOL_INPUT, attributes={AIAttributes.TOOL_INPUT_CONTENT: str(tool_input)})
        result = tool_registry[tool_name](**tool_input)
        span.add_event(AIEvents.TOOL_OUTPUT, attributes={AIAttributes.TOOL_OUTPUT_CONTENT: str(result)})

        return result

Trace an agent handoff

agent_span.py
def run_agent(agent_name: str, input_text: str) -> str:
    with tracer.start_as_current_span(AIOperationType.AGENT_INVOKE) as span:
        span.set_attribute(AIAttributes.AGENT_NAME, agent_name)

        result = call_llm(input_text)

        if should_handoff(result):
            with tracer.start_as_current_span(AIOperationType.AGENT_HANDOFF) as handoff_span:
                handoff_span.set_attribute(AIAttributes.AGENT_HANDOFF_FROM, agent_name)
                handoff_span.set_attribute(AIAttributes.AGENT_HANDOFF_TO, "specialist")
                result = run_agent("specialist", result)

        return result

Building attribute dicts by hand? create_llm_attributes(...) and create_tool_attributes(...) in rhesis.sdk.telemetry save you from remembering every constant name.

Tracking Multi-Turn Conversations

At the lowest level, you build the synthetic parent context yourself so subsequent turns share a trace_id:

conversation_span.py
from opentelemetry.trace import NonRecordingSpan, SpanContext, TraceFlags

conversations = {}  # session_id -> {"trace_id": str, "turn": int}

def handle_turn(session_id: str, user_message: str) -> str:
    parent_context = None

    if session_id in conversations:
        conv = conversations[session_id]
        trace_id_int = int(conv["trace_id"], 16)
        synthetic_span_ctx = SpanContext(
            trace_id=trace_id_int,
            span_id=0x00000000CAFECAFE,  # placeholder, stripped by exporter
            is_remote=True,
            trace_flags=TraceFlags(TraceFlags.SAMPLED),
        )
        parent_context = trace.set_span_in_context(NonRecordingSpan(synthetic_span_ctx))

    with tracer.start_as_current_span("function.chat", context=parent_context) as span:
        span.set_attribute("rhesis.conversation.is_turn_root", True)
        span.set_attribute("rhesis.conversation.id", session_id)
        span.set_attribute("rhesis.conversation.input", user_message[:10000])

        if session_id not in conversations:
            trace_id = format(span.get_span_context().trace_id, "032x")
            conversations[session_id] = {"trace_id": trace_id, "turn": 1}

        return call_llm(user_message)

For the full attribute reference (rhesis.conversation.*) and the shared-vs-separate trace_id tradeoff, see Conversation Tracing.

For most applications, Decorators cover the same ground with far less code — reach for raw OpenTelemetry only when you genuinely need it.

Related:

Telemetry: DecoratorsTelemetry: GenAI Semantic Conventions