Multi-Turn Conversations

Test conversational AI systems that maintain context across multiple interactions.

Stateful vs. Stateless Endpoints

AI endpoints handle conversation context in one of two ways:

• Stateful endpoints maintain session state on the server. Your API returns a conversation identifier (e.g., conversation_id or session_id), and the caller passes it back on subsequent requests. The API looks up the conversation history internally.
• Stateless endpoints do not maintain any server-side state. The caller must send the entire conversation history (as a messages array) with every request. This is the pattern used by most LLM provider APIs.

Rhesis detects which mode to use based on your endpoint configuration and handles both patterns transparently.

Stateful Endpoints (Conversation Tracking)

For endpoints that manage their own session state, Rhesis tracks the conversation identifier returned by your API and includes it in subsequent requests automatically.

Configuration

Map the conversation field from your API response in the response mapping:

{
  "output": "$.choices[0].message.content",
  "conversation_id": "$.conversation_id"
}

Include the conversation variable in your request body template so Rhesis can pass it back on subsequent turns:

{
"query": "{{ input }}",
"conversation_id": {{ conversation_id | tojson }}
}

The tojson filter ensures the value is null on the first turn (when no conversation has been established) and a properly quoted string on subsequent turns.

Supported Conversation Field Names

Rhesis automatically detects and handles the following conversation field names in your response mapping. You do not need any additional configuration beyond mapping the field.

Most common (Tier 1):

• conversation_id
• session_id
• thread_id
• chat_id

Common variants (Tier 2):

• dialog_id
• dialogue_id
• context_id
• interaction_id

How It Works

When a conversation field is mapped, Rhesis will:

1. Send the first request without a conversation identifier (or with null)
2. Extract the conversation ID from the API response
3. Automatically include it in subsequent requests for the same conversation
4. Maintain conversation context across all test turns

This works for both REST and WebSocket endpoints without any additional configuration.

Example Flow

// First request - no conversation_id
{
"query": "What is the capital of France?"
}

// API returns: { "output": "Paris", "conversation_id": "abc-123" }

// Second request - conversation_id automatically included
{
"query": "What is its population?",
"conversation_id": "abc-123"
}

// API maintains context and responds about Paris

Stateless Endpoints (Message History)

Some AI endpoints are stateless: they do not maintain conversation context on the server side. Instead, the caller must send the entire conversation history with every request. Rhesis supports this pattern natively by managing the conversation history internally.

Configuration

Use the messages template variable in your request body template. Rhesis detects this and switches to stateless conversation management:

{
  "messages": "{{ messages }}",
  "model": "my-model",
  "temperature": 0.7,
  "system_prompt": "You are a helpful assistant."
}

The response mapping works the same as any other endpoint. Map the output field to where the assistant’s reply is returned:

{
  "output": "$.choices[0].message.content"
}

How It Works

• Automatic detection: Rhesis identifies a stateless endpoint when the request body template contains \{\{ messages \}\}.
• History management: During multi-turn test execution, Rhesis accumulates the full conversation history (user messages and assistant responses) and sends it as the messages array with each request.
• System prompt: If you include a system_prompt field in the request body template, Rhesis prepends it to the messages array as the first entry with the system role. The system_prompt field itself is stripped from the final request before sending.
• Single-turn auto-population: For single test runs (outside multi-turn conversations), Rhesis automatically builds the messages array from the test input and system prompt so you do not need to provide it manually.
• Conversation ID: Even though your endpoint is stateless, Rhesis assigns an internal conversation_id so the platform can track the conversation across turns. This ID is returned in the API response but is not sent to your endpoint.

Messages Format

The messages array follows the standard chat completion format used by most LLM providers:

[
  {
    "role": "system",
    "content": "You are a helpful assistant."
  },
  {
    "role": "user",
    "content": "What is the capital of France?"
  },
  {
    "role": "assistant",
    "content": "The capital of France is Paris."
  },
  {
    "role": "user",
    "content": "What is its population?"
  }
]

Each message has a role (system, user, or assistant) and content (the message text). Rhesis builds this array incrementally as the conversation progresses:

1. Turn 1: [system, user] — system prompt plus the first user input
2. Turn 2: [system, user, assistant, user] — previous history plus the new user input
3. Turn N: Full history up to the current turn

System Prompt Handling

The system_prompt field in the request body template is a special platform-managed variable:

• Rhesis extracts its value from the template
• It is prepended to the messages array as the first entry with role: "system"
• The system_prompt field itself is removed from the final request body before sending to your API

This means your API only receives the standard messages array with the system prompt already included as the first message.

// Your request template:
{
"messages": "{{ messages }}",
"model": "gpt-4",
"system_prompt": "You are a helpful insurance expert."
}

// What Rhesis actually sends to your API:
{
"messages": [
  { "role": "system", "content": "You are a helpful insurance expert." },
  { "role": "user", "content": "What is term life insurance?" }
],
"model": "gpt-4"
}

Choosing Between Stateful and Stateless