Skip to Content
DocsEndpointsOverview

Endpoints

Configure and manage API endpoints that your tests execute against.

What are Endpoints? Endpoints represent the AI services or APIs that you want to test. They define how Rhesis connects to your application, sends test inputs, and receives responses for evaluation.

Why Endpoints?

Endpoints enable you to test AI applications without hardcoding API details into every test. They provide:

  • Reusability: Configure once, use across hundreds of tests
  • Flexibility: Switch between models, environments, or providers without changing tests
  • Comparison: Run identical tests against different endpoints to compare performance
  • Version Control: Track configuration changes and their impact on test results
  • Security: Centralize API keys and credentials in one place

Understanding Endpoints

An endpoint in Rhesis is a complete configuration for calling an external API. When you run tests, Rhesis:

  1. Takes your test prompt or input
  2. Formats it according to your endpoint’s request template
  3. Sends the request to your API
  4. Receives the response
  5. Evaluates the response against your metrics

Think of endpoints as the bridge between your tests and the AI system you’re evaluating.

Platform-Managed Variables

Rhesis uses a set of platform-managed variables to standardize communication between your tests and your API. You map these variables to the fields your endpoint expects (in request templates) and the fields your endpoint returns (in response mappings). This abstraction lets Rhesis work with any API regardless of its specific field names.

Request Variables

These variables are available in your request body templates using Jinja2 syntax (\{\{ variable \}\}). Rhesis populates them automatically when running tests.

VariableRequiredDescription
inputYesThe user query or test prompt. This is the primary text sent to your AI endpoint.
filesNoArray of file attachments included with the test. Each file is an object with filename, content_type, and data (base64-encoded). Use file format filters to convert files into provider-specific formats.
conversation_idNoConversation tracking identifier for multi-turn endpoints (both stateful and stateless). Rhesis uses this to track conversations across turns.
messagesNoFull conversation history as an array of message objects. Used for stateless endpoints that require the entire history with every request.
system_promptNoSystem prompt text. Rhesis prepends it to the messages array and strips it from the final request body before sending.
test_idNoUnique identifier of the test being executed. Only populated during test runs; renders as an empty string otherwise. Place it wherever your API expects it — see Test Execution Context for examples.
test_run_idNoUnique identifier of the current test run. Only populated during test runs; renders as an empty string otherwise.
test_configuration_idNoUnique identifier of the test configuration driving the run. Only populated during test runs; renders as an empty string otherwise.

Provider-specific Jinja filters such as to_anthropic, to_openai, and to_gemini are documented in Single-Turn Endpoints.

Response Variables

These variables are extracted from your API response using JSONPath or Jinja2 expressions in the response mapping. Rhesis uses them for evaluation, display, and conversation tracking.

VariableRequiredDescription
outputYesThe main response text from your API. This is what Rhesis evaluates against your metrics.
contextNoAdditional context or reasoning provided by the response (e.g., retrieved documents, sources).
metadataNoStructured data about the response (e.g., model version, token counts, confidence scores). Stored with the test result and available to custom metrics during evaluation.
tool_callsNoTool or function calls made by the API during response generation (e.g., function calls, API lookups). Stored with the test result and available to metrics that evaluate tool use.
Conversation ID fieldsFor conversationalAny of the recognized conversation identifiers: conversation_id, session_id, thread_id, chat_id, dialog_id, dialogue_id, context_id, interaction_id. Map one of these for both stateful and stateless multi-turn endpoints so Rhesis can track conversations across turns.

You can include additional custom fields in both request templates and response mappings. Custom fields are passed through and stored, but Rhesis does not actively use them for evaluation or conversation management.

How Mapping Works

Request and response mappings tell Rhesis how to translate between its standard variables and your API’s specific field names.

Request mapping uses Jinja2 templates to place platform variables into your API’s expected request body structure:

request-mapping-example.json
{
  "model": "gpt-4",
  "messages": [
    {
      "role": "user",
      "content": "{{ input }}"
    }
  ],
  "temperature": 0.7
}

Response mapping uses JSONPath expressions to extract values from your API’s response into platform variables:

response-mapping-example.json
{
  "output": "$.choices[0].message.content",
  "metadata": "$.usage"
}

For full details on template syntax, filters, and advanced mapping techniques, see Single-Turn Endpoints.

Creating an Endpoint

New to mapping? Use the Auto-Configure feature to let AI generate your request and response mappings automatically. Just paste a curl command, code snippet, or API documentation and Rhesis handles the rest.

Manual Configuration

Create an endpoint from scratch with full control over all settings.

Configure the endpoint name, description, project assignment, and environment. Then set up the request by providing the API URL, protocol (REST or WebSocket), and HTTP method.

Request Headers

Define authentication and other required headers in JSON format:

headers.json
{
  "Authorization": "Bearer {{ auth_token }}",
  "Content-Type": "application/json"
}

The auth_token variable is a platform-managed placeholder that Rhesis automatically replaces with the authentication token you configure in the endpoint settings UI. You can reference it using Jinja2 syntax (\{\{ auth_token \}\}) or the legacy \{API_KEY\} / \{auth_token\} placeholders with single braces.

If you configure an authentication token in the UI but do not include an Authorization header explicitly, Rhesis automatically adds Authorization: Bearer <token> to the request headers.

Request and Response Mappings

Configure your request body template and response mappings to bridge platform variables with your API’s fields. Rhesis supports two endpoint patterns depending on how your API handles conversations:

  • Single-Turn Endpoints: Standard request/response mapping with Jinja2 templates and JSONPath. Covers most use cases including simple query-response APIs.
  • Multi-Turn Conversations: Conversation tracking for stateful endpoints (your API manages session state) and message history for stateless endpoints (Rhesis manages conversation state).
  • Mapping Examples: Practical examples for OpenAI, Anthropic, Google Gemini, RAG pipelines, WebSocket endpoints, and more.

SDK Endpoints

If your AI logic lives in Python functions, you can register them as endpoints directly from code using the Rhesis SDK. Decorate your functions with @endpoint and the SDK handles registration and mapping automatically. See SDK Endpoints for details, or the full SDK Connector reference for advanced patterns.

Next Steps

Organizations & TeamManaging Endpoints