Endpoints

Configure and manage API endpoints that your tests execute against.

Why Endpoints?

Endpoints enable you to test AI applications without hardcoding API details into every test. They provide:

• Reusability: Configure once, use across hundreds of tests
• Flexibility: Switch between models, environments, or providers without changing tests
• Comparison: Run identical tests against different endpoints to compare performance
• Version Control: Track configuration changes and their impact on test results
• Security: Centralize API keys and credentials in one place

Understanding Endpoints

An endpoint in Rhesis is a complete configuration for calling an external API. When you run tests, Rhesis:

1. Takes your test prompt or input
2. Formats it according to your endpoint’s request template
3. Sends the request to your API
4. Receives the response
5. Evaluates the response against your metrics

Think of endpoints as the bridge between your tests and the AI system you’re evaluating.

Creating an Endpoint

Manual Configuration

Create an endpoint from scratch with full control over all settings.

Configure the endpoint name, description, project assignment, and environment. Then set up the request by providing the API URL, protocol (REST or WebSocket), and HTTP method.

Request Headers

Define authentication and other required headers in JSON format:

{
"Authorization": "Bearer {API_KEY}",
"Content-Type": "application/json"
}

Request Body Template

Templates use Jinja2 syntax for dynamic values. Use the tojson filter for proper JSON formatting:

{
"model": "gpt-4",
"messages": [
  {
    "role": "user",
    "content": "{{ input }}"
  }
],
"temperature": 0.7,
"conversation_id": {{ conversation_id | tojson }}
}

The tojson filter ensures values are properly formatted as JSON (e.g., null instead of None, properly quoted strings).

Response Mappings

Extract values using JSONPath or Jinja2 templates with conditional logic:

{
"output": "$.choices[0].message.content",
"model_used": "$.model",
"tokens": "$.usage.total_tokens"
}

For conditional logic or nested fields, use Jinja2 templates with the jsonpath() function:

{
"output": "{{ jsonpath('$.text_response') or jsonpath('$.result.content') }}",
"conversation_id": "$.conversation_id"
}

The first value evaluates as a Jinja2 template (returns first non-empty field), while pure JSONPath expressions (starting with $) work as before.

Conversation Tracking for Multi-Turn Conversations

Rhesis automatically tracks conversation state across multiple turns when you include a conversation identifier in your response mappings. This enables testing of conversational AI systems that maintain context across interactions.

To enable conversation tracking, simply map the conversation field from your API response:

{
"output": "$.choices[0].message.content",
"conversation_id": "$.conversation_id"
}

Rhesis automatically detects and handles the following conversation field names:

Most Common (Tier 1):

• conversation_id
• session_id
• thread_id
• chat_id

Common Variants (Tier 2):

• dialog_id
• dialogue_id
• context_id
• interaction_id

When a conversation field is mapped, Rhesis will:

1. Extract the conversation ID from each API response
2. Automatically include it in subsequent requests for the same conversation
3. Maintain conversation context across multiple test turns

This works for both REST and WebSocket endpoints without any additional configuration.

// First request - no conversation_id
{
"query": "What is the capital of France?"
}

// API returns: { "output": "Paris", "conversation_id": "abc-123" }

// Second request - conversation_id automatically included
{
"query": "What is its population?",
"conversation_id": "abc-123"
}

// API maintains context and responds about Paris

[SCREENSHOT HERE: Endpoint configuration form showing the Request Body Template editor with JSON syntax highlighting and the Response Mappings section below it. Both should be filled with example OpenAI API configuration.]

Importing from Swagger/OpenAPI

Click Import Swagger, enter your Swagger/OpenAPI specification URL, and click Import. This automatically populates request templates and response structures from your API documentation. You’ll still need to fill in authentication details and select which operations to configure.

Testing Your Endpoint

Before running full test suites, verify your endpoint configuration works correctly. Navigate to the Test Connection tab, enter sample input data, and click Test Endpoint. Review the response to ensure it returns expected data.

[SCREENSHOT HERE: Test Connection tab showing the input JSON editor with sample test data, Test Endpoint button, and the response output area displaying a successful API response with formatted JSON.]

Managing Endpoints

Viewing Endpoints

The Endpoints page displays all your configured endpoints organized by project. Each entry shows the project icon and name, the endpoint name, the protocol (REST or WebSocket), and the environment (development, staging, or production). Click any endpoint to view its full configuration and execution history.

Editing Endpoints

Open the endpoint details page, click Edit, modify any configuration fields, and click Save. Changes take effect immediately for new test runs.

Deleting Endpoints

Select one or more endpoints from the grid and click Delete.

Using Endpoints in Tests

Executing Test Sets

When you run a test set, select which endpoint to execute it against, configure the execution mode (parallel or sequential), and click Run Tests. Rhesis sends each test’s prompt through the endpoint and evaluates responses against your configured metrics.

Multiple Endpoints

Creating multiple endpoints opens up powerful testing scenarios. You can run the same tests against different AI models to compare their performance, set up separate endpoints for each environment to validate changes before production deployment, configure A/B tests to compare response quality across different settings, or use different endpoints for load and performance testing. Each test run is independent, allowing you to analyze differences in behavior, quality, and performance across your various configurations.

Environment Management

Organize endpoints by environment to match your deployment workflow.

Development endpoints typically point to local or development servers where you can iterate quickly, debug issues, and test configuration changes without any risk to production systems.

Staging endpoints connect to pre-production systems for validation, integration testing, and performance verification before promoting changes to production.

Production endpoints represent live production APIs. Use these for regression testing and quality monitoring of your deployed AI systems. These endpoints require extra care when modifying.

Environment tags help you quickly identify which endpoints are production-critical and which are safe for experimentation.