Skip to Content
DocsGetting StartedConnect Application

Connect Application

An Endpoint is the connection point between Rhesis and your LLM application: Rhesis sends test inputs to it and records the responses for evaluation.

Because every LLM application expects a different request format and returns data in a different shape, Rhesis uses a mapping layer: instead of rewriting your application’s API to conform to Rhesis, you map Rhesis’s standardized variables to your application’s expected format.

Rhesis supports two main approaches for connecting your application: REST Endpoints (configured via the UI) and the Rhesis Connector (using our Python SDK).

Approach 1: REST Endpoints

If your LLM application is exposed as an HTTP API, you can connect it directly from the Rhesis UI without writing any new code. To do this, log in to the Rhesis Platform and navigate to the /endpoints page.

Endpoint Creation

There are two ways to define the mapping:

  1. Auto-Configure: Provide your URL, sample payload, and API key, and Rhesis attempts to map your endpoint to the platform variables automatically.
  2. Custom Mapping: Use JSONPath expressions and Jinja2 templates to map inputs to your API request body and map the outputs back to Rhesis.

Tip: The REST approach is ideal if your AI features are already wrapped in microservices or if you use standard LLM APIs.

To learn more about setting up REST endpoints and variable mapping, see the Platform Endpoints Guide.

Approach 2: Rhesis Connector (Python SDK)

If you are developing your application in Python and want deeper integration, you can use the Rhesis Python SDK. This approach allows you to define endpoints directly in your codebase.

By decorating your Python functions with @endpoint, you create a secure, bi-directional tunnel between Rhesis and your application. Rhesis can then execute your local code directly during tests. The Connector will try to automatically map the platform-managed variables to your function’s arguments, but it also supports explicit request and response mapping.

example.py
from rhesis.sdk import RhesisClient, endpoint

client = RhesisClient(
    api_key="rh-XXXXXXXXXXXXXXXXXXXX",
    project_id="your-project-id",
)

# This function is now exposed to Rhesis for testing
@endpoint(name="My Customer Support Bot")
def run_chatbot(prompt: str, context: dict = None) -> str:
    # Your custom application logic here
    response = my_ai_chain.invoke(prompt)
    return response.text

# Start the connector to listen for tests
if __name__ == "__main__":
    client.connect()

Note: The SDK approach works even if your application runs locally behind a firewall: the connector establishes an outbound connection to Rhesis, so no ngrok or port forwarding is needed.

To learn more about the SDK integration, see the SDK Endpoints Guide.

Platform-Managed Variables & Mapping Example

Whether you are using REST Endpoints or the Python SDK, Rhesis communicates with your application using a set of standardized platform-managed variables.

Rhesis uses these variables to pass inputs to your API during testing and extract the outputs for evaluation. When creating a REST Endpoint, you map these either automatically (via Auto-Configure) or manually.

Request Variables (Sent to your API)

  • input (Required): The main user query or test input.
  • files: Array of file attachments (for multimodal testing).
  • conversation_id: Conversation tracking identifier (session_id, thread_id, and chat_id are accepted aliases).

Response Variables (Extracted from your API)

  • output (Required): The main response text to be evaluated.
  • context: Additional context or retrieved documents.
  • metadata: Structured data like token counts or model version.
  • tool_calls: Any tool or function calls made during generation.

Mapping Example

If your API expects a body like {"query": "Hello"} and returns {"data": {"answer": "Hi there"}}, you would configure the mapping as follows:

Request Mapping (Jinja2):

JSON Request Template
{
  "query": "{{ input }}"
}

Response Mapping (JSONPath):

JSON Response Mapping
{
  "output": "$.data.answer"
}

Once your application is connected as an Endpoint, you are ready for the final step: Run Evaluations.

Setup EnvironmentRun Evaluations