Installation & Setup

The Rhesis Python SDK generates test sets, runs evaluations, and traces LLM applications from code.

Using the web app instead? The same workflows are available in the browser. Start with Getting Started and the tour.

Installation

terminal
pip install rhesis-sdk

This installs the core SDK, which is enough to connect your application to Rhesis. The next section shows when you need more.

What to install

Install	Use it for	Command
Core (about 35 MB)	Connecting your application to Rhesis: the connector, `@endpoint`, `@observe` and tracing, `RhesisClient` and the entities (test sets, endpoints, test runs), `@metric`, and the native judges and synthesizers with the Rhesis-hosted model	`pip install rhesis-sdk`
`[all]` (about 560 MB)	Every other SDK feature: other model providers (OpenAI, Gemini, Anthropic, Vertex AI and the rest), DeepEval and DeepTeam metrics, document extraction and chunking for synthesizer sources, and MCP agents	`pip install "rhesis-sdk[all]"`
Add-ons	`huggingface` for local Hugging Face models (installs torch), `garak` for Garak metrics, and the tracing integrations: `langchain`, `langgraph`, `autogen`, `pydantic-ai`, `agent-framework`, `google-adk`, `haystack`, `all-integrations`	`pip install "rhesis-sdk[garak]"`

Extras combine, so rhesis-sdk[all,huggingface] installs every SDK feature plus local Hugging Face models. If you use a feature that needs [all] without it, the SDK raises an error that names the feature and the install command.

Upgrading from 0.17

In 0.18.0 the default install became the core SDK, and [all] changed meaning.

To keep the 0.17 behaviour, install pip install "rhesis-sdk[all]".
[all] no longer includes the huggingface extra. For local Hugging Face models, install pip install "rhesis-sdk[all,huggingface]".
[all] no longer includes the tracing integrations. Their extras are unchanged, so install the one you use, for example pip install "rhesis-sdk[langchain]".

Getting Started

The SDK can be used standalone or integrated with our platform. To communicate with our platform and to use LLMs provided by us you need to obtain an API key.

Obtain an API key

Visit Rhesis App
Sign up for a Rhesis account
Navigate to your account settings
Generate a new API key

Your API key will be in the format rh-XXXXXXXXXXXXXXXXXXXX. Keep this key secure and never share it publicly.

Configure the SDK

You can configure the SDK either through environment variables or direct configuration:

Environment Variables:

terminal
export RHESIS_API_KEY="your-api-key"
export RHESIS_BASE_URL="https://api.rhesis.ai"

Direct Configuration:

config.py
import rhesis.sdk

rhesis.sdk.api_key = "your-api-key"
rhesis.sdk.base_url = "https://api.rhesis.ai"  # optional

An API key bound to a project only reaches that project. An organization-wide key reaches only organization-level data unless you name a project, with RHESIS_PROJECT_ID or rhesis.sdk.project_id. Every SDK request is then scoped to that project. If the key can’t reach the project you name, or it doesn’t exist, requests fail with a 403 that names it.

Basic Usage

Once configured, you can start using the SDK to create and manage tests:

basic_usage.py
import os
from pprint import pprint

from rhesis.sdk.entities import TestSets
from rhesis.sdk.synthesizers import PromptSynthesizer

os.environ["RHESIS_API_KEY"] = "rh-your-api-key"  # Get from app.rhesis.ai settings
os.environ["RHESIS_BASE_URL"] = "https://api.rhesis.ai"  # optional

# Browse available test sets
for test_set in TestSets.all():
    pprint(test_set)

# Generate custom test scenarios
synthesizer = PromptSynthesizer(
    prompt="Generate tests for a medical chatbot that must never provide diagnosis",
)
test_set = synthesizer.generate(num_tests=10)
pprint(test_set.tests)

Next steps

Generate a test set with a synthesizer
Run tests against an endpoint with test execution
Score results with metrics

Full class and method documentation is in the API reference .