Installation & Setup

The Rhesis Python SDK generates test sets, runs evaluations, and traces LLM applications from code.

Using the web app instead? The same workflows are available in the browser. Start with Getting Started and the tour.

Installation

terminal
pip install rhesis-sdk

Getting Started

The SDK can be used standalone or integrated with our platform. To communicate with our platform and to use LLMs provided by us you need to obtain an API key.

Obtain an API key

Visit Rhesis App
Sign up for a Rhesis account
Navigate to your account settings
Generate a new API key

Your API key will be in the format rh-XXXXXXXXXXXXXXXXXXXX. Keep this key secure and never share it publicly.

Configure the SDK

You can configure the SDK either through environment variables or direct configuration:

Environment Variables:

terminal
export RHESIS_API_KEY="your-api-key"
export RHESIS_BASE_URL="https://api.rhesis.ai"

Direct Configuration:

config.py
import rhesis

rhesis.api_key = "your-api-key"
rhesis.base_url = "https://api.rhesis.ai"  # optional

Basic Usage

Once configured, you can start using the SDK to create and manage tests:

basic_usage.py
import os
from pprint import pprint

from rhesis.sdk.entities import TestSet
from rhesis.sdk.synthesizers import PromptSynthesizer

os.environ["RHESIS_API_KEY"] = "rh-your-api-key"  # Get from app.rhesis.ai settings
os.environ["RHESIS_BASE_URL"] = "https://api.rhesis.ai"  # optional

# Browse available test sets
for test_set in TestSet().all():
    pprint(test_set)

# Generate custom test scenarios
synthesizer = PromptSynthesizer(
    prompt="Generate tests for a medical chatbot that must never provide diagnosis",
)
test_set = synthesizer.generate(num_tests=10)
pprint(test_set.tests)

Next steps

Generate a test set with a synthesizer
Run tests against an endpoint with test execution
Score results with metrics

Full class and method documentation is in the API reference .