# Rhesis Documentation

> Rhesis: AI-powered testing and evaluation platform for Gen AI applications. Collaborative testing that brings together developers, domain experts, and stakeholders.

## Agent reference

> Read this section first if you are an LLM agent (Cursor, Claude Code, Telemachus, MCP).
> Fetch linked `.md` URLs — do not scrape HTML. One concept per fetch; do not load llms-full.txt unless you need the full corpus.

- [Agent reference](https://docs.rhesis.ai/docs/agent-skill/for-agents.md): routing, confusions, plan tables
- [Glossary](https://docs.rhesis.ai/glossary.md): platform terms (e.g. `glossary/behavior.md`, `glossary/test-result.md`, `glossary/metric-scope.md`, `glossary/tag.md`)
- [Metric scope](https://docs.rhesis.ai/docs/metrics/metric-scope.md): Single-Turn vs Multi-Turn alignment (critical)
- [Requirements workflow](https://docs.rhesis.ai/docs/agent-skill/requirements.md): requirements → test foundation
- [Architect workflow](https://docs.rhesis.ai/docs/architect/workflow.md): native UI agent phases
- [Agent skill install](https://docs.rhesis.ai/docs/agent-skill/platform.md): MCP + `npx skills add`
- Golden requirements example (repo only): https://github.com/rhesis-ai/rhesis/blob/main/skills/rhesis/references/use-case-bracketfeld.md

## Docs

- [Rhesis Documentation](https://docs.rhesis.ai/docs.md): **More than just evals. The testing platform for AI teams.**
- [Acknowledgments](https://docs.rhesis.ai/docs/acknowledgments.md): Rhesis includes and depends on the open-source projects below. Thanks to their developers and maintainers.
- [Agent Skill](https://docs.rhesis.ai/docs/agent-skill.md): Rhesis ships one **agent skill** () that covers the full workflow via the Rhesis MCP server — from requirements to test execution.
- [Annotations](https://docs.rhesis.ai/docs/annotations.md): Browse human reviews across test results and traces from **Improve → Annotations**.
- [API Tokens](https://docs.rhesis.ai/docs/api-tokens.md): API tokens authenticate programmatic access to Rhesis via the REST API or Python SDK. Generate and manage them at [app.rhesis.ai/tokens](https://app.rhesis.ai/t...
- [Rhesis Architect](https://docs.rhesis.ai/docs/architect.md): Architect (also known as Telemachus) is a conversational agent that designs, builds, and runs test suites for your AI endpoints. Describe what you want to test ...
- [Behaviors](https://docs.rhesis.ai/docs/behaviors.md): Behaviors are the expectations for how your LLM application should perform — response quality, safety, brand voice, and so on. You attach metrics to a behavior ...
- [Core Concepts – Rhesis](https://docs.rhesis.ai/docs/concepts.md): Rhesis evaluates whether your LLM application meets the quality bar you care about. This page has two parts: the **workflow** (what you do and in what order) an...
- [Deployment](https://docs.rhesis.ai/docs/deployment.md): You can deploy Rhesis on your own infrastructure. We provide rich guides from very easy quick start with one command, to more advanced configurations like Kuber...
- [Endpoints – Rhesis](https://docs.rhesis.ai/docs/endpoints.md): An endpoint is how Rhesis connects to your AI model or application to send test inputs and evaluate responses.
- [Experiments](https://docs.rhesis.ai/docs/experiments.md): An **experiment** is one attempt at a parameter configuration. **Versions** are its immutable history; **environments** are how running code and test runs find ...
- [Test Explorer](https://docs.rhesis.ai/docs/explorer.md): Test Explorer is an interactive workspace for building a test set. Draft or import tests, invoke your endpoint, and see responses and metric scores in the same ...
- [Getting Started with Rhesis](https://docs.rhesis.ai/docs/getting-started.md): This guide takes you from a fresh environment to your first automated evaluation of your LLM application.
- [Integrations](https://docs.rhesis.ai/docs/integrations.md): Rhesis plugs into your LLM stack across four layers, each addressing a different concern. This page is the single map: pick the layer you care about and follow ...
- [Knowledge](https://docs.rhesis.ai/docs/knowledge.md): Knowledge is domain-specific content you add to Rhesis to generate relevant, context-aware test cases. It is made up of **Sources** — documents or pages importe...
- [Metrics](https://docs.rhesis.ai/docs/metrics.md): Metrics are the quantifiable measurements that evaluate an AI response and decide whether it meets your requirements. They are organized by [**Behaviors**](/doc...
- [Models](https://docs.rhesis.ai/docs/models.md): Models are the LLM providers Rhesis uses for three purposes:
- [Organizations & Team](https://docs.rhesis.ai/docs/organizations.md): Manage your organization's profile, settings, and team members.
- [Playground](https://docs.rhesis.ai/docs/playground.md): Interact with any endpoint — send messages, attach files, view traces, and capture test cases, no code required.
- [Product Tour](https://docs.rhesis.ai/docs/product-tour.md): You spent weeks or months building something innovative. Don't let inadequate testing ruin the release. Traditional testing approaches fail with Gen AI because ...
- [Results Overview](https://docs.rhesis.ai/docs/results-overview.md): The Test Results page gives an overview of the test results across test runs and across projects. Analyze aggregated [Test Run](/docs/test-runs) performance acr...
- [Tasks](https://docs.rhesis.ai/docs/tasks.md): Tasks help teams coordinate testing work within Rhesis — tracking test failures, planning test development, and organizing reviews. Each task can be linked to a...
- [Test Results](https://docs.rhesis.ai/docs/test-results.md): **This page has moved.** Test result analytics now live under [Insights](/docs/results-overview).
- [Test Runs](https://docs.rhesis.ai/docs/test-runs.md): A test run is created when you execute a test set against an endpoint. It captures all individual test results with detailed metrics, conversation history, exec...
- [Test Sets](https://docs.rhesis.ai/docs/test-sets.md): Test sets group related tests into collections for batch execution. When you [generate tests](/docs/test-sets/tests-generation), all created tests are automatic...
- [Test Sets](https://docs.rhesis.ai/docs/test-sets-runs.md): **This page has moved.** Test sets are now documented under [Test Sets](/docs/test-sets).
- [Tests](https://docs.rhesis.ai/docs/tests.md): Tests are individual cases that validate specific inputs and expected outputs for your LLM application, evaluated using assigned metrics.
- [Tools](https://docs.rhesis.ai/docs/tools.md): Tools connect external services to Rhesis so you can export their content into your [Knowledge](/docs/knowledge) base for use in test generation and other workf...
- [Tracing – Rhesis](https://docs.rhesis.ai/docs/tracing.md): Tracing gives your LLM applications OpenTelemetry-based observability: it captures traces of LLM calls, tool invocations, and retrieval operations through a sem...
- [Agent reference](https://docs.rhesis.ai/docs/agent-skill/for-agents.md): This page is for **AI agents** (Cursor, Claude Code, Telemachus, MCP). Humans should start with [Platform workflow](/docs/agent-skill/platform) or [Core Concept...
- [Platform workflow](https://docs.rhesis.ai/docs/agent-skill/platform.md): Use Rhesis from Claude Code, Cursor, or any compatible AI interface — without the Rhesis web UI.
- [Requirements workflow](https://docs.rhesis.ai/docs/agent-skill/requirements.md): Build a **test foundation** from a **Product Requirements Document (PRD)**, product spec, or other requirements — behaviors, custom metrics, tags, and generated...
- [Chat Features](https://docs.rhesis.ai/docs/architect/chat-features.md): You can attach documents to any message. Architect extracts the text and uses it as context for that turn — for example, to ground test generation in a spec you...
- [Running and Analyzing](https://docs.rhesis.ai/docs/architect/execution-and-analysis.md): After you approve the plan, Architect runs the tests and presents results in a structured format. You can also ask it to analyze existing runs or compare two ru...
- [Endpoint Exploration](https://docs.rhesis.ai/docs/architect/exploration.md): Before designing a test suite, Architect explores your endpoint — running real conversations against it to understand what it does and how it behaves.
- [Planning Test Suites](https://docs.rhesis.ai/docs/architect/planning.md): After exploration, Architect proposes a structured plan. You review it, ask for changes, and approve before anything is created.
- [Scenarios](https://docs.rhesis.ai/docs/architect/scenarios.md): Common requests and what to expect from each one.
- [Workflow](https://docs.rhesis.ai/docs/architect/workflow.md): An Architect session follows four phases — discovery, planning, creating, executing — with an explicit review gate before anything is created. You stay in contr...
- [Docker Compose](https://docs.rhesis.ai/docs/deployment/docker-compose.md): Deploy Rhesis on your own infrastructure with proper security, authentication, and production configuration.
- [Environment Variables](https://docs.rhesis.ai/docs/deployment/environment-variables.md): Complete reference for configuring a self-hosted Rhesis deployment. For a guided walkthrough of the essentials, see the [Docker Compose](/docs/deployment/docker...
- [Quick Start](https://docs.rhesis.ai/docs/deployment/quick-start.md): Get Rhesis running locally in under 5 minutes. This setup is for local testing only.
- [Creating Endpoints](https://docs.rhesis.ai/docs/endpoints/creating-endpoints.md): There are two ways to set up an endpoint:
- [Managing Endpoints](https://docs.rhesis.ai/docs/endpoints/management.md): Open an endpoint and go to the **Test** tab to send a live request and verify your configuration before running full test suites.
- [Mapping Examples](https://docs.rhesis.ai/docs/endpoints/mapping-examples.md): Practical examples showing how to configure request and response mappings for different API patterns. Each example includes the request body template, response ...
- [Request Mapping](https://docs.rhesis.ai/docs/endpoints/request-mapping.md): Request body templates define how Rhesis formats its [platform variables](/docs/endpoints#platform-managed-variables) into the JSON structure your API expects. ...
- [Response Mapping](https://docs.rhesis.ai/docs/endpoints/response-mapping.md): Response mappings tell Rhesis how to extract [platform variables](/docs/endpoints#platform-managed-variables) from your API's response. Each field is evaluated ...
- [SDK Endpoints](https://docs.rhesis.ai/docs/endpoints/sdk-endpoints.md): Register Python functions as testable endpoints using the Rhesis SDK. Instead of configuring endpoints manually in the dashboard, you decorate your functions an...
- [Parameter Injection](https://docs.rhesis.ai/docs/experiments/connector-injection.md): Experiment parameters flow through for both REST and SDK connector endpoints. The same syntax works everywhere.
- [Parameter Schema](https://docs.rhesis.ai/docs/experiments/parameter-schema.md): A parameter schema declares the typed knobs your project supports — what can be configured, not what the values are. [Experiments](/docs/experiments) fill in th...
- [SDK Usage](https://docs.rhesis.ai/docs/experiments/sdk-usage.md): How your application code reads parameter values, with the caching and override semantics needed for production.
- [Building and Evaluating](https://docs.rhesis.ai/docs/explorer/building-and-evaluating.md): This page covers the core Test Explorer workflow: create a session, choose the endpoint and metrics, build a topic tree, generate outputs, evaluate, and export.
- [Scenarios](https://docs.rhesis.ai/docs/explorer/scenarios.md): Common ways to use Test Explorer and what to expect from each one.
- [Workflow](https://docs.rhesis.ai/docs/explorer/workflow.md): Test Explorer combines manual iteration with a **suggestions pipeline**: you organize tests and topics, then optionally generate a batch of new prompts, embed a...
- [Connect Your Application – Rhesis](https://docs.rhesis.ai/docs/getting-started/connecting-application.md): An **Endpoint** is the connection point between Rhesis and your LLM application: Rhesis sends test inputs to it and records the responses for evaluation.
- [Default Insurance Chatbot (Rosalind)](https://docs.rhesis.ai/docs/getting-started/default-chatbot.md): Rosalind is a pre-configured insurance chatbot powered by Google's Gemini, available to all new users as a demo endpoint during onboarding. It lets you generate...
- [Projects](https://docs.rhesis.ai/docs/getting-started/projects.md): Projects are the top-level organizational unit in Rhesis: each project groups the endpoints, tests, test sets, and results for one LLM application or testing in...
- [Run Evaluations](https://docs.rhesis.ai/docs/getting-started/run-evaluations.md): In Rhesis, a **Test Set** is a collection of tests — single-turn prompts or multi-turn conversation scenarios, each with optional context and expected outcomes ...
- [Setup Environment](https://docs.rhesis.ai/docs/getting-started/setup-environment.md): Rhesis runs in three ways: the managed Cloud Platform, locally via Docker, or self-hosted on your own infrastructure.
- [Code Metrics](https://docs.rhesis.ai/docs/metrics/code-metrics.md): Code Metrics let you write evaluation logic directly in Python — on your own machine or infrastructure — and have the Rhesis backend invoke it automatically dur...
- [DeepEval](https://docs.rhesis.ai/docs/metrics/deepeval.md): [DeepEval](https://github.com/confident-ai/deepeval) provides single-turn and conversational metrics for RAG, agents, toxicity, bias, and PII checks. Rhesis exp...
- [Metric scope](https://docs.rhesis.ai/docs/metrics/metric-scope.md): Every metric declares ****. Every test set declares ****. At run time the platform keeps only metrics whose scope includes the test's type. **Mismatches are sil...
- [Ragas](https://docs.rhesis.ai/docs/metrics/ragas.md): [Ragas](https://github.com/explodinggradients/ragas) evaluates RAG pipelines with referenceless and reference-based metrics (faithfulness, context relevance, an...
- [Trace Metrics](https://docs.rhesis.ai/docs/metrics/trace-metrics.md): Trace Metrics let you run evaluation metrics against live production traces automatically. Every time your endpoint handles a request, Rhesis can evaluate the r...
- [API Clients](https://docs.rhesis.ai/docs/organizations/api-clients.md): API Clients let external services exchange an organization-scoped OIDC access token for a Rhesis JWT using RFC 8693 token exchange. Use them for machine-to-mach...
- [Roles & Permissions](https://docs.rhesis.ai/docs/organizations/roles.md): Rhesis controls what each member can do through roles, assigned at the organization level and, optionally, overridden per project.
- [Single Sign-On](https://docs.rhesis.ai/docs/organizations/sso.md): Single Sign-On lets an Enterprise Edition organization authenticate users through its own OIDC provider. Use it when your organization needs centralized identit...
- [Test Reviews](https://docs.rhesis.ai/docs/tasks/test-reviews.md): A test review is a human judgement that overrides the automated pass/fail verdict for a test result. When a test run produces results you disagree with, reviews...
- [Test Execution – Rhesis](https://docs.rhesis.ai/docs/test-runs/execution.md): Executing a test set runs it against an endpoint to evaluate your LLM application. Each execution creates a [Test Run](/docs/test-runs) containing all individua...
- [Import from File](https://docs.rhesis.ai/docs/test-sets/import-from-file.md): Import an existing collection of tests directly from a CSV, Excel, JSON, or JSONL file to create a new test set in seconds.
- [Import from Garak](https://docs.rhesis.ai/docs/test-sets/import-from-garak.md): [Garak](https://github.com/NVIDIA/garak) is an open-source LLM vulnerability scanner developed by NVIDIA. Rhesis integrates Garak directly into the platform so ...
- [Test Generation](https://docs.rhesis.ai/docs/test-sets/tests-generation.md): Generate test sets automatically from templates or a custom description.
- [Adversarial Testing](https://docs.rhesis.ai/docs/tests/adversarial-testing.md): Standard software testing verifies that systems work under expected conditions. Adversarial testing probes for failures when inputs are unexpected, malicious, o...
- [Conversation Simulation](https://docs.rhesis.ai/docs/tests/conversation-simulation.md): Conversation simulation is powered by Penelope, an autonomous testing agent that runs multi-turn test scenarios against conversational LLM applications. You def...
- [Multi-modal Testing](https://docs.rhesis.ai/docs/tests/multimodal-testing.md): Multi-modal testing lets you attach files to tests and have them sent to your endpoint alongside the prompt. Use this to test how your AI handles images, docume...
- [Microsoft Agent Framework](https://docs.rhesis.ai/docs/tracing/agent-framework.md): Zero-config tracing for [Microsoft Agent Framework](https://github.com/microsoft/agent-framework) (MAF) agents and workflows.
- [Auto-Instrumentation](https://docs.rhesis.ai/docs/tracing/auto-instrumentation.md): Zero-config tracing for LangChain, LangGraph, Microsoft Agent Framework, and Pydantic AI applications. It traces LLM calls, tool invocations, and chain executio...
- [Conversation Tracing](https://docs.rhesis.ai/docs/tracing/conversation-tracing.md): Conversation tracing groups the traces from a multi-turn session into a single thread, so you can navigate every turn and agent handoff in one view alongside th...
- [Custom Spans](https://docs.rhesis.ai/docs/tracing/custom-spans.md): Create custom spans with specific names and attributes for advanced observability needs.
- [Decorators](https://docs.rhesis.ai/docs/tracing/decorators.md): Two decorators are available: for tracing-only instrumentation, and for functions that Rhesis can **invoke from the platform** (remote testing) while still emit...
- [Multi-Agent Tracing](https://docs.rhesis.ai/docs/tracing/multi-agent.md): Multi-agent tracing captures how agents in a system interact — which agent ran, what it received and produced, and when control passed from one agent to another...
- [Semantic Conventions](https://docs.rhesis.ai/docs/tracing/semantic-conventions.md): Rhesis uses a semantic layer for consistent, framework-agnostic span naming across all AI operations.
- [Setup](https://docs.rhesis.ai/docs/tracing/setup.md): Configure tracing in your application to start capturing traces.
- [Polyphemus](https://docs.rhesis.ai/docs/tests/adversarial-testing/polyphemus.md): Polyphemus is a Rhesis-managed language model built for adversarial test generation. It is designed to produce the kinds of prompts that commercial models routi...
- [Requesting Access](https://docs.rhesis.ai/docs/tests/adversarial-testing/requesting-access.md): Polyphemus can generate sensitive content for evaluation purposes. For this reason, access is not automatic—each request is reviewed by the Rhesis team before a...
- [Using Polyphemus with the SDK](https://docs.rhesis.ai/docs/tests/adversarial-testing/sdk-usage.md): Once your access is approved, Polyphemus integrates with the Rhesis SDK the same way any other model provider does. You can use it with any synthesizer to gener...
- [Configuration](https://docs.rhesis.ai/docs/tests/conversation-simulation/configuration.md): Configure Penelope's behavior through agent initialization, environment variables, or programmatic configuration.
- [Examples & Use Cases](https://docs.rhesis.ai/docs/tests/conversation-simulation/examples.md): Complete working examples and practical patterns for testing conversational AI systems with Penelope.
- [Execution Trace](https://docs.rhesis.ai/docs/tests/conversation-simulation/execution-trace.md): Penelope captures a structured, machine-readable execution trace for every multi-turn test run — for analysis, debugging, and integration with metrics systems.
- [Extending Penelope](https://docs.rhesis.ai/docs/tests/conversation-simulation/extending.md): Understand Penelope's architecture and extend it with custom tools for specialized testing needs.
- [Getting Started](https://docs.rhesis.ai/docs/tests/conversation-simulation/getting-started.md): Install Penelope and run your first test.

## Guides

- [Guides – Rhesis](https://docs.rhesis.ai/guides.md): Step-by-step guides for testing LLM applications with Rhesis, from a 10-minute quickstart to custom metrics and CI/CD.
- [How to Integrate Rhesis SDK into Your CI/CD Pipeline](https://docs.rhesis.ai/guides/ci-cd-integration.md): Run LLM tests on every commit or pull request so regressions get caught before production, with a failing test set blocking the pipeline. This guide sets up aut...
- [Building Custom Metrics with the Rhesis SDK](https://docs.rhesis.ai/guides/custom-metrics.md): While Rhesis provides integrations with DeepEval, Ragas, and other evaluation frameworks, you'll often need custom metrics tailored to your specific use case. T...
- [How to Start Testing LLM and Agentic Apps in 10 Minutes with Rhesis AI](https://docs.rhesis.ai/guides/quick-start-guide.md): Testing LLM and agentic apps is challenging: outputs are non-deterministic, edge cases are unpredictable, and manual testing doesn't scale. This guide gets a co...
- [Configuring Telemetry for Your AI Application](https://docs.rhesis.ai/guides/telemetry-configuration.md): Rhesis traces every LLM call, tool invocation, retrieval, and agent handoff your application makes, then renders them as a navigable trace tree in the dashboard...
- [How to: Testing user journeys of AI agents](https://docs.rhesis.ai/guides/testing-user-journeys.md): You spent weeks crafting your agent: mapping **user journeys**, writing requirements, implementing features and tuning prompts and tools until the experience fe...
- [Configure Telemetry: Auto-Instrumentation](https://docs.rhesis.ai/guides/telemetry-configuration/auto-instrumentation.md): Auto-instrumentation traces every LLM call, tool invocation, and chain or graph execution in supported frameworks without touching your application code — one c...
- [Configure Telemetry: Decorators](https://docs.rhesis.ai/guides/telemetry-configuration/decorators.md): Decorators sit between zero-code auto-instrumentation and manual span creation. Wrap any function with (or a typed variant) and the SDK handles span naming, I/O...
- [Configure Telemetry: OTel GenAI Semantic Conventions](https://docs.rhesis.ai/guides/telemetry-configuration/genai-conventions.md): If you're building a framework, orchestrator, or agent runtime, implement the [OpenTelemetry Semantic Conventions for GenAI](https://github.com/open-telemetry/s...
- [Configure Telemetry: Raw OpenTelemetry](https://docs.rhesis.ai/guides/telemetry-configuration/raw-opentelemetry.md): For cases where you can't decorate functions (third-party code, dynamic dispatch) or need fine-grained control over span names, attributes, and events, use the ...

## SDK

- [Agents](https://docs.rhesis.ai/sdk/agents.md): The SDK agents framework lets you build conversational workflows that can reason, call tools, and return structured results. Use it when a workflow needs multip...
- [RhesisClient – Rhesis Python SDK](https://docs.rhesis.ai/sdk/client.md): The is the central entry point for the Rhesis SDK. It initializes telemetry, manages the connector for remote testing, and provides access to the Rhesis API.
- [Connector](https://docs.rhesis.ai/sdk/connector.md): Register Python functions as testable endpoints in Rhesis using the SDK connector. This code-first approach automatically creates and manages endpoints, providi...
- [Entities](https://docs.rhesis.ai/sdk/entities.md): Entities are typed Python objects that map to platform resources, letting you create, read, update, and delete data programmatically. They appear throughout the...
- [Test Execution](https://docs.rhesis.ai/sdk/execution.md): The SDK provides methods on for executing tests against endpoints, re-scoring existing outputs with different metrics, and managing which metrics are assigned t...
- [SDK Installation & Setup – Rhesis](https://docs.rhesis.ai/sdk/installation.md): The Rhesis Python SDK generates test sets, runs evaluations, and traces LLM applications from code.
- [Overview](https://docs.rhesis.ai/sdk/metrics.md): The Rhesis SDK evaluates LLM applications with built-in metrics from several frameworks, custom metrics you define, and metrics managed on the platform.
- [Models](https://docs.rhesis.ai/sdk/models.md): Models generate test data for LLM applications and act as LLM judges during evaluation. A single function, , creates instances across every supported provider.
- [Parameters & Experiments](https://docs.rhesis.ai/sdk/parameters.md): Manage typed configuration for your LLM application. Define a schema once, create experiments with different values, and resolve the active configuration at run...
- [Statistics](https://docs.rhesis.ai/sdk/statistics.md): Two collection methods return aggregated statistics across your organization's test runs and test results, for building dashboards or reports:
- [Synthesizers](https://docs.rhesis.ai/sdk/synthesizers.md): Generate test sets for evaluating LLM applications. Synthesizers use an LLM to create test cases from prompts, configurations, or source documents.
- [Parameter Binding](https://docs.rhesis.ai/sdk/connector/binding.md): The parameter injects infrastructure dependencies (database connections, configuration, auth context) into your endpoint functions without exposing them in the ...
- [Examples](https://docs.rhesis.ai/sdk/connector/examples.md): Complete working examples demonstrating various SDK connector patterns.
- [File Attachments in SDK Connector Endpoints](https://docs.rhesis.ai/sdk/connector/files.md): Rhesis passes file attachments through SDK connector executions as metadata-rich references instead of embedding raw bytes in the WebSocket payload. Use when yo...
- [Input/Output Mapping](https://docs.rhesis.ai/sdk/connector/mapping.md): Map your function's parameters and return values to Rhesis's standardized request/response format. The SDK supports both automatic detection and manual configur...
- [Advanced Mapping](https://docs.rhesis.ai/sdk/connector/serializers.md): Learn how to map complex objects like Pydantic models, dataclasses, and custom types as function parameters and return values. The SDK automatically handles ser...
- [Endpoints](https://docs.rhesis.ai/sdk/entities/endpoints.md): An endpoint is an LLM application or API that tests run against. It defines how Rhesis connects to your application, sends test inputs, and receives responses f...
- [Experiments](https://docs.rhesis.ai/sdk/entities/experiments.md): An experiment is a named bundle of parameter values for a project. Each save creates an immutable sequential version such as , , or .
- [Files](https://docs.rhesis.ai/sdk/entities/files.md): Files are binary attachments (images, PDFs, audio) that can be associated with Tests and TestResults. They are managed through a dedicated upload API rather tha...
- [Model Entity](https://docs.rhesis.ai/sdk/entities/models.md): The Model entity stores an LLM configuration on the platform: the provider, model name, and API key. Once saved, a model can be set as a default for generation ...
- [Projects](https://docs.rhesis.ai/sdk/entities/projects.md): A project is the top-level organizational unit, holding test sets, endpoints, and other resources for one LLM application.
- [Status](https://docs.rhesis.ai/sdk/entities/status.md): Statuses track the state of entities throughout the platform. Each status is scoped to an **entity type**, meaning different entities have different applicable ...
- [Test Attributes](https://docs.rhesis.ai/sdk/entities/test-attributes.md): Test attribute entities provide structure and classification for tests. Categories, topics, and behaviors help organize test sets and enable filtered evaluation...
- [Test Runs & Results](https://docs.rhesis.ai/sdk/entities/test-runs.md): Test runs track the execution of tests against endpoints. Each run produces test results containing the endpoint's response, evaluation metrics, and review stat...
- [Test Sets & Tests](https://docs.rhesis.ai/sdk/entities/test-sets.md): Test sets are collections of individual test entities used to evaluate LLM applications. Each test entity contains a prompt, expected behaviors, and metadata fo...
- [Conversational Metrics](https://docs.rhesis.ai/sdk/metrics/conversational.md): Conversational metrics (multi-turn metrics) evaluate interactions across multiple turns — coherence, goal achievement, role adherence, and tool usage in extende...
- [Single-Turn Metrics](https://docs.rhesis.ai/sdk/metrics/single-turn.md): Single-turn metrics evaluate an individual exchange between a user input and a system output — standalone responses, RAG systems, and classification tasks.

## Contribute

- [Contribute](https://docs.rhesis.ai/contribute.md): Technical documentation for developers working on and with Rhesis. Contributions of all kinds are welcome: code, documentation, and bug reports.
- [Backend Documentation](https://docs.rhesis.ai/contribute/backend.md): This section is about the Rhesis backend: a **FastAPI** application that exposes REST APIs, handles authentication and organizations, and coordinates work with ...
- [Coding Standards](https://docs.rhesis.ai/contribute/coding-standards.md): Conventions for Python and JavaScript/TypeScript code in the Rhesis repo. Ruff enforces the Python rules; ESLint and Prettier enforce the JS/TS rules.
- [Connector](https://docs.rhesis.ai/contribute/connector.md): The connector enables bidirectional communication between your application (via the Rhesis SDK), the Rhesis backend, and worker processes. It allows remote func...
- [Development setup](https://docs.rhesis.ai/contribute/development-setup.md): Two paths: **local dev** () for day-to-day work, or **all-in-one Docker** () for a full environment without local service wiring.
- [Environment Variables](https://docs.rhesis.ai/contribute/environment-variables.md): Local development configuration for the backend, frontend, and Celery workers. For the full catalog — every variable, what it means, and whether it is required ...
- [Frontend Documentation](https://docs.rhesis.ai/contribute/frontend.md): Welcome to the Rhesis Frontend documentation. This section covers the React-based frontend application that provides the user interface for the Rhesis platform.
- [Managing Documentation](https://docs.rhesis.ai/contribute/managing-docs.md): How to run, extend, and maintain the Rhesis docs site (Nextra + Next.js). Source lives under ; MDX content is in .
- [Polyphemus (Development)](https://docs.rhesis.ai/contribute/polyphemus.md): Polyphemus is the model-serving service used for adversarial generation workloads. It proxies generation requests to Vertex AI and exposes authenticated REST en...
- [Telemetry System](https://docs.rhesis.ai/contribute/telemetry.md): How Rhesis collects and stores usage analytics from cloud-hosted and self-hosted instances, for developers and contributors working on the telemetry pipeline.
- [Tracing System](https://docs.rhesis.ai/contribute/tracing-system.md): Technical documentation for the Rhesis tracing system architecture and implementation.
- [API Clients (Token Exchange)](https://docs.rhesis.ai/contribute/backend/api-clients.md): The **API Clients** feature lets a tenant integrate machine-to-machine workloads with Rhesis without minting long-lived Rhesis API tokens by hand. The integrati...
- [API Structure](https://docs.rhesis.ai/contribute/backend/api-structure.md): FastAPI routers under , one resource family per module where possible. Paths use normal REST patterns unless noted below.
- [Architect Chat System](https://docs.rhesis.ai/contribute/backend/architect-chat-system.md): The backend Architect system combines REST session APIs, WebSocket event streaming, Celery orchestration, and in-process MCP tool dispatch to support conversati...
- [Backend Architecture](https://docs.rhesis.ai/contribute/backend/architecture.md): FastAPI app with routers, services, SQLAlchemy models, and OpenAPI docs out of the box.
- [Authentication](https://docs.rhesis.ai/contribute/backend/authentication.md): The Rhesis backend implements a native authentication system with a pluggable provider architecture. The system supports email/password authentication, OAuth pr...
- [Authorization (RBAC)](https://docs.rhesis.ai/contribute/backend/authorization.md): This page documents the backend's role-based access control (RBAC) system: how every allow/deny decision is made, where it is enforced, and how to extend it. Fo...
- [Background Tasks](https://docs.rhesis.ai/contribute/backend/background-tasks.md): The Rhesis backend uses Celery to handle asynchronous background tasks. This allows the API to offload time-consuming operations and improve responsiveness. The...
- [Cascade Deletion and Restoration](https://docs.rhesis.ai/contribute/backend/cascade-operations.md): Configured parent/child relationships drive bulk soft-delete and restore so / stay thin: you declare edges once in instead of repeating queries in every CRUD he...
- [Database Field Encryption for Sensitive Tokens](https://docs.rhesis.ai/contribute/backend/database-field-encryption.md): The Rhesis backend currently stores sensitive credentials as plaintext in the PostgreSQL database, including:
- [Database Models](https://docs.rhesis.ai/contribute/backend/database-models.md): The Rhesis backend uses SQLAlchemy ORM to interact with the PostgreSQL database. All models are defined in the directory and inherit from a common class.
- [Deployment](https://docs.rhesis.ai/contribute/backend/deployment.md): This document outlines the process for deploying the Rhesis backend to various environments. The application is containerized using Docker, making it deployable...
- [Development Workflow](https://docs.rhesis.ai/contribute/backend/development-workflow.md): Recommended day-to-day workflow for the backend.
- [Email Notification System](https://docs.rhesis.ai/contribute/backend/email-notifications.md): The worker can email users when selected Celery tasks finish. SMTP is pluggable (SendGrid and others). Sending is non-blocking and respects tenant context.
- [Encryption Troubleshooting Guide](https://docs.rhesis.ai/contribute/backend/encryption-troubleshooting.md): Check Encryption Status
- [Environment Configuration](https://docs.rhesis.ai/contribute/backend/environment-config.md): Configuration is environment variables (see [12-factor app](https://12factor.net/)); different files and hosts supply values per environment.
- [Getting Started with Rhesis Backend](https://docs.rhesis.ai/contribute/backend/getting-started.md): Set up and run the backend locally for development.
- [Multi-tenancy](https://docs.rhesis.ai/contribute/backend/multi-tenancy.md): Rhesis isolates data by organization and active project. Backend code should rely on the request-scoped database session instead of threading tenant identifiers...
- [OData Query Guide](https://docs.rhesis.ai/contribute/backend/odata-guide.md): How to use OData-style query parameters such as , , , and with the Rhesis backend API.
- [Rhesis Backend Documentation](https://docs.rhesis.ai/contribute/backend/README.md): This documentation provides comprehensive information about the Rhesis backend application, which is built with FastAPI and PostgreSQL. It covers architecture, ...
- [Security Features](https://docs.rhesis.ai/contribute/backend/security.md): The Rhesis backend implements multiple layers of security to protect sensitive data and ensure secure access to the API. This document outlines the key security...
- [Security Improvements: Organization Filtering](https://docs.rhesis.ai/contribute/backend/security-improvements.md): How we tightened org-scoped queries and added regression coverage so one tenant cannot read or mutate another’s rows through ordinary code paths.
- [Soft Deletion Architecture](https://docs.rhesis.ai/contribute/backend/soft-deletion.md): Soft-deleted rows stay in the database ( set) but are hidden from normal queries. That preserves history and FK integrity while still supporting restore and adm...
- [Test Result Statistics API](https://docs.rhesis.ai/contribute/backend/test-result-stats.md): aggregates for dashboards: multiple values, rich filters, and compact payloads.
- [Test Result Status](https://docs.rhesis.ai/contribute/backend/test-result-status.md): This guide explains how individual test result statuses are determined and what they represent in the Rhesis backend.
- [Human Reviews for Test Results](https://docs.rhesis.ai/contribute/backend/test-reviews.md): Human **reviews** on each test result sit beside automated : multiple reviewers, metric- or whole-test scope, and CRUD via the API. Nested holds rollups for qui...
- [Test Run Status](https://docs.rhesis.ai/contribute/backend/test-run-status.md): This guide explains how test run statuses are determined and what they represent in the Rhesis backend.
- [User Settings](https://docs.rhesis.ai/contribute/backend/user-settings.md): The User model stores preferences in a JSONB column, including LLM defaults, UI preferences, notifications, and localization settings.
- [API Integration](https://docs.rhesis.ai/contribute/frontend/api-integration.md): This document explains how the Rhesis frontend application integrates with the Rhesis backend API.
- [Architect Chat UI](https://docs.rhesis.ai/contribute/frontend/architect-chat-ui.md): This page documents the frontend architecture for the Architect chat experience in .
- [Architecture Overview](https://docs.rhesis.ai/contribute/frontend/architecture.md): This document provides a high-level overview of the Rhesis frontend architecture.
- [Authentication](https://docs.rhesis.ai/contribute/frontend/authentication.md): This document explains the authentication system used in the Rhesis frontend application.
- [Component Library](https://docs.rhesis.ai/contribute/frontend/components.md): This document provides an overview of the reusable components available in the Rhesis frontend application.
- [Getting Started](https://docs.rhesis.ai/contribute/frontend/getting-started.md): This guide will help you set up and run the Rhesis frontend application locally for development.
- [Rhesis Frontend Documentation](https://docs.rhesis.ai/contribute/frontend/README.md): Welcome to the Rhesis Frontend documentation. This documentation provides comprehensive guides and references for developers working on the Rhesis frontend appl...
- [Routing](https://docs.rhesis.ai/contribute/frontend/routing.md): This document explains the routing system used in the Rhesis frontend application, which is built with Next.js App Router.
- [State Management](https://docs.rhesis.ai/contribute/frontend/state-management.md): This document explains the state management approach used in the Rhesis frontend application.
- [Frontend Testing](https://docs.rhesis.ai/contribute/frontend/testing.md): This guide covers testing strategies and implementation for the Rhesis frontend application.
- [Architect Agent (SDK)](https://docs.rhesis.ai/contribute/sdk/architect-agent.md): The class lives in . It drives the conversational test-suite design workflow — managing mode transitions, plan state, tool confirmation, and event emission acro...
- [Getting Started](https://docs.rhesis.ai/contribute/sdk/getting-started.md): Read the root [](https://github.com/rhesis-ai/rhesis/blob/main/CONTRIBUTING.md) first for commit conventions and the pull request workflow that apply to all con...
- [Framework telemetry integrations (SDK)](https://docs.rhesis.ai/contribute/sdk/integrations.md): The Rhesis Python SDK wires **automatic** tracing for some AI frameworks through small integration classes under . Each integration subclasses and is registered...
- [Architecture](https://docs.rhesis.ai/contribute/tracing-system/architecture.md): Detailed architecture of the Rhesis tracing system.
- [Data Structures](https://docs.rhesis.ai/contribute/tracing-system/data-structures.md): Schemas, database design, and data formats for the tracing system.
- [Trace Lifecycle](https://docs.rhesis.ai/contribute/tracing-system/lifecycle.md): This page covers the complete trace lifecycle, including the race condition problem and the hybrid linking solution.
- [Architect Background Tasks](https://docs.rhesis.ai/contribute/worker/architect-background-tasks.md): This page documents the worker-side runtime for Architect chat execution and async resume behavior.
- [Worker Architecture and Dependencies](https://docs.rhesis.ai/contribute/worker/architecture.md): This document explains the relationships and dependencies between the Rhesis worker system, the backend API, and the SDK components.
- [Worker Tasks and Background Processing](https://docs.rhesis.ai/contribute/worker/background-tasks.md): The Rhesis backend uses Celery to handle asynchronous background tasks. This allows the API to offload time-consuming operations and improve responsiveness. The...
- [Execution Modes](https://docs.rhesis.ai/contribute/worker/execution-modes.md): When executing a test configuration (a set of tests against an endpoint), Rhesis supports two execution modes: **Sequential** and **Parallel**. The mode determi...
- [GKE Worker Troubleshooting Guide](https://docs.rhesis.ai/contribute/worker/gke-troubleshooting.md): This guide covers troubleshooting Celery workers running in Google Kubernetes Engine (GKE), including using the built-in debugging tools.
- [Worker Logging Guide](https://docs.rhesis.ai/contribute/worker/logging.md): This guide covers everything about logging in the Rhesis worker system, from configuration to analysis and troubleshooting.
- [Multi-Worker RPC Coordination](https://docs.rhesis.ai/contribute/worker/multi-worker-rpc.md): When running multiple backend workers (e.g., ), RPC requests need to be routed to the specific worker that holds the WebSocket connection to the SDK. This is ac...
- [Rhesis Worker Documentation](https://docs.rhesis.ai/contribute/worker/README.md): This directory contains documentation related to the Rhesis worker system, which handles background processing, task queues, and asynchronous operations using *...
- [Test Execution System](https://docs.rhesis.ai/contribute/worker/test-execution.md): The Rhesis test execution system provides a flexible and extensible architecture for running tests against endpoints. It supports multiple test types and execut...
- [Test Types](https://docs.rhesis.ai/contribute/worker/test-types.md): Rhesis supports multiple test types, each designed for different testing scenarios. The test type determines how the test is executed and evaluated.
- [Trace Ingestion Pipeline](https://docs.rhesis.ai/contribute/worker/trace-ingestion-pipeline.md): This document explains how traces are processed after arriving via , covering the full pipeline from storage through enrichment and metric evaluation.
- [Worker Troubleshooting Guide](https://docs.rhesis.ai/contribute/worker/troubleshooting.md): This document covers common issues you may encounter with the Rhesis worker system and how to resolve them.

## Optional

### Glossary

- [Glossary](https://docs.rhesis.ai/glossary.md): Loading glossary...}>
- [A/B Testing](https://docs.rhesis.ai/glossary/a-b-testing.md): Comparing two versions of a system by running identical tests against both to determine which performs better.
- [Agent](https://docs.rhesis.ai/glossary/agent.md): title: 'Agent - Glossary', description: 'An autonomous AI system that reasons, plans, and takes sequences of actions—such as calling tools or delegating to sub-...
- [Ambiguity Handling](https://docs.rhesis.ai/glossary/ambiguity-handling.md): How an AI system manages unclear, vague, or multi-interpretable user inputs that could have multiple valid meanings.
- [API Token](https://docs.rhesis.ai/glossary/api-token.md): title: 'API Token - Glossary', description: 'Authentication credentials used to integrate Rhesis with your systems programmatically via the SDK or API.', }
- [Auto-Configure](https://docs.rhesis.ai/glossary/auto-configure.md): title: 'Auto-Configure - Glossary', description: 'An AI-powered feature that generates endpoint request and response mappings by analyzing curl commands, API do...
- [Baseline](https://docs.rhesis.ai/glossary/baseline.md): A reference point established from initial test results that serves as a benchmark for comparing future performance and detecting regressions.
- [Behavior](https://docs.rhesis.ai/glossary/behavior.md): title: 'Behavior - Glossary', description: 'A formalized expectation that describes how your AI system should perform, such as response quality, safety, or accu...
- [Categorical Scoring](https://docs.rhesis.ai/glossary/categorical-scoring.md): title: 'Categorical Scoring - Glossary', description: 'A metric scoring type that classifies responses into predefined categories such as excellent, good, fair,...
- [Category](https://docs.rhesis.ai/glossary/category.md): title: 'Category - Glossary', description: 'A high-level classification for tests, such as Harmful or Harmless, used to organize and filter test cases.', }
- [Chain-of-Thought](https://docs.rhesis.ai/glossary/chain-of-thought.md): An LLM prompting technique where the model shows its reasoning process step-by-step before providing a final answer.
- [Confidence Score](https://docs.rhesis.ai/glossary/confidence-score.md): A numeric indicator of the AI's certainty or reliability in its response, used in evaluation metrics and threshold-based decisions.
- [Confusion Matrix](https://docs.rhesis.ai/glossary/confusion-matrix.md): A table showing the performance of a classification system, displaying true positives, false positives, true negatives, and false negatives.
- [Connector](https://docs.rhesis.ai/glossary/connector.md): title: 'Connector - Glossary', description: 'The SDK integration layer that connects your LLM application to Rhesis via the @endpoint decorator, enabling automa...
- [Containment Rate](https://docs.rhesis.ai/glossary/containment-rate.md): The percentage of user interactions successfully handled by the AI system without requiring escalation, human intervention, or failure.
- [Context Switching](https://docs.rhesis.ai/glossary/context-switching.md): Changing topics or focus within an ongoing conversation, testing the AI's ability to handle abrupt subject transitions.
- [Context Window](https://docs.rhesis.ai/glossary/context-window.md): The maximum amount of text (measured in tokens) that an LLM can process at once, including both input and output.
- [Edge Case](https://docs.rhesis.ai/glossary/edge-case.md): Unusual, boundary, or extreme scenarios that test the limits of an AI system's capabilities and robustness.
- [Endpoint](https://docs.rhesis.ai/glossary/endpoint.md): title: 'Endpoint - Glossary', description: 'A complete configuration for calling an external API that represents the AI services or APIs you want to test.', }
- [Evaluation Prompt](https://docs.rhesis.ai/glossary/evaluation-prompt.md): title: 'Evaluation Prompt - Glossary', description: 'Instructions provided to the judge model specifying what to evaluate and the criteria to use when assessing...
- [Evaluation Steps](https://docs.rhesis.ai/glossary/evaluation-steps.md): title: 'Evaluation Steps - Glossary', description: 'A breakdown of the evaluation process into clear steps that guide the LLM judge when producing a score and r...
- [Explorer](https://docs.rhesis.ai/glossary/explorer.md): Topic-based sessions that organize tests by topic and let suites evolve through outputs, evaluation, and AI suggestions.
- [F1 Score](https://docs.rhesis.ai/glossary/f1-score.md): The harmonic mean of precision and recall, providing a single balanced metric that considers both false positives and false negatives.
- [Fallback Behavior](https://docs.rhesis.ai/glossary/fallback-behavior.md): How an AI system responds when it cannot understand, answer, or fulfill a user's request, providing a default or alternative response.
- [False Positive](https://docs.rhesis.ai/glossary/false-positive.md): When a test incorrectly passes or a metric incorrectly identifies something as acceptable when it should fail.
- [Garak](https://docs.rhesis.ai/glossary/garak.md): title: 'Garak - Glossary', description: 'An open-source LLM vulnerability scanner integrated into Rhesis that tests LLM applications for prompt injection, jailb...
- [Goal Achievement](https://docs.rhesis.ai/glossary/goal-achievement.md): title: 'Goal Achievement - Glossary', description: 'A metric used by Penelope to evaluate whether a multi-turn conversation accomplished its stated goal, produc...
- [Graceful Degradation](https://docs.rhesis.ai/glossary/graceful-degradation.md): The ability of an AI system to maintain partial functionality and provide useful responses when facing errors, limitations, or unexpected conditions.
- [Ground Truth](https://docs.rhesis.ai/glossary/ground-truth.md): Known correct answers, expected outputs, or validated reference data used to evaluate AI system accuracy and performance.
- [Hallucination](https://docs.rhesis.ai/glossary/hallucination.md): When an LLM generates false, fabricated, or nonsensical information presented as fact, often with high confidence.
- [Intent Understanding](https://docs.rhesis.ai/glossary/intent-understanding.md): The AI's ability to comprehend what a user wants to accomplish from their input, beyond just the literal words used.
- [Jailbreak](https://docs.rhesis.ai/glossary/jailbreak.md): title: 'Jailbreak - Glossary', description: 'An adversarial technique that attempts to bypass an LLM\'s safety constraints to produce outputs the model was trai...
- [Knowledge](https://docs.rhesis.ai/glossary/knowledge.md): title: 'Knowledge - Glossary', description: 'Domain context and source materials used to generate context-aware test scenarios for your LLM application.', }
- [Latency](https://docs.rhesis.ai/glossary/latency.md): The time delay between receiving user input and producing a response, a critical performance metric for conversational AI systems.
- [LLM as a Judge](https://docs.rhesis.ai/glossary/llm-as-a-judge.md): An approach where an LLM evaluates AI responses against defined criteria, providing automated quality assessment.
- [MCP](https://docs.rhesis.ai/glossary/mcp.md): title: 'MCP - Glossary', description: 'Model Context Protocol — in Rhesis, either importing external knowledge into Sources or connecting AI agents to Rhesis to...
- [Metric](https://docs.rhesis.ai/glossary/metric.md): title: 'Metric - Glossary', description: 'A quantifiable measurement that evaluates AI behavior using an LLM as a judge, returning pass/fail results with option...
- [Metric Scope](https://docs.rhesis.ai/glossary/metric-scope.md): title: 'Metric Scope - Glossary', description: 'The test types a metric can evaluate (), which must align with each test set\'s type or scores are silently drop...
- [Model](https://docs.rhesis.ai/glossary/model.md): title: 'Model - Glossary', description: 'An AI model configuration used for test generation, evaluation, or as a judge in metric assessments.', }
- [Multi-Turn Test](https://docs.rhesis.ai/glossary/multi-turn-test.md): title: 'Multi-Turn Test - Glossary', description: 'Goal-based conversation tests that evaluate your AI system across multiple turns, powered by Penelope.', }
- [Numeric Scoring](https://docs.rhesis.ai/glossary/numeric-scoring.md): title: 'Numeric Scoring - Glossary', description: 'A metric scoring type that uses a numeric scale (e.g., 0-10) with a defined pass/fail threshold.', }
- [OpenTelemetry](https://docs.rhesis.ai/glossary/opentelemetry.md): title: 'OpenTelemetry - Glossary', description: 'An open observability standard for collecting and exporting traces, metrics, and logs, used by the Rhesis SDK t...
- [Organization](https://docs.rhesis.ai/glossary/organization.md): title: 'Organization - Glossary', description: 'The top-level organizational unit that provides data isolation and manages team access through invitations.', }
- [Out-of-Scope Query](https://docs.rhesis.ai/glossary/out-of-scope-query.md): User requests that fall outside the AI system's designed capabilities, knowledge domain, or intended purpose.
- [Overfitting](https://docs.rhesis.ai/glossary/overfitting.md): When a system performs well on test data but poorly on new, unseen inputs due to overly specific tuning or memorization.
- [Pass/Fail Threshold](https://docs.rhesis.ai/glossary/pass-fail-threshold.md): title: 'Pass/Fail Threshold - Glossary', description: 'The minimum score required for a test to be considered passing, defined in the metric configuration.', }
- [Penelope](https://docs.rhesis.ai/glossary/penelope.md): title: 'Penelope - Glossary', description: 'An autonomous testing agent that powers multi-turn tests, adapting its strategy based on AI responses to evaluate co...
- [Playground](https://docs.rhesis.ai/glossary/playground.md): title: 'Playground - Glossary', description: 'An interactive, real-time interface for testing conversational endpoints and converting live conversations directl...
- [Polyphemus](https://docs.rhesis.ai/glossary/polyphemus.md): title: 'Polyphemus - Glossary', description: 'The Rhesis-hosted LLM service that provides access to open-source models with built-in access control, rate limiti...
- [Precision and Recall](https://docs.rhesis.ai/glossary/precision-and-recall.md): Complementary metrics measuring accuracy: precision shows how many positive predictions were correct, while recall shows how many actual positives were found.
- [Project](https://docs.rhesis.ai/glossary/project.md): title: 'Project - Glossary', description: 'The top-level organizational unit that groups related endpoints, tests, test sets, and results together for a specifi...
- [Prompt Injection](https://docs.rhesis.ai/glossary/prompt-injection.md): Malicious inputs that attempt to override system instructions, extract sensitive information, or manipulate AI behavior in unintended ways.
- [Reasoning Instructions](https://docs.rhesis.ai/glossary/reasoning-instructions.md): title: 'Reasoning Instructions - Glossary', description: 'Guidance provided to the judge model explaining how to reason about the evaluation and weight differen...
- [Regression Testing](https://docs.rhesis.ai/glossary/regression-testing.md): Testing to ensure that new changes, updates, or deployments don't break existing functionality or degrade AI performance.
- [Rescoring](https://docs.rhesis.ai/glossary/rescoring.md): title: 'Rescoring - Glossary', description: 'Re-evaluating stored test outputs against metrics without re-invoking the endpoint, enabling cost-efficient experim...
- [Retrieval-Augmented Generation (RAG)](https://docs.rhesis.ai/glossary/retrieval-augmented-generation.md): An LLM approach that retrieves relevant information from a knowledge base before generating responses, grounding outputs in specific source documents.
- [Score Configuration](https://docs.rhesis.ai/glossary/score-configuration.md): title: 'Score Configuration - Glossary', description: 'Settings that define how metrics score responses, including numeric scales or categorical classifications...
- [SDK](https://docs.rhesis.ai/glossary/sdk.md): title: 'SDK - Glossary', description: 'Software Development Kit - A Python library that provides programmatic access to Rhesis platform features for integration...
- [Single-Turn Test](https://docs.rhesis.ai/glossary/single-turn-test.md): title: 'Single-Turn Test - Glossary', description: 'A test type that checks how the AI responds to a single prompt with no follow-up conversation.', }
- [Smoke Testing](https://docs.rhesis.ai/glossary/smoke-testing.md): Quick, high-level validation tests that check basic functionality and critical features to determine if a system is stable enough for more detailed testing.
- [Source](https://docs.rhesis.ai/glossary/source.md): title: 'Source - Glossary', description: 'A document or knowledge artifact uploaded to a Rhesis project that provides context for test generation or grounding A...
- [Synthesizer](https://docs.rhesis.ai/glossary/synthesizer.md): title: 'Synthesizer - Glossary', description: 'An SDK component that automatically generates test cases from prompts, documents, or existing tests using LLM-pow...
- [Tag](https://docs.rhesis.ai/glossary/tag.md): An organization-scoped label attached to behaviors or metrics for filtering and PRD traceability — not the same as test category or topic.
- [Task](https://docs.rhesis.ai/glossary/task.md): title: 'Task - Glossary', description: 'A work item used to track testing activities, issues, or improvements within the platform.', }
- [Temperature](https://docs.rhesis.ai/glossary/temperature.md): A model parameter controlling randomness and creativity in LLM outputs, with lower values producing more deterministic responses and higher values producing mor...
- [Test](https://docs.rhesis.ai/glossary/test.md): title: 'Test - Glossary', description: 'An individual test case that represents a prompt or input sent to your LLM application, including metadata about behavio...
- [Test Generation](https://docs.rhesis.ai/glossary/test-generation.md): title: 'Test Generation - Glossary', description: 'The process of automatically creating test cases using AI, based on prompts, configurations, and source mater...
- [Test Result](https://docs.rhesis.ai/glossary/test-result.md): title: 'Test Result - Glossary', description: 'The outcome for one test in one test run — status, metric scores, and optional response text.', }
- [Test Run](https://docs.rhesis.ai/glossary/test-run.md): title: 'Test Run - Glossary', description: 'A snapshot capturing the complete result of executing a test set against an endpoint, including individual test resu...
- [Test Set](https://docs.rhesis.ai/glossary/test-set.md): title: 'Test Set - Glossary', description: 'A collection of tests that can be executed together against an endpoint, similar to test suites in traditional softw...
- [Test Set Type](https://docs.rhesis.ai/glossary/test-set-type.md): title: 'Test Set Type - Glossary', description: 'A required classification for a test set that determines which tests it can contain and how execution is handle...
- [Test Type](https://docs.rhesis.ai/glossary/test-type.md): title: 'Test Type - Glossary', description: 'The classification of an individual test as either single-turn or multi-turn, determining how it is executed and wh...
- [Topic](https://docs.rhesis.ai/glossary/topic.md): title: 'Topic - Glossary', description: 'A specific subject matter classification for tests, such as healthcare or financial advice, used for organization and a...
- [Trace](https://docs.rhesis.ai/glossary/trace.md): title: 'Trace - Glossary', description: 'A record of an LLM application\'s execution path that captures inputs, outputs, intermediate steps, and timing data for...
- [Turn-Taking](https://docs.rhesis.ai/glossary/turn-taking.md): The exchange pattern in conversations where the user and AI alternate speaking, fundamental to multi-turn dialogue evaluation.
- [Utterance](https://docs.rhesis.ai/glossary/utterance.md): A single unit of communication in a conversation, either a user's input or the AI's response in a dialogue exchange.

### Changelog

- [Rhesis Changelog](https://docs.rhesis.ai/changelog.md): All notable changes to the Rhesis project are documented in this file.

## Resources

- Full content (all pages concatenated): https://docs.rhesis.ai/llms-full.txt
- SDK API reference: https://rhesis-sdk.readthedocs.io/en/latest/
- GitHub: https://github.com/rhesis-ai/rhesis
- Discord: https://discord.rhesis.ai
- Website: https://www.rhesis.ai