Integrations
Rhesis plugs into your LLM stack across four layers, each addressing a different concern. This page is the single map: pick the layer you care about and follow the link into the matching guide.
| Layer | What it covers | Where to start |
|---|---|---|
| LLM providers | The model that runs your test generation and LLM-as-Judge evaluation. | Models |
| Tracing | Streaming OpenTelemetry spans from your application to Rhesis. | Tracing · Auto-instrumentation |
| Test execution | Letting Rhesis invoke entry points in your application remotely to run test cases. | Connector |
| REST API | Programmatic access to test sets, runs, and platform resources. | api.rhesis.ai/docs |
Beyond those four layers, Rhesis also bundles evaluation frameworks (DeepEval, Ragas, Garak) and tool connections (Notion, GitHub, Jira, and more) as additional sources of metrics, tests, and knowledge context.
Tracing your application
Your application emits OpenTelemetry spans through the Rhesis SDK. How you attach them depends on the framework:
- LangChain, LangGraph, Microsoft Agent Framework, and Pydantic AI auto-instrument with a single
auto_instrument()call. - Other Python frameworks (CrewAI, OpenAI Agents SDK, LlamaIndex, …) use the
@observe.*decorators on the functions, tools, or agents you want to capture. - Any OpenTelemetry-compatible exporter can target the Rhesis ingestion endpoint directly.
See: Tracing overview · Auto-instrumentation · Decorators · Multi-agent tracing
Rhesis provides observability via OpenTelemetry-based tracing. Auto-instrumentation works out of the box for LangChain, LangGraph, and Microsoft Agent Framework; connect any Python app using the Connector with the @endpoint decorator.
Test execution: the connector
To run test cases against your application, Rhesis needs to call your code. Register an entry point with the @endpoint decorator; the SDK holds a persistent outbound WebSocket to Rhesis, so test runs invoke your function locally without exposing a public URL. The same call path serves single-turn test cases and multi-turn conversations.
See: Connector · Parameter binding · Connector examples
LLM providers
Choose any provider for the LLMs that drive test synthesis and LLM-as-Judge evaluation. Provider routing is powered by LiteLLM , giving you a single interface to 100+ models — cloud (OpenAI, Anthropic, Google Gemini, Mistral, Cohere, Groq, Together AI) or local/self-hosted (Ollama, vLLM, LiteLLM proxy).
See: Models · API tokens
Supported LLM backends you can configure under Models. For API access, see API Tokens.
Evaluation frameworks
In addition to Rhesis-native metrics, you can use metrics from DeepEval and Ragas, and import Garak probes as test sets for adversarial scanning.
See: DeepEval · Ragas · Import from Garak · Metrics
Use DeepEval and Ragas metrics in test suites, or import Garak probes as test sets.
Tools
Pull knowledge into Rhesis from external services like Notion or Github, or create a Jira issue directly from a Rhesis task.
See: Tools
REST API
Direct API access for custom integrations and CI/CD pipelines: manage test sets, trigger test runs, fetch results, and inspect traces programmatically. Language-agnostic — call from Python, TypeScript, Go, shell scripts, or anywhere else.
See: OpenAPI spec · API tokens
Tip: New to Rhesis? Start with Getting started.