Skip to Content
GlossaryTest - Glossary

Test

Back to GlossaryTesting

An individual test case that represents a prompt or input sent to your AI application, including metadata about behavior and expected results.

Also known as: test case

Overview

Tests are the fundamental building blocks of AI evaluation in Rhesis. Each test represents a specific scenario or input that you want to evaluate your AI system against.

Test Components

  • Prompt: The input sent to your AI system
  • Expected Behavior: What you're testing for
  • Metadata: Category, topic, and tags for organization
  • Context: Additional information for multi-turn tests

Test Types:

  • Single-Turn: One prompt, one response
  • Multi-Turn: Conversational tests with multiple exchanges

Creating Tests

Manual Creation: Create tests manually through the Rhesis web interface in the Tests section of your project.

Automated Generation with SDK: Use the SDK's synthesizers to generate tests:

python
from rhesis.sdk.synthesizers import PromptSynthesizer

synthesizer = PromptSynthesizer(
      prompt="Generate tests for a medical chatbot that provides medication information"
)
test_set = synthesizer.generate(num_tests=10)

Generate tests based on:

  • Behaviors you want to test
  • Knowledge bases or documentation
  • Real user interactions
  • Edge cases and scenarios

Best Practices

  • Be specific: Clear prompts lead to better evaluation
  • Cover edge cases: Test boundary conditions and unusual inputs
  • Use metadata: Proper categorization helps with analysis
  • Regular updates: Keep tests aligned with your AI's capabilities

Documentation

Related Terms

Terms