Skip to Content
GlossaryTest Set - Glossary

Test Set

Back to GlossaryTesting

A collection of tests that can be executed together against an endpoint, similar to test suites in traditional software development.

Also known as: test suite

Overview

Test sets group related tests together for organized execution and analysis. They function like test suites in traditional software testing, allowing you to run comprehensive evaluations with a single command.

Key Benefits

Test sets help you organize testing by grouping related tests around features, behaviors, or scenarios. Rather than running tests individually, you can execute an entire set with a single command, ensuring consistent evaluation where the same tests run identically every time. This makes it easy to track performance trends across multiple runs and spot regressions Test sets integrate smoothly into CI/CD pipelines, enabling automated quality checks as part of your deployment process.

Common Test Set Patterns

By Feature:

  • Customer Support Scenarios
  • Product Recommendation Tests
  • Search Functionality Tests

By Behavior:

  • Safety and Harm Prevention
  • Accuracy and Factuality
  • Tone and Professionalism

By Purpose:

  • Smoke Tests (quick validation)
  • Regression Tests (comprehensive coverage)
  • Performance Tests (stress testing)

Example Usage

python
from rhesis import TestSet

# Create a test set
safety_tests = TestSet(
      name="Safety Evaluation",
      tests=[
          harmful_content_tests,
          bias_detection_tests,
          privacy_tests
      ]
)

# Run against an endpoint
results = safety_tests.run(endpoint="production-bot")

Documentation

Related Terms

Terms