Skip to Content
GlossaryAdaptive Testing - Glossary

Adaptive Testing

Back to GlossaryAdvanced Testing

A testing approach that organizes tests by topic and allows test suites to evolve based on coverage analysis and observed model behavior.

Also known as: adaptive test suite

Overview

Adaptive Testing extends static test sets by organizing tests around topics and allowing test suites to grow and adapt over time. Instead of a fixed set of tests, adaptive testing lets you explore coverage gaps, add new test cases systematically, and generate outputs that feed back into the testing pipeline.

Key Concepts

Topics: Adaptive tests are organized around topics—thematic groupings of related test cases. Topics act as a navigation layer, letting you see which areas of your application are well-covered and which need more tests.

Test Explorer: The Test Explorer UI provides a browsable view of all adaptive tests and their associated topics. You can create, update, and delete tests and topics directly from the explorer.

Output Generation: Adaptive testing supports generating test outputs in bulk, using asynchronous processing for improved performance. Generated outputs can be used directly for evaluation or as inputs for further test creation.

Workflow

  1. Define topics that reflect the behavioral areas you want to cover
  2. Create tests within each topic, either manually or via test generation
  3. Generate outputs for your test cases against the target endpoint
  4. Evaluate outputs to identify gaps and failures
  5. Expand coverage by adding new tests in under-covered topics

SDK Usage

python
from rhesis.sdk import RhesisClient

client = RhesisClient()

# List all topics
topics = client.topics.list()

# Create a new adaptive test
new_test = client.tests.create(
      topic_id=topics[0].id,
      input="What happens if I provide an empty message?"
)

Difference from Static Test Sets

Traditional test sets are a fixed collection of tests executed together. Adaptive testing treats the test suite as a living artifact—one that grows and changes based on discovered behaviors, coverage analysis, and ongoing development.

Best Practices

  • Define topics at the level of user intent or behavioral category, not individual question variants
  • Review output generation results regularly to identify topics with consistently high failure rates
  • Balance coverage across topics before releasing a new model version to avoid blind spots
  • Use adaptive tests alongside static test sets to maintain both stability and ongoing discovery

Related Terms

TermsAcknowledgments