Test Generation

Generate test sets automatically using AI-powered templates or custom configurations.

Getting Started

Generate test sets on the Generation page . Choose your test type: multi-turn or single-turn Test Sets. See Tests for details on the difference.

Select Test Types Generation

To generate a test set, you can either:

Provide your custom configuration (such as behaviors, topics, and categories)
Select a predefined templates (skip to Step 2)

Step 1: Initial Setup

Test Generation Configuration Page

Provide the following information to configure your test generation:

Project (optional): Associate with a project
Sources (optional): Add documents for context
Description: What you want to test

Step 2: Review & Refine

Test Generation Samples Page

Based on the test description provided or the selected template, the system generates configuration (left side) and test samples (right side):

The configurations are divided into three categories:

Behaviors: The type of behavior the target system should exhibit when responding to the prompt
Topics: The specific subject matter or theme for the test
Categories: High-level classification for grouping tests

Use “Further refine test generation” to modify the configuration chips, which automatically generates new test samples based on your changes. Activate “Live Responses” to test samples against your endpoint in real-time.

Step 3: Generate

Name (optional): Add descriptive name for your test set to be generated
Size: Choose test set size
- Small (25-50): Quick validation
- Medium (75-150): Standard testing (Recommended)
- Large (200+): Production-scale

Generation takes 2-5 minutes. You’ll receive a notification when ready.

Next Steps - View and manage your tests in Tests - Organize tests into Test Sets for execution - Configure Endpoints to run tests against