Tests

Individual test cases that evaluate specific aspects of your AI system’s performance.

Understanding Tests

Tests are individual test cases that check specific inputs and expected outputs for your AI system. Each test includes an input prompt, expected response criteria, assigned metrics for evaluation, and execution history.

Test Types

Single-Turn Tests

Single-turn tests evaluate individual interactions:

• Single prompt and response
• Immediate evaluation
• Quick validation of specific behaviors

Multi-Turn Tests

Multi-turn tests simulate conversations:

• Multiple turns of interaction
• Conversational context maintained
• Evaluation of complete workflows
• Goal achievement tracking

Creating Tests

Manual Creation

Create tests manually for specific scenarios:

1. Navigate to the Tests page
2. Click “Create Test”
3. Select test type (single-turn or multi-turn)
4. Enter test details:
   • Name: Descriptive test name
   • Input: Initial prompt or conversation starter
   • Expected Behavior: What should happen
   • Metrics: Select evaluation metrics
5. Save the test

Automated Generation

Generate tests automatically based on your requirements:

1. Go to the Generation page
2. Select generation method:
   • From behaviors
   • From knowledge sources
   • From scenarios
3. Configure generation parameters
4. Review and refine generated tests
5. Add to test sets

See Generation for detailed generation options.

Running Tests

Execute individual tests:

1. Open a test from the Tests page
2. Click “Run Test”
3. Select the endpoint to test against
4. Monitor execution progress
5. Review results

For batch execution, add tests to Test Sets.

Viewing Test Results

After execution, review:

• Actual response from your AI system
• Metric scores and evaluation details
• Pass/fail status
• Execution time and metadata
• Historical trends

Managing Tests

Editing Tests

Modify existing tests:

1. Open the test
2. Click “Edit”
3. Update fields as needed
4. Save changes

Organizing Tests

Keep tests organized:

• Add tests to test sets
• Tag tests by feature or category
• Link tests to knowledge sources
• Associate with behaviors

Deleting Tests

Remove obsolete tests:

1. Select test(s) to delete
2. Click delete action
3. Confirm deletion

Note: Tests in test sets will be removed from those sets.