Category
Back to GlossaryTesting
A high-level classification for tests, such as Harmful or Harmless, used to organize and filter test cases.
Overview
Categories provide high-level organization for your tests, making it easy to filter, analyze, and run specific groups of tests.
Common Categories
Safety-Based:
- Harmless: Safe, appropriate content
- Harmful: Content requiring refusal
- Edge Cases: Boundary scenarios
Domain-Based:
- Medical: Healthcare-related queries
- Financial: Money and finance topics
- Legal: Legal information requests
- General: Everyday questions
Capability-Based:
- Knowledge: Factual information
- Reasoning: Problem-solving
- Creative: Generation tasks
- Conversational: Dialogue management
Using Categories
Assign categories when creating tests to enable organized analysis and targeted test execution. Filter test results by category to understand performance across different areas. Run specific categories during development to focus on particular aspects without executing your entire test suite. Compare performance across categories to identify strengths and weaknesses in your system's capabilities or knowledge domains.
Best Practices
- Consistent naming: Use standard category names across your organization
- Not too many: Keep categories broad; use topics for specificity
- Clear definitions: Document what belongs in each category
- Analyze separately: Track performance by category for insights