Skip to Content
GlossarySynthesizer - Glossary

Synthesizer

Back to GlossaryDevelopment Tools

An SDK component that automatically generates test cases from prompts, documents, or existing tests using LLM-powered analysis.

Also known as: test synthesizer, PromptSynthesizer, DocumentSynthesizer

Overview

Synthesizers are the core of Rhesis's AI-powered test generation capability. Rather than writing test cases manually, you provide a synthesizer with source material and it produces a set of test inputs designed to evaluate specific behaviors of your LLM application.

Built-In Synthesizer Types

PromptSynthesizer: Generates test cases from a text prompt or description. Useful when you want to explore variations of a particular scenario.

python
from rhesis.sdk.synthesizers import PromptSynthesizer

synth = PromptSynthesizer()
tests = synth.generate(
      prompt="Generate tests for a banking chatbot handling balance inquiries",
      count=10
)

DocumentSynthesizer: Extracts content from a document (PDF, DOCX, TXT, etc.) and generates test cases based on the document's content. Ideal for creating tests from product documentation, FAQs, or knowledge bases.

python
from rhesis.sdk.synthesizers import DocumentSynthesizer

synth = DocumentSynthesizer()
tests = synth.generate_from_file("faq.pdf", count=20)

ParaphrasingSynthesizer: Takes an existing test case and generates semantically equivalent variants with different phrasing. Useful for expanding test coverage and testing robustness to input variation.

Chain-of-Thought Generation

Synthesizers use Chain-of-Thought prompting techniques to improve the quality and diversity of generated test cases. This produces tests that are more realistic and better at uncovering edge cases than naive prompt completion.

Integration with Behaviors

Synthesizers are most effective when guided by a behavior definition. Providing a behavior ensures that generated tests target specific quality criteria and are organized for meaningful evaluation.

Custom Synthesizers

You can extend the base synthesizer class to implement custom generation strategies for specialized testing scenarios.

Best Practices

  • Guide synthesizers with a specific behavior definition to keep generated tests focused on evaluable criteria
  • Review generated tests before adding them to a test set—synthesizers can occasionally produce ambiguous inputs
  • Use ParaphrasingSynthesizer to expand coverage after an initial working test set is in place
  • Combine DocumentSynthesizer with your knowledge base sources to generate tests grounded in real product content

Related Terms

TermsAcknowledgments