Skip to Content
GlossaryMetric Scope - Glossary

Metric Scope

Back to GlossaryTesting

The test types (single-turn or multi-turn) that a metric can evaluate, defined during metric configuration.

Also known as: scope

Overview

Metric scope defines which test types (single-turn, multi-turn, or both) a metric can evaluate. Different metrics work better for different conversation patterns.

Scope Types

Single-Turn Only: Metrics that evaluate individual responses.

Good for:

  • Factual accuracy
  • Format compliance
  • Safety checks
  • Response quality

Multi-Turn Only: Metrics that evaluate conversational behavior.

Good for:

  • Context awareness
  • Conversation flow
  • Goal completion
  • Clarification handling

Both: Metrics applicable to any test type.

Good for:

  • Tone evaluation
  • Helpfulness
  • Brand voice
  • General quality

Choosing Scope

Questions to Ask:

  1. Does evaluation need conversation history?

    • Yes → Multi-turn only
    • No → Single-turn or Both

  2. Is it about individual responses or dialogue?

    • Individual → Single-turn or Both
    • Dialogue → Multi-turn only

  3. Can it be evaluated in isolation?

    • Yes → Single-turn or Both
    • No → Multi-turn only

Examples by Scope

Single-Turn:

  • Factual accuracy
  • Safety/harm refusal
  • Format compliance
  • PII handling
  • Source citation

Multi-Turn:

  • Context retention
  • Clarification requests
  • Goal achievement
  • Conversation coherence
  • Information gathering

Both:

  • Response helpfulness
  • Tone and style
  • Professionalism
  • Clarity
  • Conciseness

Best Practices

  • Be specific: Choose narrowest applicable scope
  • Test both: If using "both", validate on each type
  • Separate concerns: Different metrics for different patterns
  • Document reasoning: Explain why scope was chosen

Documentation

Related Terms

Terms