Skip to Content
SDKStatistics

Statistics

The SDK provides built-in methods for querying aggregated statistics across test runs and test results. Use these to build dashboards, generate reports, or analyze trends without writing manual API calls.

Two collection-level methods are available:

  • TestRuns.stats() — aggregated test run statistics (status distribution, result breakdown, timeline, top executors)
  • TestResults.stats() — aggregated test result statistics (pass rates by metric, behavior, category, topic, timeline)

Test Run Statistics

Use TestRuns.stats() to get aggregated data across all test runs in your organization.

Available Modes

ModeDescription
ALLComplete dataset with all sections
SUMMARYOverall summary and metadata only
STATUSStatus distribution (Completed, Partial, Failed, etc.)
RESULTSTest result pass/fail/pending breakdown
TEST_SETSMost frequently run test sets
EXECUTORSTop test executors
TIMELINEMonthly trend data

Usage

test_run_stats.py
from rhesis.sdk.entities import TestRuns, TestRun
from rhesis.sdk.entities.stats import TestRunStatsMode

# Get all stats for the last 6 months
stats = TestRuns.stats(mode=TestRunStatsMode.ALL, months=6, top=5)

# Access typed results
print(f"Total runs: {stats.metadata.total_test_runs}")

for item in stats.status_distribution:
    print(f"  {item.status}: {item.count} ({item.percentage}%)")

print(f"Pass rate: {stats.result_distribution.pass_rate}%")

# Stats for a single test run
run = TestRun(id="your-run-id")
run_stats = run.stats()

Test Result Statistics

Use TestResults.stats() to get pass/fail breakdowns across test results, grouped by different dimensions.

Available Modes

ModeDescription
ALLComplete dataset with all sections
SUMMARYOverall pass rates and metadata only
METRICSPass rates per evaluation metric
BEHAVIORPass rates per behavior
CATEGORYPass rates per category
TOPICPass rates per topic
OVERALLAggregate pass/fail rates
TIMELINEMonthly trend data
TEST_RUNSPer-run summary breakdown

Usage

test_result_stats.py
from rhesis.sdk.entities import TestResults
from rhesis.sdk.entities.stats import TestResultStatsMode

# Get topic breakdown
stats = TestResults.stats(mode=TestResultStatsMode.TOPIC, months=6)

for topic, rates in stats.topic_pass_rates.items():
    print(f"  {topic}: {rates.passed}/{rates.total} ({rates.pass_rate}%)")

# Get everything at once
all_stats = TestResults.stats(mode=TestResultStatsMode.ALL)

# Access different dimensions from the same response
all_stats.topic_pass_rates
all_stats.behavior_pass_rates
all_stats.category_pass_rates
all_stats.metric_pass_rates
all_stats.overall_pass_rates
all_stats.timeline

Filtering

Both methods accept filter parameters to narrow results by time range, test sets, users, and more.

ParameterTypeDescription
monthsintMonths of historical data (default: 6)
topintMax items in ranked lists
start_datestrISO date, overrides months
end_datestrISO date, overrides months
test_set_idslistFilter by test set IDs
test_run_idslistFilter by test run IDs
user_idslistFilter by user IDs
filtered_stats.py
from rhesis.sdk.entities import TestResults
from rhesis.sdk.entities.stats import TestResultStatsMode

# Filter by test set and time range
stats = TestResults.stats(
    mode=TestResultStatsMode.ALL,
    months=3,
    test_set_ids=["test-set-id-1", "test-set-id-2"],
)

# Filter by specific topics
topic_stats = TestResults.stats(
    mode=TestResultStatsMode.TOPIC,
    topic_ids=["topic-id"],
)

Pandas Integration

Both TestRunStats and TestResultStats provide a to_dataframe() method for converting results to pandas DataFrames. This is useful for charting and further analysis in notebooks.

pandas is an optional dependency. Install it with pip install pandas if you want to use to_dataframe().

stats_dataframe.py
from rhesis.sdk.entities import TestRuns, TestResults
from rhesis.sdk.entities.stats import TestRunStatsMode, TestResultStatsMode

# Test run timeline as a DataFrame
run_stats = TestRuns.stats(mode=TestRunStatsMode.ALL, months=6)
timeline_df = run_stats.to_dataframe("timeline")
print(timeline_df)
#       date  total_runs                      result_breakdown
# 0  2026-01          10  {'passed': 8, 'failed': 2, 'pending': 0}
# 1  2026-02          15  {'passed': 12, 'failed': 3, 'pending': 0}

# Topic pass rates as a DataFrame (dict sections use the key as index)
result_stats = TestResults.stats(mode=TestResultStatsMode.TOPIC)
topic_df = result_stats.to_dataframe("topic_pass_rates")
print(topic_df)
#            total  passed  failed  pass_rate
# Safety        50      40      10      80.00
# Accuracy      30      25       5      83.33

Next Steps

Test ExecutionModels