Architect Agent (SDK)

The ArchitectAgent class lives in sdk/src/rhesis/sdk/agents/architect/. It drives the conversational test-suite design workflow — managing mode transitions, plan state, tool confirmation, and event emission across conversation turns.

Module layout

sdk/src/rhesis/sdk/agents/architect/
├── __init__.py          # Public exports
├── agent.py             # ArchitectAgent class
├── config.py            # ArchitectConfig dataclass
├── plan.py              # ArchitectPlan Pydantic model + spec models
├── tool_registry.py     # TOOL_REGISTRY mapping tools → modes
└── prompt_templates/
    ├── system_prompt.j2        # Core behavior, rules, workflow phases
    ├── personality.j2          # Tone and persona shaping
    ├── streaming_response.j2   # Streaming acknowledgment format
    └── iteration_prompt.j2     # Injected between ReAct loop iterations

ArchitectAgent class

ArchitectAgent extends BaseAgent with multi-turn conversation state, plan tracking, mode management, and the two-layer write guard.

Constructor parameters:

Internal state:

Public methods:

# Synchronous — runs asyncio.run() internally
response = architect.chat("Test my chatbot", attachments={"files": [...]})

# Async — preferred when running inside an existing event loop
response = await architect.chat_async("Test my chatbot", attachments={"files": [...]})

ArchitectConfig

ArchitectConfig is a frozen dataclass. Pass a custom instance to the config parameter to override any value.

from rhesis.sdk.agents.architect import ArchitectAgent, ArchitectConfig

config = ArchitectConfig(max_iterations=25, max_attachment_chars=50_000)
agent = ArchitectAgent(model="vertex_ai/gemini-2.0-flash", config=config)

ArchitectPlan model

ArchitectPlan (in plan.py) is the Pydantic model that holds the agent’s structured test suite proposal. The agent populates it by calling the internal save_plan tool during the planning phase.

Spec models

reuse_status is a Literal["reuse", "improve", "new"] on RequirementSpec and MetricSpec.

save_plan auto-generation

The build_save_plan_tool() function in plan.py generates the save_plan JSON schema automatically from ArchitectPlan’s Pydantic field definitions. Internal fields (completed, existing_id) are stripped from the tool schema — the LLM never sees them; they are populated by the agent at creation time.

This means the plan schema stays in sync with the Pydantic model automatically. If you add a new field to a spec class, build_save_plan_tool() picks it up without any manual schema update — unless the field should be hidden from the LLM, in which case add it to the exclusion list in build_save_plan_tool().

Tool registry

TOOL_REGISTRY in tool_registry.py maps tool names to their AgentMode and optional PlanCategory. The agent uses this to:

1. Determine mode transitions — when the agent calls a tool registered under CREATING, the mode switches to AgentMode.CREATING.
2. Track plan progress — when a creating tool succeeds, its PlanCategory is used to tick the corresponding checkbox in the plan.

Tools not in the registry (read-only platform tools, explore tool) do not trigger mode changes.

Prompt templates

Templates are Jinja2 files in prompt_templates/. They are rendered at the start of each turn.

When modifying behavior, change system_prompt.j2 first. The other templates are secondary.

Write guard (two-layer safety)

The write guard prevents the agent from creating or modifying platform entities without user approval.

Layer 1 — prompt: system_prompt.j2 instructs the LLM to always present a plan and ask for confirmation before calling any mutating tool.

Layer 2 — structural: agent.py intercepts tool calls at execution time. When it holds a mutating call, it records the approval scope in _approved_scope: each held call and each item of the plan as saved at that moment, as (kind, name) pairs. A call with no entity name is keyed by its tool and *_id arguments, so approving it never approves every call of that tool. On the next turn, if the user confirms, only calls inside that scope run. add_requirement_to_metric passes when both the requirement and the metric are in scope. Anything else is held again, including plan items added with save_plan during the approved turn.

The scope is part of guard_state, so it survives the backend rebuilding the agent for each message.

_auto_approve_all = True bypasses layer 2 for the session (set when the UI auto-approve toggle is on). Layer 1 (prompt) is always active.

A tool is considered mutating if its HTTP method is not in ArchitectConfig.readonly_http_methods (GET, HEAD, OPTIONS).

Failed tool calls

BaseAgent stops a turn from looping on calls that can’t succeed. It refuses, without sending, an identical call (same tool, same arguments) that already failed this turn, and any call to a tool that has failed three times in a row (_MAX_FAILURES_PER_TOOL); a success clears the count, so a tool called many times a turn can correct a few rejected calls. The refusal comes back to the model as the tool’s error and emits no events. The record resets at the start of each turn.

The Architect adds two checks of its own:

• An *_id argument that holds a placeholder rather than a real id (for example pending_id_resolution) is rejected before dispatch, telling the model to wait for the create to return its id (find_placeholder_ids in arg_validation.py).
• A failure that a later call of the same tool recovered from is left out of the tool history in the model’s prompt.

Chart guards

The Architect’s charts come from ShowChartTool (chart_tool.py), which runs a get_insights query and keeps a chart spec built from the returned rows. A few checks keep a chart honest about what the reply says. Where the right chart is clear they correct the call instead of rejecting it, and the tool result says what changed: a rejected call shows the user a failed tool row for what is only a drawing choice.

• Nudge to chart. A turn that read get_insights rows and tries to finish without calling show_chart is turned back once with CHART_NUDGE, a failed finish result asking for a chart per breakdown. The model may finish again if no chart fits. It never fires on the last iteration.
• Kept filters. Before show_chart runs, _fill_chart_filters compares its query with this turn’s latest get_insights call for the same breakdown (same entity and group_by). A filter that call set and the chart leaves out (environments, endpoint_ids, months and the like) is added to the chart query, so the chart shows the data the reply was written from (dropped_filters in chart_tool.py).
• Named environments. A trace query (get_insights or show_chart) with no environments filter gets the environments the user’s message names, matched as whole words against the project’s trace environments (_scope_to_named_environments). The names come from one get_insights grouped by environment, run without events so it shows no tool row.
• Counts only in stacks. stacked_bar and area add values up, so for any other measure they are drawn as bar and line.
• No duplicates. A chart identical to one already on the reply is not added again.
• Honest titles. A “top N” over fewer rows is dropped from the title.
• One kind per axis. Measures of different kinds, such as latency and cost, are split into one chart per kind from the same query, titled by measure.

On a turn with a chart, the reply writer (streaming_response.j2) gets a chart rule in place of its table rule: the findings in a sentence each, and none of the chart’s numbers laid out again.

How to add a new MCP tool

1. Define the tool in mcp_tools.yaml (backend):

- name: my_new_tool
description: "What this tool does."
method: POST
path: /my-resource/
requires_confirmation: true   # set true for mutating tools

2. Register the tool in TOOL_REGISTRY if it should trigger a mode change or track plan progress:

# tool_registry.py
TOOL_REGISTRY["my_new_tool"] = ToolEntry(
    mode=AgentMode.CREATING,
    plan_category=PlanCategory.REQUIREMENT,  # or None
)

3. Update the system prompt (system_prompt.j2) if the tool requires specific usage guidance — when to call it, what arguments to pass, and how to interpret results.

4. Test with the playground scripts in playground/telemachus/ — architect_e2e.py and tool_call_chain.py are good starting points for exercising new tool integrations.

Related pages

• Backend — session API, WebSocket, MCP
• Frontend — chat UI, hook, streaming
• Worker — Celery task, async resume
• User guide — Architect overview