Polyphemus (Development)

Polyphemus is the model-serving service used for adversarial generation workloads. It proxies generation requests to Vertex AI and exposes authenticated REST endpoints.

Runtime and deployment notes

• Runtime baseline: Python >=3.12
• Router module: apps/polyphemus/src/rhesis/polyphemus/routers/services.py
• Request schemas: apps/polyphemus/src/rhesis/polyphemus/schemas/schemas.py
• Docker image: API-only service image. PyTorch is not bundled in the Polyphemus container; model weights and serving runtime live behind Vertex AI.
• Dependency shape: Polyphemus depends on backend core only. The SDK, Penelope, Garak, and model-serving stacks live outside the runtime image.

Building the Docker image

Build the Polyphemus image from the repository root. The Dockerfile depends on monorepo paths outside apps/polyphemus, so using apps/polyphemus as the build context will fail.

# From repository root
docker build -t polyphemus:latest -f apps/polyphemus/Dockerfile .

The image uses a two-stage build: the builder installs dependencies with uv sync --no-dev, and the runtime stage copies only the API service and required backend package files. The builder copies uv and uvx from mirror.gcr.io/astral/uv:0.11.19 for reproducible and reliable Docker builds.

API endpoints

Polyphemus exposes two primary generation endpoints:

/generate_batch accepts up to 50 items per call (MAX_BATCH_SIZE).

Request fields

Both endpoints use the same GenerateRequest shape. Batch requests wrap multiple GenerateRequest objects under requests.

The service retries transient Vertex AI failures (429, 500, 502, 503, 504) up to three attempts with exponential backoff.

Environment configuration

Polyphemus reads Vertex AI target configuration from environment variables:

If required variables are missing, the service returns HTTP 400 with configuration error details.

Authentication and rate limits

Polyphemus accepts two Bearer-token formats:

API tokens are validated through the backend token store and require an active, verified user. Delegation JWTs are validated by the Polyphemus delegation-token validator. Rate limiting uses the authenticated user identity when available.

Deployment region variable mapping (v0.2.8+)

Region configuration uses two separate variables depending on context:

The workflow maps REGION → POLYPHEMUS_LOCATION automatically for service deployments. The model deployment script reads GCP_REGION independently; when running it locally you must export GCP_REGION yourself (see apps/polyphemus/model_deployment/.env.example).

vLLM logging level (v0.2.9+)

When deploying Polyphemus to Vertex AI, you can control serving container verbosity with VLLM_LOGGING_LEVEL.

export VLLM_LOGGING_LEVEL=DEBUG
python apps/polyphemus/model_deployment/deploy.py --skip-existing

If set, deployment injects VLLM_LOGGING_LEVEL into the serving container environment.

Request hardening and adversarial primer

Polyphemus enforces adversarial behavior on the server side before forwarding a request to Vertex AI. This keeps SDK, platform, and direct API callers aligned even if they assemble messages differently.

The hardening path is:

1. The service validates that at least one non-system message has content.
2. It resolves the public model alias (polyphemus-default or polyphemus-opus).
3. It injects an adversarial primer into the system message if that primer is not already present.
4. It builds the Vertex AI rawPredict request body.

Structured-output requests and direct conversation requests are handled slightly differently:

The injection is idempotent: if the primer text is already present, Polyphemus does not add it again.

Batch request and response format

{
  "requests": [
    {
      "messages": [
        {
          "role": "user",
          "content": "Summarize this policy document."
        }
      ],
      "temperature": 0.7,
      "max_tokens": 1024
    },
    {
      "messages": [
        {
          "role": "user",
          "content": "Extract key risks from this response."
        }
      ],
      "temperature": 0.2
    }
  ]
}

{
  "responses": [
    {
      "choices": [
        {
          "message": {
            "content": "..."
          }
        }
      ],
      "model": "vertex_ai/model",
      "usage": {
        "prompt_tokens": 120,
        "completion_tokens": 85
      }
    },
    {
      "error": "Generation timeout"
    }
  ]
}

Batch concurrency is capped by POLYPHEMUS_BATCH_CONCURRENCY to avoid overwhelming the shared HTTP pool or Vertex quota. The default is 10.

Related pages

• Adversarial Testing Overview
• Polyphemus product docs
• Development Changelog