Guard — policy for chatbots

Shipping a chatbot on a public page means answering an awkward question: what stops a visitor from using it as a free coding assistant, talking it into something embarrassing, or coaxing out the system prompt?

Gopura Guard is a policy that lives on the API key. Every request that key makes is enforced against it — the ring-fence holds even if the key leaks or the frontend is compromised, because the policy was never in the request in the first place.

What it does

{
  "version": 1,
  "scope": "questions about Acme's products, orders, shipping and returns",
  "deny": ["code", "medical"],
  "systemPrompt": "You are Acme's support assistant. Acme sells outdoor furniture.",
  "refusal": "I can only help with Acme orders — let me pass you to a colleague.",
  "escalate": { "enabled": true },
  "maxOutputTokens": 500
}

Set that on a key in the admin console and every /v1/chat/completions call made with it is governed. There is no client-side change and no new endpoint — your existing OpenAI SDK code keeps working.

A blocked turn comes back as a normal 200 so a chat widget keeps rendering:

{
  "choices": [{
    "message": { "role": "assistant", "content": "I can only help with Acme orders — let me pass you to a colleague." },
    "finish_reason": "content_filter"
  }],
  "gopura_guard": { "action": "blocked", "rule": "deny.code", "escalate": true },
  "usage": { "cost": 0.00012 }
}

escalate: true is your cue to route the conversation to a human. Set guard.onBlock: "error" instead if you would rather have a 403.

How it is enforced

Three layers, cheapest first.

Static rules, free. maxOutputTokens clamps the caller's max_tokens, and systemPrompt is prepended server-side where the caller cannot strip or replace it.

A guard classifier, ~$0.12 per 1,000 turns. When scope is set, one cheap call judges the incoming turn against your ring-fence before your model is ever dispatched — so a blocked turn costs a fraction of a cent instead of a full completion. It runs only when scope is set, because that is the only part of a policy that needs semantic judgement; a deny: ["code"] policy is handled by the next layer for free.

The classifier is not deterministic, even at temperature zero. In our own measurements roughly one in twelve clearly in-scope questions drew a block on some runs — so a block is always confirmed by a second, independent judgement, and only stands if both agree. Turning away a real customer is the expensive mistake, and forbidden output is caught deterministically no matter what the classifier says. Ordinary traffic never pays for the second opinion; only a turn that was going to be blocked does, which is why a blocked turn costs about $0.24 per 1,000 rather than $0.12.

Measured on a live ring-fenced assistant after that change: 15 of 15 in-scope product, shipping and returns questions answered, and 8 of 8 attacks — code requests, off-topic freeloading, instruction override and a persona jailbreak — blocked.

A deterministic output scan, free. Fenced code blocks, unmistakable source syntax, verbatim recital of your system prompt, and any regex you supply in bannedPatterns. This layer is the one we actually stand behind: a model can be argued with, a regex over what the model produced cannot. On a streamed response the scan runs as tokens arrive and aborts mid-flight.

What we do not claim

We do not claim immunity to prompt injection, and you should be sceptical of anyone who does. Published research demonstrates evasion of detectors in this class. What Gopura Guard gives you is layered enforcement where the strongest layer is deterministic, a policy an attacker cannot reach, an audit trail of every blocked turn, and a red-team corpus we run in CI so regressions surface before you meet them.

Two limits worth stating plainly. On a streamed response in the default scan mode, tokens already sent cannot be recalled — the scanner fires on the opening code fence, before any code follows it, but if you need zero leakage set guard.stream: "buffer" and trade time-to-first-token for it. And when the output scan catches an answer, the upstream call already happened, so it is billed; what changes is that the caller receives your refusal instead.

Tuning a new policy

Start in monitor mode. Violations are recorded and reported in gopura_guard with action: "flagged", but the answer still ships — so you can watch real traffic for a day before anything gets blocked.

{ "version": 1, "scope": "…", "guard": { "mode": "monitor" } }

Switch to "block" when the flagged turns look right.

Failure behaviour

If the guard classifier is unreachable, the default is to block: a key that asked for a ring-fence should not start answering without one. Set guard.onError: "allow" if availability matters more to you than containment for that particular assistant.

Tightening per request

A request may carry its own policy object, and it is merged tighten-only — it can add denied categories, narrow the scope, lower the token ceiling and turn the guard up. It can never remove a denial, widen the scope, or downgrade block to monitor. That is what makes the key-level policy a floor rather than a default.

Picking a model

For a ring-fenced assistant, instruction adherence and a graceful refusal matter more than raw capability, and cost matters because volume is high. google/gemini-3.6-flash and anthropic/claude-haiku-4.5 are the usual picks; reach for a frontier model only if the subject matter genuinely needs it. Guard works with any chat model, including gopura/auto.