Consensus check

A single model gives you a confident answer. What it cannot tell you is whether other models, trained differently, would have said the same thing.

POST /v1/consensus asks three models from three different labs the same question independently, then reports whether they agreed and where they split — handing back every answer in full.

curl https://api.gopura.net/v1/consensus \
  -H "Authorization: Bearer $GOPURA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "Our Postgres primary sits at 80% CPU and replicas lag 30-90s. Shard, move to distributed SQL, or optimise?"}]}'

{
  "object": "consensus.report",
  "agreed": true,
  "shared": [
    "Optimise the current setup rather than adopting sharding now",
    "80% CPU and replication lag point at queries and configuration, not a scaling wall",
    "Sharding would add operational complexity out of proportion to the team size"
  ],
  "split_on": [],
  "answers": [
    { "model": "anthropic/claude-sonnet-5", "answer": "…" },
    { "model": "openai/gpt-5.4", "answer": "…" },
    { "model": "google/gemini-3.1-pro-preview", "answer": "…" }
  ],
  "usage": { "cost": 0.0772, "steps": [ … ] }
}

What it is for

Use it when being wrong is expensive and you will have to defend the call: an architecture decision, a security response, a vendor commitment. Three independent models reaching the same conclusion is evidence a single confident answer cannot give you. Three models splitting is a warning worth having before you act, not after.

Don't use it for everyday questions. It costs roughly four times one model and takes around 30 seconds.

Independence is the product

The panel is one model per lab. Two models from the same house share training data, post-training and failure modes — they agree with each other for reasons that have nothing to do with being right, and that agreement is not evidence.

Each model sees your question exactly as you wrote it. We add nothing: the moment a model is told it is one of several being compared, you are measuring something other than what it independently thinks. The agreement analysis that follows sees the answers labelled "Expert A/B/C" and never learns which model wrote which, because a reader who knows the brands weighs the brands.

Nothing is merged. Every answer comes back whole. We learned this the hard way — see below.

Options

If a model fails or times out, the report continues with the rest and tells you in panel_errors. Below two answers there is nothing to compare, and you are billed only for what was consumed. If the agreement analysis itself fails, agreed comes back null and the answers are reported unjudged — we would rather say we do not know than guess.

From any OpenAI SDK

gopura/consensus works as a model id. The lead model's answer is the assistant message, in full; the others ride alongside as consensus.

r = client.chat.completions.create(
    model="gopura/consensus",
    messages=[{"role": "user", "content": "Is Kubernetes worth it for 3 services and one ops engineer?"}],
)
print(r.choices[0].message.content)   # the lead model's answer
print(r.consensus["agreed"])          # did the others independently concur?
print(r.consensus["split_on"])        # and where didn't they

The model field reports whichever model actually wrote the message, never the virtual id. This surface does not stream: several models answer before there is anything to compare.

What this replaced, and why

This endpoint first shipped as a council — the panel cross-examined each other anonymously and an impartial arbiter wrote a verdict. We evaluated it and it lost.

On 48 questions with computed answers, majority voting beat it. On 12 open-ended judgement questions scored by blind pairwise comparison, a single model was preferred 5–1, and half the councils never finished at all: 139 seconds and an upstream timeout, against 22 seconds for one model.

The cause was structural. Experts were told to be brief and the arbiter compressed three brief answers into one — council answers averaged 1,475 characters against a single model's 3,449, and the judges' stated reasons were all about substance the shorter answer lacked. Every stage shed the detail a decision-maker is paying for.

So we stopped selling the answer and started selling the comparison. This version makes a claim we can always back — these models independently said this, and here is where they diverged — rather than one we tested twice and could not demonstrate. The full method and results are in the repository under reference/research/council-eval/.

POST /v1/council and gopura/council still work, and now do this.