Judge panel

Goal enforcement checks that the loop keeps moving; it says nothing about whether the final answer is actually right. A judge panel is an independent review of that answer, run after the turn produces it - one or more reviewer personas, each with real tool access, check the output and can send it back for revision before it reaches you.

text

answer -> [judge 1] -> [judge 2] -> [judge 3] -> all approved? -> answer
              |             |             |
              v (tools)     v (tools)     v (tools)
          read files    run tests    search the web
                    any rejected -> revise -> re-judge

A judge is a full agent, not a rubber stamp. Each judge runs its own ephemeral tool-calling loop (the same round loop the main turn uses), so it can read a file, run a test, or search the web to verify a claim before ruling - not just re-read the answer and guess. A judge settles its review by calling judge_verdict{approved, feedback} exactly once; there is no way to approve or reject by prose alone.

The panel, not one judge. Every judge in the roster reviews the same output, concurrently. If any judge rejects it, the rejecting judges' feedback is combined into a revision directive, the loop produces a new answer, and the whole panel reviews it again - up to JudgeMaxIterations times (default 2). Once the budget is spent the last attempted answer is returned regardless - a judge panel that never approves must not be able to block the turn forever.

You can see it happening. Because a judge is a full tool-calling loop, a review can take as long as a real turn - the REPL prints [judge] status lines so you are never left staring at a silent prompt:

[judge] reviewing output - correctness, security
[judge] approved by correctness, security

or, on rejection:

[judge] reviewing output - correctness, security
[judge] revision requested (1/2): security: uses a shell command with unescaped input
[judge] reviewing output - correctness, security
[judge] approved by correctness, security

Configuring the roster.

cfg := agent.Config{
    // Explicit roster - always wins when set.
    Judges: []agent.JudgeSpec{
        {Name: "correctness", Criteria: "every factual claim must be verifiable from the codebase"},
        {Name: "security", Criteria: "no secrets, credentials, or unsafe shell input"},
    },
    // Or: generate a roster (1-3 judges) per turn instead of hand-configuring one.
    AutoJudges: true,
    JudgeMaxIterations: 2, // revise-and-rejudge budget
}

A JudgeSpec.Tools list scopes which registered tools that judge may call (empty means every tool available to the main loop). With AutoJudges, the roster is generated by one LLM call per turn - the same synthetic-workflow JSON-mode pattern goal decomposition uses - and skipped for trivial chat the same way goal decomposition is. Any failure anywhere in the panel (a judge that errors, times out, or never calls judge_verdict) degrades to approval rather than blocking the turn.

On by default in the interactive REPL. AutoJudges is enabled automatically when you start the REPL, the same way GoalEnforcement is - every turn gets an auto-generated review panel with no configuration needed. Library and test callers get no panel unless they set Config.Judges or Config.AutoJudges explicitly. /judges auto on|off persists across sessions the same way /tools full|lean does; a hand-configured roster (/judges add/remove) is session-only.

From the REPL, the same roster is managed with /judges - no restart needed:

/judges                                        # show the current roster / auto-judges state
/judges auto off                               # turn off the default auto-generated panel
/judges add correctness every claim must cite a source   # or: hand-configure an explicit roster instead
/judges remove correctness                     # drop it again
/judges clear                                  # disable the panel entirely

Judge panel ​

See also ​

Judge panel

See also