← use cases · evaluation
Running a multi-engine bake-off
Give the same task to Claude Code, Aider, and Codex at once and compare the branches side by side.
who it's for
Teams deciding which coding agent to standardise on, or which engine is strongest for a given class of task.
the problem
You cannot fairly compare coding agents by running them on different days against different code. You want the identical task, the identical starting commit, three engines, three branches — and a record of what each produced.
how brat does it
- 1
Define a convoy with three legs, each pinned to a different engine in .brat/config.toml (claude, aider, codex).
- 2
Witness spawns all three sessions in parallel against the same base commit, each in an isolated actor directory.
- 3
Every prompt, session, and result is an event in refs/grite/wal, so the run is fully reproducible and auditable.
- 4
Compare the three resulting branches by hand, or add a synthesis step that asks a fourth agent to pick the best.
.brat/workflows/bakeoff.yaml
type: convoy
name: engine-bakeoff
legs:
- engine: claude
task: "implement the feature in FEATURE.md"
- engine: aider
task: "implement the feature in FEATURE.md"
- engine: codex
task: "implement the feature in FEATURE.md"
the outcome
An apples-to-apples engine comparison on real code, with a durable event log you can replay instead of a screenshot of three terminal panes.