When should Codex use multiple agents? A benchmark, not a slogan

Both the single-agent and orchestrated conditions start from the same commit,
task text, supplied tests, and frozen contract. The orchestrated condition may
use at most two implementation workers. The controller owns evaluation and the
final receipt.

Both conditions face the same external evaluator. It checks fixture integrity,
candidate scope, concurrent persistence, strict atomic writes, validation and
error mapping, live HTTP behavior, path traversal, static browser requirements,
and required handoff topics.

What the first run actually showed

The first smoke run qualified the benchmark harness:

The orchestrated candidate returned live browser evidence covering creation,
filtering, state transitions, validation feedback, console errors, and a mobile
viewport. The single agent found and corrected a conflict-message behavior in
final review. An external controller later exercised its live HTTP behavior.

That result establishes feasibility, not superiority. The runs overlapped, the
environment did not expose aggregate controller-plus-worker tokens, and there
was only one candidate per method. It would be invalid to claim that
orchestration was faster, cheaper, or generally more reliable.

Read the complete
smoke receipt
before interpreting the table.

A controller contract that limits fan-out

A large goal should not become permission to create an agent for every noun in
the prompt. Freeze the contract and state the maximum useful topology:

Implement the incident-response task end to end. Before delegation: - freeze the HTTP and data contract; - confirm that incident/** and web/** are independent write surfaces; - keep evaluator and handoff ownership with the controller. Delegation budget: - at most two implementation workers; - backend owns incident/** only; - frontend owns web/** only; - workers must not modify TASK.md, tests, evaluator files, or each other's paths. Each worker returns: - files changed; - focused checks and outcomes; - observable behavior exercised; - failures, retries, and anything unverified. The controller then runs supplied and external checks, exercises integration, reviews the complete diff, fixes verified findings, and writes one receipt. 

This contract does not force delegation. If the controller discovers that the
interface is unstable or the surfaces are coupled, the correct choice is a
single agent or sequential pipeline.

Measure quality before speed

For each candidate, record raw facts before interpreting them:

Do not substitute controller-only telemetry for total tokens. Do not remove
failed candidates. Do not compare overlapping runs as if they were isolated.
Write unavailable when the execution surface does not expose a metric.

Use one
orchestration receipt
per candidate. Run at least three fresh sequential pairs and alternate which
method runs first. Compare acceptance and evidence completeness before elapsed
time or tokens.

Where orchestration usually loses

Prefer one agent or a sequential workflow when:

• one clear agent can hold the relevant context;
• workers would edit the same files;
• one subtask depends on another’s unresolved design;
• the task is primarily a single debugging chain;
• the evaluator cannot attribute failures to a candidate;
• per-agent cost is invisible and cost is the decision criterion; or
• the only justification is that parallelism is available.

For small bounded fixes, workflow guidance itself may cost more context than it
saves. Codex How To’s earlier measurements found that the no-skill control was
the cheapest successful variant on a small backend defect, while a lean
engineering loop was cheapest on a medium browser-game task. Task class matters.

Where orchestration may earn its cost

Good candidates include:

• independent modules with frozen interfaces;
• read-only review through independent security, reliability, and test lenses;
• noisy investigation that can be summarized before implementation;
• frontend and backend implementation after the API contract is fixed; and
• tasks where one specialist can add independently verifiable runtime evidence.

The benefit may be coverage rather than speed. If a specialist catches a real
defect or produces missing browser, deployment, or security evidence, higher
token use may still be rational. That is a quality decision, not a token-saving
claim.

Replicate or falsify it

The next useful result is not another success screenshot. It is a controlled
pair that changes the recommendation:

1. Create identical disposable starting copies.
2. Keep model, reasoning effort, permissions, tools, task, time limit, and evaluator fixed.
3. Run one single-agent and one bounded-orchestration candidate sequentially.
4. Alternate order across at least three pairs.
5. Preserve failures and total controller-plus-worker telemetry.
6. Publish sanitized receipts and limitations.

Submit a result through the
replication issue or add a
tested benchmark edition through a focused pull request. Negative and neutral
results are explicitly useful.

Codex How To is an independent community project. Official OpenAI documentation
remains authoritative for current product behavior.