Orchestrator-workers is the pattern for work whose subtasks cannot be enumerated ahead of time — the orchestrator decides the breakdown per input. Evaluator-optimizer adds a critic loop. Build both, then combine them, and instrument the stopping condition, because that is where these systems actually fail.
What to build
- Orchestrator: given a feature request, have it emit a list of files to change and why.
- Workers: one call per file, each receiving only the context that file needs.
- Aggregate the worker outputs into a single coherent change summary.
- Evaluator: a separate call that scores the aggregate against a written rubric and returns concrete, actionable feedback — not a bare number.
- Optimizer: feed the feedback back to the generator. Loop.
- Cap the loop, and stop early when the evaluator reports no material improvement.
Done when
- The subtask list is produced at runtime and varies with the input.
- Workers do not receive the orchestrator's full context — only their slice.
- The evaluator's feedback is specific enough to act on mechanically.
- The loop has both an iteration cap and a convergence check, and you can show both firing.
- You can name a task where evaluator-optimizer is wrong because no clear rubric exists.
If you want to go further
- Vary the evaluator's model tier and measure whether a weaker critic still improves output.
The trap this exercise teaches
An unbounded optimizer loop, or a rubric so vague the evaluator's feedback cannot be acted on. Both burn tokens without converging.
Before you start
The theory behind this build is covered in Orchestrator-workers and evaluator-optimizer. If any step below is unfamiliar, read that first — the exercise assumes it. Primary source: Anthropic — Building effective agents.
Check yourself against the exam
This exercise sits in Domain 1, which is 27% of the CCAR‑F exam. Once you have built it, run a domain drill in the study app and see whether the questions read differently.