A tool that fails with "error" teaches the model nothing and usually produces a blind retry. Build a tool with a deliberate error taxonomy, and decide for each class whether the model should ever see it.
What to build
- Build a tool that hits a flaky external API.
- Retry transient failures inside the tool with backoff — the model should never see these.
- For a malformed argument, return an error result that names the field and the expected format.
- For a genuine not-found, return an explicit empty-with-reason, distinct from a failure.
- For an authorisation failure, return a permanent error the model should not retry.
- Confirm your results use the tool result error flag correctly.
- Observe the model's next action for each class.
Done when
- A malformed argument produces a corrected retry, not the same call again.
- Transient failures are invisible to the model.
- "No results" and "lookup failed" are distinguishable in the payload.
- You are not inventing error fields the protocol does not define — the standard flag plus your own structured payload.
The trap this exercise teaches
Inventing retry-hint fields you half-remember from a study guide. Signal retryability inside your own result payload; the protocol itself only marks a result as an error.
Before you start
The theory behind this build is covered in Tool errors and trust boundaries. If any step below is unfamiliar, read that first — the exercise assumes it. Primary source: Claude Docs — Tool use.
Check yourself against the exam
This exercise sits in Domain 4, which is 18% of the CCAR‑F exam. Once you have built it, run a domain drill in the study app and see whether the questions read differently.