Most tool-selection failures are description failures. Start from a deliberately vague tool set, watch the model misuse it, and fix it by rewriting the interface rather than by patching the system prompt.
What to build
- Define three overlapping tools with terse names and one-line descriptions — search, find, lookup.
- Run twenty varied requests and log which tool the model picks.
- Record every wrong selection and why it was plausible.
- Rewrite each description to state purpose, when to use it, and when not to.
- Replace freeform string parameters with enums wherever the value set is closed.
- Add units and formats to parameter descriptions.
- Re-run the same twenty requests and compare.
Done when
- Selection accuracy improved measurably against the same request set.
- No two tools could reasonably be confused from their descriptions alone.
- Closed value sets are enums, so invalid values are impossible rather than discouraged.
- You fixed this in the tool definitions, not in the system prompt.
If you want to go further
- Add a fourth tool that genuinely overlaps and write a disambiguating line into both descriptions.
The trap this exercise teaches
Fixing tool confusion by adding rules to the system prompt. That scales badly and leaves the interface ambiguous for every other caller.
Before you start
The theory behind this build is covered in Tool descriptions are the primary lever. If any step below is unfamiliar, read that first — the exercise assumes it. Primary source: Claude Docs — Tool use.
Check yourself against the exam
This exercise sits in Domain 4, which is 18% of the CCAR‑F exam. Once you have built it, run a domain drill in the study app and see whether the questions read differently.