An answer without provenance cannot be audited, and hybrid retrieval exists because neither semantic nor keyword search alone is sufficient. Build the pipeline end to end and add the check that catches ungrounded claims.
What to build
- Chunk a corpus with deliberate overlap and preserve section metadata on each chunk.
- Build a semantic index and a keyword index over the same chunks.
- Query both and fuse the rankings.
- Generate an answer that must cite chunk identifiers for each claim.
- Add a grounding check that flags any sentence with no supporting citation.
- Test with a question the corpus cannot answer and confirm the system says so.
Done when
- Hybrid retrieval beats either index alone on your own query set.
- Every factual sentence carries a citation resolvable to a chunk.
- An unanswerable question yields an explicit "not in the corpus", not an invention.
- Chunk metadata is sufficient to show a user where an answer came from.
If you want to go further
- Add publication_date to chunk metadata and handle two sources that disagree because one is simply older.
The trap this exercise teaches
Semantic-only retrieval. It misses exact identifiers, error codes and product names — precisely the queries where users expect an exact hit.
Before you start
The theory behind this build is covered in Production RAG: multi-index, citation and grounding. If any step below is unfamiliar, read that first — the exercise assumes it. Primary source: Claude Docs — Citations.
Check yourself against the exam
This exercise sits in Domain 5, which is 15% of the CCAR‑F exam. Once you have built it, run a domain drill in the study app and see whether the questions read differently.