Production RAG: multi-index, citation and grounding
What separates a demo pipeline from one you can put in front of users.
5 min read · Lesson 9 of 9 in this domain
A retrieval system that works on one clean corpus meets three problems in production. First, sources differ: policy documents, code and support tickets have different structures and different useful chunk sizes, which is the argument for multiple indexes routed by query type rather than one undifferentiated store. Second, users need to verify claims, so retrieved chunks must carry identifiers that survive all the way into the answer — attribution is a pipeline property, and prose summarisation destroys it. Third, the model must be constrained to the retrieved evidence, because the failure mode that erodes trust fastest is a fluent answer built from parametric knowledge that the citations appear to support but do not.
- Multiple indexes let you tune chunking and retrieval per source type, with routing by query kind.
- Carry a stable identifier on every chunk from ingestion through to the final answer — attribution is a pipeline property, not an output-formatting step.
- The citations feature returns cited text with document indexes and locations, giving verifiable spans rather than a claimed source.
- Note the constraint: citations are incompatible with
output_config.formatand the combination returns a 400. - Instruct the model to answer only from retrieved context and to say when the evidence is insufficient — an admitted gap beats a confident fabrication.
- Include
publication_dateso genuine contradiction can be distinguished from change over time. - Evaluate retrieval and generation separately: a wrong answer from correct chunks is a different bug from a right-sounding answer over wrong chunks.
Treating citation as a formatting step at the end. If the identifier was lost during synthesis, no output format restores it.
Why evaluate retrieval separately from generation?
A combined score tells you the answer was wrong but not which stage caused it. Separating them tells you whether to fix chunking and retrieval or the prompt and grounding instructions.
What happens if you combine citations with output_config.format?
This is a documented incompatibility, so a pipeline needing both verifiable citations and a strict JSON envelope has to choose or split the work across calls.
Practise this domain with 15%%-weighted questions in the study app.
Open in study appSource: Claude Docs — Citations · Independent study aid, not affiliated with or endorsed by Anthropic.