Domain 5 · Task 5.6

Information Provenance & Multi-Source Synthesis

Preserve information provenance and handle uncertainty in multi-source synthesis.

When a multi-agent system synthesizes findings from many sources, two things tend to get lost: which source backs which claim, and the fact that credible sources sometimes disagree. Summarization is where citations quietly drop and where two conflicting numbers get silently reconciled into one. This lesson is about preserving provenance through synthesis and representing genuine uncertainty honestly instead of manufacturing false certainty.

Key concept

Preserve claim→source through every hop; annotate conflicts with attribution and dates; don't collapse everything into one value or one format.

What you need to know

Attribution is lost during summarization

The reason a final report loses its citations is that the claim→source mapping wasn't preserved when intermediate results were summarized. The fix is to require subagents to output structured claim-source mappings — URLs, document names, excerpts — and to carry and merge those mappings through every synthesis hop, not just at the leaves. Provenance is a property you have to actively propagate; it does not survive a summarization pass on its own.

Annotate conflicts, don't resolve them

When two credible sources report conflicting statistics, the wrong move is to arbitrarily pick one value. The right move is to annotate the conflict with attribution — keep both values, each with its source, date, and methodology — and let the coordinator (or the reader) reconcile. Structure reports to distinguish well-established findings from contested ones, preserving each source's original characterization and methodology rather than flattening them into a single confident claim.

json

{
  "claim": "AI-generated share of streaming music",
  "values": [
    { "value": "12%", "source": "Spotify 2024", "date": "2024-03", "methodology": "auto-classification" },
    { "value": "8%", "source": "MIA Survey", "date": "2024-07", "methodology": "survey of 500 labels" }
  ],
  "conflict_detected": true,
  "possible_explanation": "different methodology and time period"
}

Temporal data and dates

A common false conflict comes from temporal mismatch: two figures differ only because they were collected at different times. Require publication or collection dates on every source so a time difference isn't misread as a contradiction — the 12% (March) vs 8% (July) example above may reflect change over time plus differing methodology, not an error in either source. Dates and methodology together let the coordinator explain a discrepancy instead of resolving it away.

Render by content type

Presentation is part of provenance-faithful synthesis. Render by content type rather than forcing one uniform format: financial data as tables, news as prose, technical detail as lists. A single format flattens meaningful structure — a table of conflicting statistics with their dates communicates the uncertainty far better than the same numbers dissolved into a paragraph. Match the rendering to the information so the structure that carries the provenance stays visible.

Exam traps

The trap	The reality
Two credible sources give different numbers, so the synthesis agent should pick the more reliable one and report a single value.	Preserve both values with attribution, dates, and methodology, and mark the finding contested. Arbitrarily collapsing conflicting sources hides real uncertainty.
The final report dropped its citations — that's just a formatting issue to fix at the end.	Citations are lost because claim-source mappings weren't carried through synthesis. Preserve them at every hop, not just the final render.
If two sources disagree, one of them must be wrong.	The difference is often methodology or collection date. Require dates so a temporal gap isn't misread as a contradiction.
A single consistent output format makes the report cleaner and easier to read.	Render by content type — tables for financial data, prose for news, lists for technical detail. A uniform format flattens the structure that carries provenance.

Practice scenario

Real questions from the bank that test this topic — the correct answer is highlighted.

You're building a Q&A system over a corpus of internal policy documents. End users must see exactly which document and which passage supports each part of Claude's answer — and you cannot tolerate fabricated citations.

Which approach gives the strongest guarantees against hallucinated citations?

AUse Claude's built-in citations feature: pass documents as inputs with citations enabled; Claude returns structured citation objects linking output spans to source spans in the documentsCorrect

BPrompt Claude with "always cite your sources at the end of each sentence" and parse the cited document names yourself

CUse RAG: retrieve top-k documents and prepend them to the prompt, asking Claude to cite by document title in its prose

DTwo-pass: one request to answer, a second request to attribute claims to documents

Why: Citations is a first-class API feature. When documents are passed as input with citations enabled, Claude returns structured citation objects that link spans in the output to specific spans in the source documents — these are real references the model produced, not paraphrases or fabrications. Prompt-based citing (B), RAG with prompt-instructed citations (C), and two-pass attribution (D) are all vulnerable to hallucination: the model can invent citations, name nonexistent documents, or get spans wrong. Use the API feature, not prompt engineering, when you cannot tolerate fabrication.

A research team builds a multi-agent system: a coordinator that delegates to four specialized subagents — web_search , document_analyst , synthesizer , report_writer . In production they hit two recurring problems:

The synthesizer occasionally produces reports that confidently state contradictions as if they were facts. Example: "Adoption was 35% in 2024" and "Adoption was 18% in 2024" both appear in one report.
The report_writer sometimes produces citations like "[Source: industry report]" without the source actually being one of the documents the document_analyst examined.

The team is considering a fix package. Which combination most directly addresses both root causes?

ALower the synthesizer's temperature to 0; add a citation-validation post-processing step that drops any citation not matching a known source ID

BRequire web_search and document_analyst to emit structured outputs that include publication dates with every data point (so the synthesizer can distinguish trend from contradiction), and require all subagents to emit structured claim-to-source mappings that the synthesizer must preserve and merge — so the report_writer never has to invent attributionsCorrect

CAdd a verification subagent that runs after the synthesizer and re-checks every claim against the original sources

DSwitch the synthesizer to a more capable model tier

Why: The two production problems have specific structural root causes: Contradictions about adoption percentages: The model isn't distinguishing trend from contradiction. The fix isn't temperature or a verifier — it's giving every data point a publication date so the synthesizer can reason about time properly. "35% in 2024" vs "18% in 2022" isn't a contradiction; it's growth. Fabricated citations: The fix isn't a post-processing validator (which can drop legitimate citations whose IDs got slightly garbled); it's structural claim-to- source mappings that the synthesizer preserves and merges, so the report_writer never has to invent attributions — they're already bound to the claims. A's temperature fix doesn't address the structural information loss. C (verification subagent) is a layer of defense, not a root-cause fix, and burns substantial tokens. D (better model) doesn't fix architectural information loss.

Build exercise

Synthesize a report that keeps citations and flags conflicts

~45 min

1
Set up two source subagents that return the same claim with different statistics, each with a URL, a date, and a methodology note.
Why: You need a genuine cross-source conflict with full provenance to test that it survives synthesis.
2
Run a baseline synthesizer that summarizes the two results into a single paragraph and record what happens to the numbers and citations.
You should see: The synthesizer collapses to one value and drops the source attributions — the two failures made visible.
3
Require each subagent to emit structured claim-source mappings and have the synthesizer merge those mappings rather than free-text summaries.
Why: Carrying the mapping through the hop is what keeps citations attached to claims.
4
When values conflict, have the synthesizer produce a conflict annotation listing both values with source, date, and methodology, plus conflict_detected and a possible explanation.
You should see: Both figures appear side by side, attributed and dated, instead of one silently winning.
5
Have the report distinguish well-established findings from contested ones and preserve each source's original characterization.
You should see: Contested claims are clearly marked, not folded into the confident findings.
6
Render the conflicting statistics as a table and a news finding as prose, and confirm the format matches the content type.
Why: Content-type rendering keeps the structure that communicates the uncertainty visible.

Sources

How we built our multi-agent research system Reduce hallucinations and preserve grounding Building effective agents

Drill Context Management & Reliability

Practice only this domain’s questions, untimed, with instant explanations.