Studies that collect each question once per engine report how little the engines agree on their sources. Part of that disagreement is not between the engines at all: the same engine, asked the same question twice, already cites different sources. Our measurements ask every question at least twice per engine, so the two can be measured side by side.
When two answer engines cite different sources for the same question, how much of the difference would appear anyway if you asked one engine twice? That is the floor any cross-engine comparison has to clear before a difference can be put down to the engines.
Asked the same question twice, in the same run, the same engine's cited sources overlapped by 61.3% (95% interval 60.1%–62.6%) on average (6,119 pairs, 1,175 questions). Two different engines answering the same question overlapped by 11.3% (95% interval 10.7%–11.8%) (5,394 pairs).
Put as disagreement: two engines disagree on 88.7% of their sources, and one engine disagrees with itself on 38.7%. The second is 43.6% (95% interval 42.2%–45.0%) of the first, on the 1,175 questions that have both kinds of pair. A study that collects each question once per engine cannot separate the two: every one of those points is read as a difference between engines.
| Engine | Overlap | 95% interval | Identical sources | One side cited nothing | Pairs |
|---|---|---|---|---|---|
| Google AI Mode | 43.3% | 39.7% – 47.5% | 9.8% | 0.8% | 256 |
| Gemini | 38.8% | 37.5% – 40.1% | 0.5% | 13.9% | 1,490 |
| Google AI Overviews | 54.5% | 51.9% – 57.2% | 10.2% | 14.1% | 679 |
| ChatGPT | 61.9% | 59.3% – 64.2% | 30.8% | 3.9% | 1,389 |
| Perplexity | 79.6% | 77.8% – 81.3% | 46.9% | 1.0% | 2,305 |
The last column is the yardstick for each row: the average of the two engines' own overlap with themselves. A row well below it is a real difference between those engines; the gap between the yardstick and 100% is what re-asking alone produces.
| Engines | Overlap | 95% interval | Pairs | Same-engine yardstick |
|---|---|---|---|---|
| Google AI Mode · Gemini | 11.5% | 9.8% – 13.2% | 309 | 41.0% |
| Google AI Mode · Google AI Overviews | 26.3% | 22.7% – 29.9% | 147 | 48.9% |
| Google AI Mode · ChatGPT | 8.0% | 6.3% – 9.7% | 204 | 52.6% |
| Google AI Mode · Perplexity | 11.5% | 10.0% – 12.9% | 324 | 61.4% |
| Gemini · Google AI Overviews | 14.4% | 13.3% – 15.5% | 566 | 46.6% |
| Gemini · ChatGPT | 7.0% | 6.2% – 7.9% | 738 | 50.3% |
| Gemini · Perplexity | 15.5% | 14.5% – 16.5% | 927 | 59.2% |
| Google AI Overviews · ChatGPT | 9.8% | 8.2% – 11.6% | 253 | 58.2% |
| Google AI Overviews · Perplexity | 14.3% | 13.1% – 15.5% | 467 | 67.0% |
| ChatGPT · Perplexity | 7.6% | 7.1% – 8.2% | 1,459 | 70.7% |
Low cross-engine overlap is real — the engines do draw on different parts of the web — but a single collection overstates it. Before reading a gap between two engines as a difference in what they prefer, subtract what one engine does to itself on a second ask. The same applies over time: a source that "appeared" or "disappeared" between two single collections may only have been re-drawn.
This is the reason every AnswerRange rate is built from repeated asks and carries an interval, and why a change is only reported after a test (methodology).
node bin/rerun-noise.js reads the stored observation files and writes the summary this
page renders. The bootstrap is seeded, so the same observations give the same numbers. Only aggregates
are published: no questions, domains or answers leave the script.
Data: 9,996 answers, 1,225 questions, 17 runs, 6 markets, engines: Google AI Mode, Gemini, Google AI Overviews, ChatGPT, Perplexity. Generated 2026-10-29.
AnswerRange (2026). How much cross-engine disagreement is just re-run noise? Methods note. https://answerrange.com/notes/rerun-noise