# How much is another AI answer worth?

Canonical: https://devtune.ai/blog/repeat-ai-answers-sampling-study

Published: 2026-11-07T00:00:00.000Z

Author: Ben Williams


AI answers change from run to run, so every answer is a sample. A citation rate is only as precise as the number of answers behind it.

If one run a day leaves a range too wide to read, the obvious fix is to run each prompt more often. We wanted to know how much that buys.

Less than you'd hope. Answers to the same prompt come out alike, so a second run on the same day adds well under a second answer. Adding prompts, meaning new questions on the same topic rather than rewordings of one you already track, narrows a range faster than repeating them.

## What we wanted to know

DevTune re-runs the same prompts every day. A rate from 100 independent answers is more precise than one from 20, but answers to the same prompt aren't independent. If ChatGPT cites one of a category's brands for a prompt this morning, it's likely to cite one again this afternoon.

We measured that with ρ, which says how alike two answers to the same prompt are. At 0, a repeat is as good as a new answer. At 1, it adds nothing. In between, m runs of one prompt are worth m ÷ (1 + (m − 1) × ρ) independent answers.

We measured ρ three ways: for back-to-back repeats, for runs spread through the day and for runs on different days.

## How we ran it

- 20 prompts from a tracked prompt set in one developer-tool category, run on ChatGPT search and Perplexity's Sonar API.
- Six days, November 2 to 7, 2026.
- Five back-to-back runs of every prompt at 09:00 UTC, on five of those days.
- One run of every prompt at 00:00, 06:00, 12:00 and 18:00 UTC.
- 1,676 answers in all, 836 from ChatGPT and 840 from Perplexity.

We scored each answer on whether it cited at least one of the 14 brands tracked in that category, using the same parser and citation classifier as DevTune. ChatGPT did in **34.3%** of answers and Perplexity in **36.1%**. That's the yes-or-no question behind a citation rate, and at about one answer in three it varies enough to show how alike repeats are.

ρ is the one-way random-effects intra-cluster correlation on those yes-or-no scores. Each range is a 90% bootstrap over prompts, with 1,000 resamples.

## What we found

### Repeats are alike, but not copies

None of the 998 back-to-back answers repeated an earlier answer from the same burst word for word. Whether they cited a tracked brand was much more alike.

![Measured ρ with 90% ranges. Back-to-back repeats: ChatGPT 0.27 (0.17 to 0.35), Perplexity 0.22 (0.09 to 0.35). Runs about 6 hours apart: ChatGPT 0.16 (0.03 to 0.27), Perplexity 0.22 (0.09 to 0.33). Across days: ChatGPT 0.25 (0.03 to 0.46), Perplexity 0.25 (0.08 to 0.41).](https://devtune.ai/images/posts/repeat-runs-rho.svg)

_ρ for whether an answer cited one of the tracked brands, with 90% ranges from resampling prompts. 20 prompts, November 2 to 7, 2026._

Back-to-back repeats on ChatGPT had a ρ of **0.27**. Runs about six hours apart had **0.16**. On Perplexity, ρ was 0.22 both ways. Across days, both platforms were at 0.25.

The ranges are wide. With 20 prompts, ChatGPT's spread figure could be anywhere from 0.03 to 0.27, and it overlaps the back-to-back range. Treat the gap between them as a lead, not a settled difference.

### Each extra run adds less


![Independent answers that 1 to 10 runs of one prompt in a day are worth, against a dashed line for fully independent runs. Back-to-back, 10 runs are worth 2.9 on ChatGPT and 3.4 on Perplexity. Spread through the day, 10 runs are worth 4.1 on ChatGPT and 3.4 on Perplexity.](https://devtune.ai/images/posts/repeat-runs-effective-answers.svg)


On ChatGPT, five back-to-back runs were worth **2.4** independent answers. Four runs spread through the day were worth **2.7**, more from fewer runs.

On Perplexity, both estimates came out at 0.22, so the study found no difference between back-to-back and spread runs. With ranges this wide, that doesn't show there is none. Five runs were worth 2.7 and four were worth 2.4.

There's a ceiling too. However many times you run a prompt in a day, the runs can't be worth more than 1 ÷ ρ independent answers. That's about 3.7 for back-to-back ChatGPT runs, 6.3 for ChatGPT runs spread through the day and 4.5 on Perplexity.

### More prompts narrow the range faster

Take a topic of five prompts with a citation rate around 35%, close to what we measured. At one run a day for 30 days, the 90% range on those five prompts' rate is about **±6.4 points**.


![The 90% range on a 35% citation rate for a topic of 5 prompts over 30 days. One run a day: ±6.4 points on both platforms. 2 runs a day: ±4.9 on ChatGPT, ±5.0 on Perplexity. 10 prompts once a day: ±4.5. 4 runs a day: ±3.9 on ChatGPT, ±4.1 on Perplexity. 20 prompts once a day: ±3.2.](https://devtune.ai/images/posts/repeat-runs-range-width.svg)


On ChatGPT, two runs a day narrows the range to ±4.9. Ten prompts run once a day use the same number of answers and narrow it to ±4.5. Four runs a day gets to ±3.9, and twenty prompts once a day to ±3.2. Perplexity follows the same order.

## What it means if you track AI search

### Add prompts before you add repeats

A second run of a prompt on the same day added between **0.57** and **0.72** of an answer in our data, depending on platform and spacing. A new prompt adds a whole one. To narrow the range on a topic, track more prompts in it.

By more prompts we mean new questions on the same topic, not rewordings of one you already track. Say a topic covers error monitoring for a Next.js app, and you track "best error monitoring for a Next.js app". Prompts worth adding ask the topic's question from another angle:

- "how do I catch frontend errors in production"
- "error monitoring with session replay for a small team"
- "cheapest error tracking for an early-stage startup"

Keep them inside the topic, because the range is for that topic's rate. Make each one a question someone would ask on its own. "Best error tracking for a Next.js app" next to "best error monitoring for a Next.js app" is likely to get much the same answers and count for little more than a repeat. We didn't measure how alike different prompts are, so treat that last point as a rule of thumb.

### If you repeat, spread the runs out

On ChatGPT, runs six hours apart were less alike than back-to-back runs, 0.16 against 0.27. Spreading them out takes no extra runs. On Perplexity, the study found no difference.

### One prompt over many days speaks for that prompt

Across days, ρ was 0.25 on both platforms. A prompt you run every day keeps giving answers like its own earlier answers. A long series of one prompt tells you a lot about that prompt and much less about the prompts you'd group with it.

If you read one prompt's daily series as evidence about the wider set of prompts like it, that prompt counts for at most 1 ÷ 0.25, about 4 independent answers, however many days it runs. The across-days ranges were the widest we measured, so treat that as rough.

The range chart above gives the range for the rate on the prompts in the topic, which is how DevTune calculates ranges. Read the same 30 days of five prompts as the rate for prompts like them, and the across-days correlation widens the one-run-a-day range from ±6.4 to about ±18 points. Adding prompts then helps far more than adding runs.

## Limits

- One prompt set of 20 prompts in one category. Other prompt sets may behave differently.
- We measured one outcome, whether any tracked brand was cited. A single brand's citations may be more or less alike from run to run.
- The ranges are wide. Every ρ we measured has a 90% range at least 0.18 wide.
- ChatGPT doesn't report which model answered, so we can't rule out a model change during the run. Perplexity reported `perplexity/sonar` throughout. We saw no sign of a model change on either platform.
- The range calculation treats different prompts as independent of each other. Prompts in one topic can be alike too, which would widen every option in the range chart, and more so for close rewordings.
- Six days in early November. ρ could move when the platforms change.

## How DevTune uses it

DevTune shows a 90% range and the number of answers behind its visibility rates. [How DevTune measures AI search visibility](https://devtune.ai/blog/how-devtune-measures-ai-search-visibility) explains how we calculate that range and how to read it.