# How DevTune measures AI search visibility for dev tools

Canonical: https://devtune.ai/blog/how-devtune-measures-ai-search-visibility

Published: 2026-10-28T00:00:00.000Z

Author: Ben Williams


An AI assistant can recommend your product in one answer and leave it out of the next. Before you decide what to fix, you need to know how often that happens, on which questions and platforms, and from how many answers.

DevTune runs your prompts on a schedule, records the answers, shows the uncertainty around each rate and pairs it with what AI systems fetch from your site. That evidence drives the fixes we draft for your approval and the before/after comparison once you ship.

## Prompts

Our [State of AI Search for Dev Tools](https://devtune.ai/dev-tools-state-of-ai-search/2026) uses the same tracking data. The 2026 edition covers 43 categories, 1,075 test scenarios and 54,747 recorded answers from a frozen 90-day window, 2 April to 30 June 2026. Those answers produced 126,814 citation records, which the report defines like this:

> A citation record is one source link in one AI answer.

An answer can cite several pages, so a citation count isn't a count of answers, buyers or recommendations. The report benchmarks how five platforms answered questions we chose, asked weekly. It isn't a log of what buyers asked, a search-volume estimate or a live market ranking.

For your project, choose prompts around the decisions your buyers face, such as exploring a category, checking an integration or planning a migration, and group them into topics. Generated prompts avoid company and product names by default and vary buyer context and phrasing. They're a starting point, not evidence that customers asked them, so improve them with language from sales calls, support conversations and customer research. [Prompt selection](https://docs.devtune.ai/search-tracking/prompts).

Keep questions that name your product in separate topics from category questions. A high mention rate on prompts that supply your name doesn't show whether AI would name you unprompted. Changing questions, platforms or tracked competitors also changes what an aggregate measures, so check for that before reading a movement.

## Sample size and uncertainty

The standard tracking platforms run daily. Enterprise Claude and Grok checks run weekly unless the contract specifies daily checks. The report's weekly cadence is separate from your project's schedule. [Platform coverage and cadence](https://docs.devtune.ai/search-tracking/platform-comparison).

Prompt tables show a Presence Range: a 90% Wilson score interval calculated from citation-eligible answer counts. Below ten observations, the range is labelled directional. With no eligible observations, there's no range to show. [Presence Ranges](https://devtune.ai/changelog/ai-visibility-checks-presence-ranges).

![Illustrative 90% Wilson ranges for the same 50% presence rate: 27% to 73% with 10 eligible answers, 42% to 58% with 100, and 47% to 53% with 1,000.](https://devtune.ai/images/posts/measurement-presence-range-sample-size.svg)

_Calculated examples, not customer results._

The range models sampling uncertainty and nothing else. It doesn't correct an unrepresentative question set, make repeated answers independent or account for every platform change. A small movement needs more scrutiny than a large, sustained change across comparable observations.

## Platforms and metrics

Platforms cite differently. In the frozen report, ChatGPT Search produced 3.42 citation records per answer execution and Perplexity 1.56. Those figures describe that period and question set, not either platform permanently.

A blended number can move because one platform supplied more observations, so DevTune shows each platform's answers and sources separately. Daily trend lines leave out weekly-only platforms, so one weekly check doesn't look like a spike followed by six empty days. Whole-window totals still include them.

| Metric            | What it tells you                                                               |
| ----------------- | ------------------------------------------------------------------------------- |
| Brand mention     | The answer contains your brand terms                                            |
| Citation presence | The answer cites at least one of your tracked URLs                              |
| Share of voice    | Your citations divided by your citations plus configured competitors' citations |
| Recommendation    | The answer presents a product as an option to use                               |

A source can be cited without its publisher being named, and a brand can be named without a link. Share of voice counts only the competitors you configured. Reading the answer explains what the counts can't. [Metric definitions](https://docs.devtune.ai/search-tracking/visibility-metrics).

## Coverage and missing data

A timeout, missing citation evidence or a disconnected sensor doesn't tell you that you were absent. DevTune separates measured zeroes from unavailable measurements and shows coverage beside the results. Citation presence uses the answers with usable citation evidence, rather than treating every scheduled attempt as a measured miss.

![A Results view for one page: citations 4 to 17 and answer fetches 210 to 320 marked Observed; AI referrals marked Unavailable.](https://devtune.ai/images/posts/measurement-outcome-dimensions.webp)

_Illustrative product view from devtune.ai. A measure with no coverage shows as Unavailable, not zero._

## AI Traffic

AI Traffic counts crawls, answer fetches, referrals and selected conversion events separately. Cloudflare or edge middleware supplies server-side requests, including requests for missing pages. GA4 or PostHog supplies referral sessions and conversion events. Search Console adds classic-search performance. [AI Traffic](https://docs.devtune.ai/ai-traffic/overview).

![A live feed of answer fetches by page and AI platform, beside today's fetch counts by platform.](https://devtune.ai/images/posts/measurement-answer-fetches.webp)

_Illustrative product view from devtune.ai._

A browser snippet can't see a server-side fetch that never runs JavaScript. If a server-side sensor covered the full period and a page has no answer fetches, that zero is observed. Without that coverage, an empty chart can't show that AI systems never requested the page, and connecting a sensor today won't recover last month's requests. A citation doesn't establish a visit either, and these measures don't trace a path from one answer to a purchase. [Sensor capabilities and coverage](https://docs.devtune.ai/ai-traffic/choosing-sensors).

## Accuracy

Accuracy checks sample up to three qualifying answers per prompt, platform and week, and compare claims about your product with pages we've crawled on your site. Coverage shows which answers were checked, which are waiting and which couldn't be checked. Not every collected answer gets a factual review.

![An Accuracy summary: facts checked, share matching your site, contradictions and facts not on your site, broken down by AI platform.](https://devtune.ai/images/posts/measurement-accuracy-checks.webp)

_Illustrative product view from devtune.ai._

A fact missing from your site stays separate from a contradiction. Comparisons that span a sampling-method change are labelled as a method change, not shown as a trend. [Accuracy methodology](https://docs.devtune.ai/search-tracking/accuracy).

## Actions and Results

The evidence points to different fixes. Repeated requests for a missing page create 404 demand. A cited comparison page can carry an outdated claim. A docs page can attract answer fetches while the relevant questions still cite competitors. DevTune carries that evidence into Actions, where agents prepare a brief or draft for you to approve.

![A 404 demand list: missing paths ranked by AI requests and bots, each with a Create page action.](https://devtune.ai/images/posts/measurement-404-demand.webp)

_Illustrative product view from devtune.ai._

When an action goes live, Results compares the 14 days after the change with the 14 days before. A citation or answer-fetch change counts as movement only when it passes the chance threshold, moves the rate by at least 20%, and has at least three observations in the higher window spread across at least three days. Citation comparisons account for how many probe answers each window contained. The rule applies to rises and falls alike.

A result reads improved, mixed, no clear change, declined, still measuring or unmeasured, and lists other completed actions on the same page in the window. It shows what moved, not what caused it: platform updates, other publications and changes elsewhere on the site can contribute. [Results methodology](https://docs.devtune.ai/insights-actions/outcomes).

## Agent Picks

Agent Picks gives coding agents such as Claude Code and Codex a request in a fresh project, then records which tools they choose from the resulting code and answer. Paid tests repeat each environment, request, agent and model combination, and report pick rates, install rates, completed-run counts and 95% Wilson likely ranges. Prompt Presence Ranges use 90%.

A recorded pick or install doesn't show the integration works. Failed runs, incomplete coverage and outcomes we couldn't classify stay visible. [Agent Picks measurement](https://docs.devtune.ai/agent-picks/reading-results).

## API and MCP

Your own tools and agents can read the same measurement data through the API and MCP, including coverage and unavailable states. [API and MCP](https://docs.devtune.ai/api-reference/overview).

Start with the [State of AI Search for Dev Tools](https://devtune.ai/dev-tools-state-of-ai-search/2026) for category patterns. Then [measure your own product](https://devtune.ai/pricing) against the questions your buyers ask, on the platforms they use.