Why this is harder than rank tracking
In classic search a position is fairly stable: if you are third for a phrase today, there is a good chance you are third tomorrow too. A tool checks once a day, and the time series means something.
An AI answer does not behave that way. The same question, in the same minute, asked twice gives two different answers — different wording, different sources, sometimes different brands. That is not a fault, it is how generation works. Measure once a day and you are not watching a trend, you are watching a coin toss.
The variance you have to understand first
Several independent measurements have put numbers on this, and the results agree to a startling degree.
- In a study with close to three thousand runs, the exact same brand list came back less than one percent of the time.
- In the same order, with a probability of a thousandth of a percent.
- Roughly two thirds of the cited sources turn over from one day to the next.
- And the one that matters most: the spread within a single day is about as large as the spread between days. So the swing does not come from the web, it comes from the model.
That last point is the crucial one. If the variance within a day is as big as the variance between days, then a single day's measurement cannot show you a change — "we moved up today" is most likely chance.
How many times you have to ask
There is a concrete recommendation here, and two independent sources say the same thing: seven to eight runs per prompt is the entry point for the ratio to mean anything. Below that you are not measuring, you are taking a sample from a noisy distribution.
There is guidance on the number of questions too: under the 2026 framework of an industry association, below fifty queries a measurement cannot be called indicative at all, only exploratory. So an honest monthly measurement is on the order of fifty questions, seven to eight runs each — around four hundred queries. That is the number to budget for.
What is stable, and what is not
The situation is not hopeless, and this is the good news. In the same study where the exact list barely repeated, the top three in narrow categories stayed put 55 to 77 percent of the time.
So: the order is random, the field is not. Asking "what position am I in" is pointless. Asking "am I in the running at all" is meaningful and measurable. Which is why the right metric is not position but appearance rate: in how many of a hundred runs you show up.
Official data: almost none
Anyone hoping the providers will hand out a report is going to be disappointed. None of the big providers has an official brand visibility tool.
What does exist: from the summer of 2026 the largest search engine reports some data on its generative surfaces in its console for webmasters — but impressions only, no clicks and no query terms, and with every AI surface merged into a single data set. Another provider points to the tracking parameter it uses in its references. The third has nothing. That is the entire official offering today.
The tools, and what their numbers mean
Dozens of tools on the market sell AI visibility measurement, and by the industry association's own analysis they share no definition of what counts as a mention — which means their numbers are not comparable.
- The most transparent solution queries the public interfaces with browser automation, and the vendor itself writes that the result is "a directional signal, not an exact number".
- Another builds on licensed conversation data and probabilistic modelling. It starts from a different base, so it gives a different number.
- Prices run from $29 to $1,500 a month.
- Several tools validate the metric they sell with that same metric — worth recognising.
The arithmetic that decides your plan
There is a practical collision here that nobody tends to spell out. A mid-tier plan typically gives you thirty checks per prompt per month. You can spend that in two ways.
Either you measure once a day — then you have thirty data points, but every one of them is noisy, and you saw above that a single run supports no conclusion. Or you measure four times a month, seven to eight runs each time — then you have four reliable data points. The second one is right. Fewer points, but each of them means something. Draw a chart from daily measurements and you are looking at the curve of the noise.
A measurement routine that works
1. FIFTY questions a real buyer would actually ask.
Not keywords: questions, the way they say them.
2. QUARTERLY or monthly, NOT daily.
7-8 runs per question each time.
3. WHAT YOU RECORD per question:
- was your brand there (yes/no), run by run
- who showed up instead of you
- which sources the answer cited
4. THE METRIC: appearance rate, not position.
"We are in 12 of 50 questions" — that is a meaningful sentence.
"We are third" — that is not.
5. SAME account, same country, logged out.
Memory and location shape the answer — the vendors say so.
Common misconceptions
- "The tool measures daily, so it is accurate." The spread within a day is as big as the spread between days. Daily measurement draws noise.
- "What position am I in the AI answer?" Wrong question: the order barely repeats. The right one is whether you appear at all.
- "Two tools give two numbers, so one of them is lying." Neither is: they share no definition of what counts as a mention.
- "The providers will publish official data." Today not one of them gives a usable brand visibility report.
What to take away
Before you subscribe to any tool, accept the two base numbers: seven to eight runs per question, and below fifty questions there is nothing to say. With that frame the measurement becomes meaningful — without it you will be paying monthly for a random number generator. And let the metric be the appearance rate, not the position. What the appearance itself depends on is set out in the How to Get Cited by AI Answer Engines post, factor by factor; and if you want to point the same routine at your rivals, Competitor Monitoring with AI is the post that covers it.