Why AI recommends different businesses every time
You ask ChatGPT "Which auto repair shop near me can I trust?" and it recommends you. You're pleased and take a screenshot. The next day a friend asks the same question, and you're not on the list. Both results are real. Unlike a traditional search engine, AI search has no fixed "ranking."
Why answers change
- Generation has randomness built in. A language model picks each word from several possible words based on probability. For the same question, the wording and the names listed can be different every time.
- It finds different web pages each time. An AI with search looks up web pages before it answers. Which keywords it searches and which results it opens can vary each time, and when it reads different material, it reaches different conclusions.
- Different people are asking. Location, language, earlier conversation and whether memory is turned on all affect the answer.
- The question is phrased a little differently. "Which one is good?", "Which one is cheap?" and "Which one is good for kids?" each get a different list.
- Models and indexes keep updating. Next month, the model behind the answers and the pages it finds may both have changed.
So don't go by a single result
A screenshot that says "AI recommended me" and a screenshot that says "AI didn't recommend me" prove nothing on their own. What matters is the proportion after asking many times:
Mention rate = answers that mention you ÷ total answers
For example, 10 questions asked 3 times each gives 30 answers. If 6 of them mention you, your mention rate is 20%.
How to read a mention rate
- It's an estimate. A 20% rate from 30 answers could come out as 13% or as 27% with a different batch of 30. Only draw conclusions when the difference is clear.
- Look at each engine separately. You might show up often in one AI and not at all in another, because they use different search sources.
- Look at each question separately. An overall rate of 20% might mean two questions mention you every time and the other eight never do. That's more useful than the average: it tells you which kinds of customers the AI thinks you're a good fit for.
- Compare with competitors. How often competitors appear in the same set of answers says more about where you stand than your own number alone.
- Watch the trend. Test again with the same questions after a while. The direction of change matters more than any single number.
Be skeptical of "guaranteed placement"
Since the answers themselves keep changing, no one can guarantee that AI will recommend a particular business, and no one can guarantee you'll be "number one." What you can do is raise the odds of being mentioned: make sure AI can read your website, make your website clearly say who you are, where you are and who you serve, and make sure other trustworthy websites have consistent information about you. Then test again and see whether your mention rate has changed.
Related articles
Want to know your business's mention rate right now? The free check asks 10 questions for you, on 3 AIs, 3 rounds each.
Apply for a free check