Chapter 13

One count can fool you. Mine did.

A gift shop. A robot behind the counter hands over a paper list so long that it runs off the counter and curls at the bottom. A jar of sweets stands on the counter. A child watches from the left, and a man with a clipboard stands on the right.

Imagine a shop robot giving gift ideas while a helper ticks every company name it hears. The robot says “a weekend getaway”. The helper hears “Away”, a suitcase brand, and ticks it.

Your customers can ask robots which business to choose, and people, me included, can sell you counts of the names that come up. Those counts can go wrong with nobody noticing.

It happened to me. So I counted 574 saved gift answers again, using two reading robots that looked for names in each answer.

The short version. A number about AI answers comes from a counter, and a counter can be wrong. Two of mine were, and their numbers looked fine. Before you use such a number, ask how many answers it comes from, which names it holds, and whether a person checked it.

What I did

The Christmas and Valentine’s answers were already saved. I used two reading robots to count the names in them.

A counter can mistake a word for a name

My old counter found a suitcase brand in up to 21 of 30 answers. Robots really named it once in 150.

Picture it. Each suitcase is one answer. Green means Away was counted. The old counter looked for the letters, so “getaway” counted, and so did the plain word “away”.

What my old counter ticked30 suitcases in a row. 21 are green and 9 are white.21 of 30
What the answers really said150 suitcases in five rows. 1 is green and 149 are white.1 of 150

At Joost’s: like a search for a novel called Away that lands on 21 of 30 reviews, when 1 review in 150 names it.

On four of my five Valentine’s questions, Away looked like the top name. It was a mistake in my count.

A broken counter can turn long lists into nothing

My next counter used two reading robots. It kept a name when both found it. But if one reader failed to finish its list, the program treated that as “no names”. Any names found by the other reader then disappeared from the count.

A reading robot, an unfinished page with a question mark, and a completed page. A loop arrow links the pages to show that an unfinished read needs another try. This diagram shows a checking process, not measured counts.
An unfinished read needs another try

Picture it. The robot reads an answer. The middle page is unfinished. The loop arrow means try again before counting its names.

At Giulia’s: like a clerk leaving an unread folder in the “to check” tray, then opening it again before filing it as empty.

The new counter checks whether each reader finished. If a read fails, it tries again. Two readers can still miss the same name, so agreement does not make a count error-free.

Two jars of the same size. The left jar holds 16 green sweets. The right jar holds 95 green sweets.
Same robot, same question, ten answers each time: 16 different names once, 95 another time

Picture it. Each jar is one set of ten answers. Each sweet is one different company name in that set.

At Kadri’s: like ten buyers naming 16 builders between them, and ten others naming 95.

How to check this for your business

Mikko VirtanenCar dealer, Finland. Made-up example.

A report says robots keep naming Mikko’s rival, Drive. He reads the answers and finds “test drive” again and again. He asks who checked that those words meant the dealer.

Try this on Monday

Ask for the answers behind a report. Count the names in a few of the longest answers. If the counts differ, ask for a checked recount before using the report.

No report yet? Count the names in a saved answer with someone else and compare.

References

Chapter 13. One count can fool you. Mine did.

Data for the reanalysis. Dmitrij Żatuchin (2026). Supplementary Materials: Volume and Distribution as Separable Dimensions of Gender Bias in Large Language Model Brand Recommendation. Zenodo, version 1.0. 10.5281/zenodo.21904788.

What does AI say about your business?

Put the book into practice. Tell us your company and website to request an AI visibility audit. We’ll reply by email to agree what to check.

What the audit shows

  • Where your company appears in the answers, and next to whom
  • The pages AI cites in your market
  • What to publish first on your own site

Fields marked * are required.

Who is it for?

Your choice about cookies

We use cookies this site needs to work, and others only if you say yes. Choose what you allow, and change your answer whenever you like. Read our privacy policy