Chapter 9
A tone score changed more with language than company
Imagine you ask a shop robot about a toy car. A program on the counter reads the answer and gives it a tone score: above zero, zero or below zero. You ask again in another language. The toy is the same, and the score can change.
Your customers can ask a robot about your business in their own language, and a tool can put a score like this on the answer.
So I compared scores across languages for 20 companies operating in Central and Eastern Europe.
The short version. Most answers got a tone score of zero. Where scores differed, they went with the language of the question far more than with the company. English answers got most of the negative scores. Before you change your marketing because a score fell, read the answers and check whether they actually became worse.
What I did
A scoring program gave each saved answer a tone score from minus one to plus one. Then I compared how much of the movement went with language, company, robot and question.
Most answers got a score of zero
Picture it. Each dial is one answer. A green dial with an upright needle got zero. On the white dials the needle leans left for a negative score, on 5, or right for a positive one, on 3.
At Joost’s: like a program giving 92 of every 100 book reviews a score of zero. Some of them may still say whether the book is good.
A score is one program’s reading
A zero means the program labelled the text neutral. It does not mean a person read the answer and formed an opinion about the company.
The score went with language far more than company
Picture it. Say all the ups and downs fill 100 sheets of paper. These rows show three parts of that pile. White is what the sum could not split.
At Giulia’s: like a program scoring client letters. Of every 100 parts of its ups and downs, 26 or 27 go with the letter’s language and 1 or 2 with the client.
English answers got most of the negative scores
Picture it. The bar is all the answers, English in green. Each dial is one answer that got a negative score from a scoring program: green for English, white for the other seven.
At Mikko’s: like a program reading 12,933 car descriptions in 8 languages, and giving 591 of its 668 negative scores to the 1,620 in one language. Mikko still has to read them.
A second set, without English, gave a smaller share
I compared the full set with a second set that leaves English out. This is an exploratory check: it shows how the result changes when the set of answers changes.
Picture it. Each row is its own pile of 100 sheets: all the ups and downs in that set. Green goes with language. The piles stand for different totals, so do not subtract one from the other.
At Samira’s: like a program scoring ads in 8 languages, then with the English ones out. Language goes with 26 or 27 of 100 parts, then 5, each of its own total.
The program may read languages unevenly, or the answers may really differ. This check cannot tell those explanations apart.
How to check this for your business
Giulia’s clients in Milan ask in Italian, and some ask in English.
A tool gives the firm’s English answers lower tone scores. First, Giulia covers the scores and reads some answers in each language, writing down what each one says. Then Giulia uncovers the scores and checks where the tool and the notes match.
Try this on Monday, with your sheet
Check the tool before you act on its scores. Take answers to one question in each language your customers use. Put each answer on its own page with the score hidden. That makes it a blind sample: you read without knowing the score.
Write beside each answer what it says about you: good, bad, mixed or only facts. Then uncover the scores and compare them with your notes, one language at a time. For a language you cannot read, ask a person who can.
References
Chapter 9. A tone score changed more with language than company
Revised manuscript. Dmitrij Żatuchin (2026). Query Language Sensitivity in Automated Sentiment Scores of LLM Responses. Original preprint. arXiv:2607.13304v1. Related aggregate data. 10.5281/zenodo.20794390.
What does AI say about your business?
Put the book into practice. Tell us your company and website to request an AI visibility audit. We’ll reply by email to agree what to check.
What the audit shows
- Where your company appears in the answers, and next to whom
- The pages AI cites in your market
- What to publish first on your own site
