Chapter 9

A tone score changed more with language than company

A shop counter in front of a wall of toy cars. One robot has two speech bubbles. Each bubble holds the same toy car and a small dial, one with its needle upright and one with its needle leaning left. A machine with a dial reads a page on the counter. A child looks on, and a man with a clipboard takes notes.

Imagine you ask a shop robot about a toy car. A program on the counter reads the answer and gives it a tone score: above zero, zero or below zero. You ask again in another language. The toy is the same, and the score can change.

Your customers can ask a robot about your business in their own language, and a tool can put a score like this on the answer.

So I compared scores across languages for 20 companies operating in Central and Eastern Europe.

The short version. Most answers got a tone score of zero. Where scores differed, they went with the language of the question far more than with the company. English answers got most of the negative scores. Before you change your marketing because a score fell, read the answers and check whether they actually became worse.

What I did

A scoring program gave each saved answer a tone score from minus one to plus one. Then I compared how much of the movement went with language, company, robot and question.

Most answers got a score of zero

100 small dials in rows. 92 are green with the needle upright. 5 are white with the needle leaning left, and 3 are white with the needle leaning right.
About 92 of every 100 answers got a score of exactly zero

Picture it. Each dial is one answer. A green dial with an upright needle got zero. On the white dials the needle leans left for a negative score, on 5, or right for a positive one, on 3.

At Joost’s: like a program giving 92 of every 100 book reviews a score of zero. Some of them may still say whether the book is good.

A score is one program’s reading

A zero means the program labelled the text neutral. It does not mean a person read the answer and formed an opinion about the company.

The score went with language far more than company

Language went with 26.5 of every 100 parts of the ups and downs in the scores. The company went with 1.5.

Picture it. Say all the ups and downs fill 100 sheets of paper. These rows show three parts of that pile. White is what the sum could not split.

The language of the question26 and a half green sheets.26.5
Which company1 and a half green sheets.1.5
What the sum could not split69 white sheets and a bit more.69.3

At Giulia’s: like a program scoring client letters. Of every 100 parts of its ups and downs, 26 or 27 go with the letter’s language and 1 or 2 with the client.

English answers got most of the negative scores

English answers got 591 of the 668 negative scores, from 1,620 of the 12,933 answers

Picture it. The bar is all the answers, English in green. Each dial is one answer that got a negative score from a scoring program: green for English, white for the other seven.

English answers, out of all the answersA bar with its first eighth in green.1,620
English answers, out of all the negative scores668 tiny dials with the needle leaning left. 591 are green and 77 are white.591

At Mikko’s: like a program reading 12,933 car descriptions in 8 languages, and giving 591 of its 668 negative scores to the 1,620 in one language. Mikko still has to read them.

A second set, without English, gave a smaller share

I compared the full set with a second set that leaves English out. This is an exploratory check: it shows how the result changes when the set of answers changes.

Two sets of answers give two shares for language: 26.5 and 5.1

Picture it. Each row is its own pile of 100 sheets: all the ups and downs in that set. Green goes with language. The piles stand for different totals, so do not subtract one from the other.

All 12,933 answers, 8 languages100 sheets, 26 and a half of them green.26.5
11,313 answers, 7 languages, no English100 sheets, 5 and a bit of them green.5.1

At Samira’s: like a program scoring ads in 8 languages, then with the English ones out. Language goes with 26 or 27 of 100 parts, then 5, each of its own total.

The program may read languages unevenly, or the answers may really differ. This check cannot tell those explanations apart.

How to check this for your business

Portrait of Giulia, the accountant.Giulia RomanoAccountant, ItalyMade-up example

Giulia’s clients in Milan ask in Italian, and some ask in English.

A tool gives the firm’s English answers lower tone scores. First, Giulia covers the scores and reads some answers in each language, writing down what each one says. Then Giulia uncovers the scores and checks where the tool and the notes match.

Try this on Monday, with your sheet

Check the tool before you act on its scores. Take answers to one question in each language your customers use. Put each answer on its own page with the score hidden. That makes it a blind sample: you read without knowing the score.

Write beside each answer what it says about you: good, bad, mixed or only facts. Then uncover the scores and compare them with your notes, one language at a time. For a language you cannot read, ask a person who can.

References

Chapter 9. A tone score changed more with language than company

Revised manuscript. Dmitrij Żatuchin (2026). Query Language Sensitivity in Automated Sentiment Scores of LLM Responses. Original preprint. arXiv:2607.13304v1. Related aggregate data. 10.5281/zenodo.20794390.

What does AI say about your business?

Put the book into practice. Tell us your company and website to request an AI visibility audit. We’ll reply by email to agree what to check.

What the audit shows

  • Where your company appears in the answers, and next to whom
  • The pages AI cites in your market
  • What to publish first on your own site

Fields marked * are required.

Who is it for?

Your choice about cookies

We use cookies this site needs to work, and others only if you say yes. Choose what you allow, and change your answer whenever you like. Read our privacy policy