A study on language fairness in AI

Speaking বাংলা to AI can cost over 5× more.

AI models read text in small pieces called tokens, and every token is billed. The same sentence in Bangla is cut into far more pieces than in English. That can mean paying more, waiting longer, and fitting less into the model's memory.

Try your own text See the proof

The problem in 3 steps

One word is enough to see it. Both words below mean the same thing.

  1. 1

    AI reads text in pieces

    Before a model can read anything, it cuts the text into tokens. A common English word is usually a single piece.

  2. 2

    Bangla is cut into more

    The same word in Bangla is broken into many more pieces. Some letters are even split into raw bytes.

  3. 3

    Every piece is billed

    AI companies charge per token. Same meaning, more pieces, higher price.

Example measured with GPT-4's tokenizer.

The calculator

Pick a sample or write your own Bangla sentence with its English translation.

Model Tokens বাংলা English Bangla ÷ English If English costs ৳100

Check it yourself

You don't have to trust this site. Paste the same text into OpenAI's own tokenizer and compare the numbers.

  1. Copy the Bangla text, then the English text.
  2. Open OpenAI's tokenizer and pick the matching model.
  3. You should see:
Open OpenAI tokenizer ↗

Under the knife

Every block is one piece the model sees. A red number means a single piece cost several tokens, usually because one Bangla letter was chopped into raw bytes. Tap any block to see inside it.

বাংলা
English

The ranking

How many Bangla tokens each model needs for every English token, across all 8 sample pairs. The line marks 1×, where Bangla would be treated as fairly as English.

See every sentence

Each cell is Bangla ÷ English tokens for one sentence pair. If the colours in a row are similar, the result is consistent and not driven by one sentence.

Small hand-written sample set. The full FLORES-200 study with 1,012 sentences gives the same ranking; see Proof & method.

Proof & method

What this site measures, what it doesn't, and how you can check every number.

How strong is each claim?

Measured

Bangla needs more tokens than English for the same meaning.

Counted on this page with each model's own tokenizer. Anyone can repeat it and get the same numbers.

Expected

Slower replies and less room in the model's memory.

Models write one token at a time and have token-based memory limits, so this follows from the counts. Not measured on this site yet.

Checked at scale: 1,012 professional translations

The demo sentences on this page are few and short. To test the result properly, all 8 tokenizers were run on FLORES-200, a standard research benchmark of 1,012 sentences professionally translated from English into Bangla. The ranking is the same, and the gap is slightly larger.

Black lines show 95% confidence intervals (10,000 bootstrap resamples). GPT counts match OpenAI's official encodings in 8,036 of 8,036 cases. The result repeats on a second set of 997 sentences.

Read the full report Download the results (CSV)

How the numbers are made

  1. Your text
  2. Tokenizer runs in your browser
  3. Token IDs
  4. Count
  5. Compare
Data
8 Bangla–English sentence pairs on everyday topics, written for this demo. They are not professional translations.
What is counted
The tokens each tokenizer produces for the text, without the special start and end tokens.
The ratio
Total Bangla tokens ÷ total English tokens across all pairs. 1× means equal treatment.
Verified
GPT-4 and GPT-4o counts were cross-checked against OpenAI's published encodings (cl100k_base, o200k_base) using js-tiktoken: 16 of 16 counts match.
Measured
October 2026
Privacy
Nothing you type is sent anywhere. Only the tokenizer files are downloaded.

Tokenizers used

Get the raw data

Every sentence, every model, every count. Open it in Excel or Google Sheets and check the math yourself.

Earlier research on the same problem

This site applies those ideas to Bangla with current models and makes the result something anyone can try.

Why it matters

Documented

Higher cost

AI APIs bill per token. If a Bangla message needs 5× the tokens, a Bangladeshi startup pays 5× for the same feature.

Expected

Slower replies

Models write one token at a time. More tokens for the same answer means people writing in Bangla wait longer.

Expected

Less memory

A model's context window is measured in tokens. Bangla fills it faster, so the model can read less of your document.

So why does it happen?

Tokenizers learn their vocabulary from training text that is mostly English, so common English words become a single token. Bangla is rare in that text, and every Bangla letter takes 3 bytes in UTF-8. The tokenizer often has no piece for it and falls back to raw bytes, splitting a single ক into two or three tokens. Newer tokenizers with larger, more multilingual vocabularies, like GPT-4o's, close much of the gap.

Questions

What is a token, in plain words?

A small piece of text. AI models never see whole sentences; they see a list of numbered pieces. In English a piece is often a whole word. In Bangla it is often a part of a letter.

Is my text sent to a server or to an AI company?

No. Counting happens inside your browser. The site only downloads each model's tokenizer file (its "dictionary") from Hugging Face. Your text stays on your device.

Are these the real tokenizers the companies use?

Yes, but some come from community copies on Hugging Face rather than the company itself; the "Tokenizers used" table marks which. GPT-4 and GPT-4o were checked against OpenAI's published encodings and all 16 counts matched. You can check any GPT number with the "Check it yourself" box.

Does more tokens mean the AI understands Bangla worse?

Not necessarily. This site measures cost in tokens, not understanding. Research has found that poor tokenization can hurt quality too, but that is a separate question this site doesn't test.

Why is BLOOM fair?

BLOOM's tokenizer was trained on data that deliberately included Bangla and other Indic languages, so it learned whole Bangla words as pieces. It suggests the gap comes from what a tokenizer learns from, not from the language itself.

The demo has only 8 sentences. Is that enough?

They are enough to show the effect, but not for a scientific answer. That is why the same test was also run on FLORES-200, a standard benchmark of 1,012 professionally translated sentences. It confirms the result; see "Checked at scale" in Proof & method.

Does the ৳ figure show real prices?

No. It is a comparison: if a message costs ৳100 in English, how much would the same message cost in Bangla on the same model. Real prices differ by model and change over time.

Who made this, and why?

Lamia Islam Mim, a Computer Science graduate from North South University in Dhaka. The goal is to make a hidden cost visible, so Bangladeshi developers can plan for it and more people ask for AI that treats Bangla fairly.