Sinhala & Tamil LLM (AI) Cost Calculator

Every call to an LLM API is billed per token— the sub-word pieces a model reads and writes. Tokenizers are trained mostly on English, so English words map to few tokens. Sinhala and Tamil use non-Latin scripts stored as multi-byte UTF-8, and the model has to spend several tokens to represent a single character. The result is a measurable “token tax” documented by Petrov et al. (NeurIPS 2023): low-resource scripts need multiples more tokens than English for the same meaning, and you pay for every one.

This calculator quantifies that tax in four steps:

1. Measure real token density.A parallel set of five meaning-equivalent sentences in English, Sinhala, and Tamil is tokenized with OpenAI's open-source o200k_base tokenizer (the exact one GPT-4o and GPT-5 use), which runs in your browser via gpt-tokenizer. The measured averages are 1.18 tokens/word for English, 3.25 for Sinhala, and 2.53 for Tamil.
2. Compute the tax ratio. Per word, Sinhala costs 2.75× and Tamil 2.14× the tokens of English. On the older cl100k_base tokenizer (GPT-4, GPT-3.5) the same corpus is far worse — about 7.6× for Sinhala.
3. Convert words to tokens per message.Your average input and output word counts are multiplied by the language's token density for the chosen model: tokens = words × tokensPerWord(language, model). OpenAI models use the exact o200k count; Claude, Gemini, and Llama apply a clearly-labelled per-vendor multiplier because they do not publish a browser tokenizer.
4. Price the month. Monthly tokens = messages/day × 30 × tokens/message. Cost = (inputTokens × Pin + outputTokens × Pout) ÷ 1,000,000 in USD, then × your USD→LKR rate for rupees. The same maths is run for English to show the extra spend, and across every model for the comparison table.

As an independent cross-check, the per-word densities also reconcile through the character path — characters-per-word × tokens-per-character reproduces the same tokens-per-word to five decimals — confirming the stored corpus constants are internally consistent. All prices and the exchange rate are dated constants; nothing is fetched at runtime and no text you enter ever leaves your browser.

• OpenAI — tiktoken (o200k_base / cl100k_base BPE tokenizers)
• gpt-tokenizer — the in-browser JS port that produces the exact counts
• Petrov et al. (NeurIPS 2023) — Language Model Tokenizers Introduce Unfairness Between Languages
• OpenAI — API pricing (per-1M input/output token rates)
• Anthropic — Claude API pricing
• Google — Gemini API pricing
• Central Bank of Sri Lanka — indicative USD→LKR exchange rate

Token densities were measured with gpt-tokenizer's o200k_base and cl100k_base encoders, and prices and the exchange rate were last cross-checked against the pages above on 2026-07-08. Claude, Gemini, and Llama token counts are labelled approximations because those vendors do not publish a browser tokenizer.