Ep. 666: Why It Costs More to Talk to AI in Your Native Tongue

Episode summary: In this episode, Herman and Corn dive deep into the "Great Data Exhaustion" and the widening digital divide in artificial intelligence. While major frontier models seem like magic in English, speakers of "long-tail" languages face a "tokenization tax" that makes AI slower, more expensive, and prone to Western-centric hallucinations. From the grassroots efforts of the Masakhane project in Africa to the specialized architecture of models like Jais, we explore how the industry is finally being forced to look beyond the English-speaking bubble to ensure cultural sovereignty in the age of machine learning.

Show Notes

On a chilly February afternoon in Jerusalem, podcast hosts Herman and Corn Poppleberry sat down to tackle one of the most pressing, yet often overlooked, crises in the development of artificial intelligence: the linguistic digital divide. Triggered by a listener's question regarding the performance of AI for speakers of "long-tail" languages, the brothers explored whether the current AI revolution is a universal human achievement or merely a sophisticated echo chamber for the English-speaking world.

### The Great Data Exhaustion and the Long Tail The discussion began with Herman defining the current state of AI training, a period researchers are calling the "Great Data Exhaustion." For years, AI developers have relied on the massive, easily accessible troves of English-language data found on the internet. However, as the industry runs out of high-quality English text to scrape, they are finally being forced to look toward the "long tail" of human language.

Herman explained that if you graph languages by the amount of available digital data, English is the undisputed king, followed by high-resource languages like Spanish, Chinese, and French. However, the curve drops off sharply. "Long-tail" languages—such as Icelandic, Quechua, Wolof, or specific dialects of Arabic—have a much smaller digital footprint. This scarcity isn't necessarily a reflection of the number of speakers, but rather a reflection of digital literacy, internet access, and oral traditions that haven't been archived by projects like Common Crawl.

### The Tokenization Tax: A Literal Cost of Language One of the most striking insights from the episode was the concept of the "tokenization tax." Corn and Herman broke down the technical reality that AI models do not read words, but "tokens"—small chunks of text. In high-resource languages like English, common words are often a single token. In contrast, when a model encounters a long-tail language it hasn't seen much of, it must break words into five or six tiny, nonsensical fragments to process them.

Herman argued that this creates a two-tiered system of AI utility. First, it fills up the model's "context window" much faster, meaning a speaker of a long-tail language has a significantly shorter functional memory for their prompts compared to an English speaker. Second, because AI companies charge by the token, users of languages like Telugu or Amharic are literally paying more for the same amount of information. It is a financial and technical penalty for simply using one's native tongue.

### The English-Speaking Bubble and Cultural Hallucination The conversation then shifted to the "English-speaking bubble." Even when models are capable of speaking a long-tail language through a process called "cross-lingual transfer," they often carry a heavy Western bias. Herman explained that the model essentially "thinks" in the logic of its primary training data—English—and then maps those concepts onto the target language.

This results in "cultural hallucinations," where the AI might use grammatically correct words but apply Western-centric values to concepts like family, justice, or property. Corn noted that even in a mid-resource language like Hebrew, the AI often feels "stiff" or "formal," failing to capture the lived-in reality of modern slang or the blending of cultures. The risk, the brothers noted, is that the world is being told that to use the most powerful tools in history, they must conform to a Western worldview.

### Moving Toward Linguistic Sovereignty Despite the challenges, the episode highlighted several beacons of hope. Herman pointed to a shift away from the "scrape everything" mentality toward more intentional, community-led data collection. He cited the **Masakhane project** in Africa as a primary example. Instead of relying on Silicon Valley to "solve" African languages, Masakhane is a grassroots organization of native speakers and researchers building their own high-quality, culturally relevant datasets.

The hosts also discussed the rise of specialized, sovereign models. Rather than one giant "God-model" trained on the whole web, nations and regions are building models tailored to their specific linguistic needs. The **Jais model** in the United Arab Emirates was highlighted as a success story—a model that outperformed much larger Western counterparts in Arabic because it was built from the ground up with that language and culture in mind.

### Conclusion: The Future of the Digital Divide As the sun set over the stone walls of Jerusalem, Herman and Corn concluded that the fight for language equity in AI is about more than just translation—it is about sovereignty. If the future of work and creativity is to be built on AI, then every culture must have the right to be "legible" to the machine on its own terms.

The "tokenization tax" and the English-speaking bubble are significant hurdles, but the move toward synthetic data and community-led initiatives offers a path forward. The goal, as Herman put it, is to ensure that the AI revolution doesn't just create a new kind of digital divide, but instead provides a platform where the "long tail" of human culture can finally be heard.

Listen online: https://myweirdprompts.com/…

My Weird Prompts is an AI-generated podcast. Episodes are produced using an automated pipeline: voice prompt → transcription → script generation → text-to-speech → audio assembly. Archived here for long-term preservation. AI CONTENT DISCLAIMER: This episode is entirely AI-generated. The script, dialogue, voices, and audio are produced by AI systems. While the pipeline includes fact-checking, content may contain errors or inaccuracies. Verify any claims independently.