rohitmahali01/Dia-Rag

A research proposal for a Dialect-Aware RAG architecture designed to solve the Standardization Bottleneck in low-resource languages (Santali). # Dia-RAG: Dialect-Aware Retrieval-Augmented Generation Dia-RAG is a research framework that addresses a fundamental limitation of current Retrieval-Augmented Generation (RAG) systems when applied to **low-resource dialects**. Most multilingual RAG pipelines rely on **translation-based standardization** of user queries into high-resource languages (e.g., English or Hindi). While effective for fluency, this process often **erases cultural meaning**, introduces **factual distortion**, and amplifies **bias**—especially for dialectal and indigenous languages. The full research proposal detailing the problem formulation, architecture, and evaluation plan is provided in `docs/Dia-RAG-Proposal-V1.pdf`. Dia-RAG proposes a principled alternative: **decouple and augment**, rather than translate and rewrite. --- ## Core Problem: The Standardization Bottleneck When a user queries in a dialect (e.g., Santali): - The query is translated into a standard language - Dialect-specific terms are approximated or mistranslated - Cultural context is lost - The generator hallucinates facts that “fit” the standard language This leads to errors such as: - Translating *Manjhi* as *boatman* instead of *village headman* - Rewriting queries in ways that invent entities or events - Producing culturally generic or incorrect answers We refer to this failure mode as the **Standardization Bottleneck**. --- ## Key Idea Dialectal queries are **not monolithic strings**. They are composed of: - **Invariant components**: entities, proper nouns, dates (stable across languages) - **Variant components**: dialect-specific, culturally loaded terms (prone to mistranslation) Dia-RAG explicitly models this distinction. --- ## Dia-RAG Architecture Dia-RAG consists of three core modules: ### 1. Diversity-Sensitive Decoupling The input query is decomposed into invariant and variant components using a lightweight instruction-tuned model. - Invariant components → exact matching in the knowledg …