ValGrace/RAG-translation-engine

A Retrieval Augmented Generation (RAG) system for translating English text to Kalenjin using the Taaitta Kalenjin linguistic document. # English to Kalenjin RAG Translation System A Retrieval Augmented Generation (RAG) system for translating English text to Kalenjin using the Taaitta Kalenjin linguistic document. ## Setup ### Step 1: Install Dependencies **Option A - Using batch file (Windows):** ```bash install.bat ``` **Option B - Using pip:** ```bash py -m pip install -r requirements.txt ``` This may take several minutes as it downloads ML models. ### Step 2: Get Google API Key 1. Visit aistudio.google.com 2. Create or copy your API key 3. Create a `.env` file in the project root: ``` GOOGLE_API_KEY=your_api_key_here ``` ### Step 3: Extract PDF and Setup ```bash py extract_pdf.py py setup.py ``` ### Step 4: Start Translating ```bash py translate_cli.py ``` ## How It Works 1. **PDF Extraction**: Extracts text from the Kalenjin linguistic document 2. **Text Chunking**: Splits the document into manageable chunks 3. **Vector Embeddings**: Creates embeddings using sentence-transformers 4. **Vector Store**: Stores embeddings in ChromaDB for fast retrieval 5. **RAG Translation**: Retrieves relevant context and uses Google Gemini Flash 2.5 to generate translations ## Usage ### CLI Interface ```bash python translate_cli.py ``` ### Programmatic Usage ```python from rag_system import KalenjinTranslator translator = KalenjinTranslator() translator.setup() translation = translator.translate("water") print(translation) ``` ## Files - `extract_pdf.py` - Extracts text from the PDF document - `rag_system.py` - Core RAG translation system - `translate_cli.py` - Command-line interface - `requirements.txt` - Python dependencies - `.env.example` - Example environment variables ## Notes - The system uses the linguistic document as its knowledge base - Translation quality depends on the vocabulary and examples in the source document - The RAG approach retrieves relevant context before generating translations