IsaakAlemu/amharic-rag-assistant

# Conversational Amharic RAG Assistant An end-to-end **conversational Retrieval-Augmented Generation (RAG)** system tailored for the Amharic language over an AmQA-derived Wikipedia knowledge base. The system features **hybrid retrieval (dense vector search + lexical BM25 fused via Reciprocal Rank Fusion)**, conversational multi-turn query rewriting, **multi-layer prompt-injection defenses**, strict factual grounding with inline citations, real-time token streaming, and automated CI testing. --- ## 1. Key Features - **Amharic-Native QA & Real-Time Streaming:** Responds to questions in natural, fluent Amharic with live token-by-token streaming in a Streamlit web interface. - **Hybrid Retrieval (Dense + BM25 via RRF):** Combines `intfloat/multilingual-e5-small` dense embeddings with an in-memory Amharic BM25 keyword retriever using Reciprocal Rank Fusion ($k=60$) to capture both semantic meaning and exact entity/acronym matches (e.g. `የተ.መ.ድ / ዩኤን ኤድስ`). - **Conversational Multi-Turn Context:** Uses an LLM rewriter to resolve pronouns, ellipsis, and context across turns into standalone retrieval queries without polluting the retriever with raw chat history. - **Evidence-Grounded Generation:** Enforces strict boundary rules where answers are derived exclusively from retrieved documents, citing sources inline as `[1]`, `[2]`. - **Bilingual Grounded Refusal:** Accurately outputs an explicit refusal (`"ከተሰጡት ሰነዶች በመነሳት ጥያቄውን መመለስ አልተቻለም።"`) when context is missing, preventing hallucinated assertions. - **Multi-Layer Security & Guardrails:** - Input sanitization (strips null bytes, control characters, and ` ` injections). - Adversarial classifier detecting prompt-injection and jailbreak attempts in both English and Amharic. - XML delimiter isolation (` `, ` `) preventing context confusion. - **Ge'ez Sentence-Boundary Chunking:** Custom text chunker aware of Ethiopic punctuation (`።`, `፤`, `?`, `!`) with sliding character overlap. - **Multi-Provider LLM Support:** Confi …