IIbitoye/NLP_RAG

This Research Portal focuses on Low-Resource Natural Language Processing (NLP), specifically targeting African languages. The domain presents unique challenges such as data scarcity, code-switching, and the need for culturally grounded evaluation metrics. # 🌍 Personal Research Portal (PRP) **Author:** Iteoluwa Ibitoye **Course:** AI Systems Management ## 📖 Overview This is a retrieval-augmented generation (RAG) pipeline and interactive web portal designed to answer research questions about **Low-Resource NLP and African Languages**. It ingests 30+ academic papers, chunks them, and uses OpenAI's GPT-4o with a "Retrieval-First" prompt strategy to ensure highly grounded, citation-backed answers. ## 🚀 Key Features Domain-Specific Corpus: Indexed 34 high-impact papers (Masakhane, NLLB, AfroBench). MMR Reranking: Uses Maximal Marginal Relevance to retrieve diverse perspectives for synthesis questions. Trusted Citations: Automatically maps vector chunks to formal academic citations (e.g., (Adebara et al., 2022)). Dual Logging: Generates clean reports for users and detailed retrieval logs for debugging. Hybrid Search Engine: Combines OpenAI Vector search with BM25 Keyword search for 30% higher recall on specific technical terminology. Reciprocal Rank Fusion (RRF): Intelligently merges and reranks results from multiple retrieval streams. ### ✨ What's New in Phase 3 * **Interactive Streamlit UI:** A conversational interface with real-time citation tracking and session memory. * **Automated Artifact Generation:** Instantly exports Evidence Tables (CSV), Annotated Bibliographies (APA Markdown), and 800+ word Synthesis Memos (Markdown). * **LLM Hot-Swapping:** A defensive engineering fallback that automatically routes requests to a local `Llama 3.2` model if the OpenAI API fails or hits rate limits. * **Evaluation Dashboard:** In-app metrics tracking system latency, groundedness, and citation correctness. --- ## 🛠️ Architecture * **Ingestion:** `PyPDFLoader` + `RecursiveCharacterTextSplitter` (Chunk size: 1000, Overlap: 200). * **Embedding:** OpenAI `text-embedding-3-small`. * **Vector Store:** ChromaDB (Persistent local database). * **Retrieval:** MMR (`k=12`, `fetch_k=20`) to reduce redundancy and enforce diverse context …