ahmedcharef/Tunisia-Open-Gov-Data-RAG

RAG pipeline over Tunisian open government data using LangChain # Tunisia Open Government Data RAG Pipeline **Retrieval-Augmented Generation** over real Tunisian public datasets using LangChain, multilingual embeddings and frontier LLMs via OpenRouter. ## ✨ Features - Ingestion of Tunisian open government CSV datasets about **education establishments** - Multilingual embedding model that handles **Arabic + French** very well - Persistent Chroma vector database - Conversational RAG chain with history awareness - Easy model switching via OpenRouter (Qwen, Llama 3.3, Mistral, Claude, Gemini, …) - Local fallback to Ollama possible - Clean separation: ingestion / querying / configuration - CLI interface + **Streamlit Web UI** ## 🏗️ Project Structure ```text tunisia-rag/ ├── data/ # Put downloaded CSVs here ├── chroma_db/ # Persistent Chroma vector store (git ignored) ├── .env # API keys (git ignored) ├── requirements.txt ├── README.md ├── ingest.py # One-time data → vector store pipeline ├── query.py # Conversational RAG interface └── config.py # Central model & path configuration ``` ## Installation Guide Follow these steps to get the **Tunisia Open Government Data RAG Pipeline** up and running on your machine. ### 1. Prerequisites - **Python** ≥ 3.10 - **Ollama** (optional – only if you want to use a local LLM fallback instead of OpenRouter) - An **OpenRouter** account and API key → Create one (free tier available): openrouter.ai ### 2. Clone and Install Dependencies ```bash # Clone the repository git clone github.com cd Tunisia-Open-Gov-Data-RAG ``` #### (Recommended) Create and activate a virtual environment ```bash python -m venv .venv # On macOS / Linux source .venv/bin/activate # On Windows # .venv\Scripts\activate # Install all required Python packages pip install -r requirements.txt ``` ### 3. Configure Environment Variables copy and edit .e …