AhmedBensalah8603/Tunisian_Agentic_RAG

Agentic RAG for tunisian dialect and arabic based documents # 🇹🇳 Agentic RAG TN (Tunisia): Safe, Agentic Retrieval + Gemini Fallback Agentic RAG tailored for Tunisian contexts that enforces a curated XLSX-based bad-word filter and falls back to a Gemini LLM when no retrieval answer exists. Designed to avoid exposing abusive or violent language to young users (ages 10–18). ## 🎯 Features - **Multilingual Document Processing**: Language-aware chunking with RTL text support (Arabic, Hebrew, etc.) - **Intelligent Vector Search**: ChromaDB-powered semantic search with automatic deduplication - **Conversation Memory**: Context-aware responses using semantic similarity and temporal relevance - **Web Scraping**: Automatically extracts and ranks URLs from documents - **Interactive CLI**: User-friendly command-line interface ## 📋 Components ### 1. **MultilingualDocumentProcessor** (`multilingual_processor.py`) - Language detection with `langdetect` - RTL (Right-to-Left) text normalization - Semantic chunking with metadata preservation ### 2. **IntelligentWebScraper** (`web_scraper.py`) - URL extraction from documents - Semantic ranking of URLs based on query relevance - Context-aware web scraping ### 3. **OptimizedVectorStore** (`vector_store.py`) - ChromaDB integration with persistent storage - Automatic document deduplication using MD5 hashing - Metadata indexing for efficient filtering ### 4. **ConversationMemoryEngine** (`memory_engine.py`) - Semantic similarity matching with past conversations - Temporal relevance weighting (exponential decay) - Token-aware context management ### 5. **MultilingualRAGSystem** (`rag_system.py`) - Main orchestrator integrating all components - PDF processing with `pypdf` - Query handling with multi-source retrieval ## 🚀 Installation & Setup ### 1. Activate Virtual Environment ```bash source ~/python/bin/activate ``` ### 2. Install Dependencies All major dependencies are already installed in your environment. To verify: ```bash pip list | grep -E "sentence-transformers|chromadb|langchain|t …