Massilia-Mazouzi/RAG-PIL-Thesis

Supplementary code, data, and results for my Master's thesis on an AI assistant for Algerian Patient Information Leaflets. Overview This repository contains the full source code, data, and evaluation results for my Master's thesis project on an AI assistant for Algerian Patient Information Leaflets (PILs) using a Retrieval-Augmented Generation (RAG) architecture. Two RAG systems were implemented and compared: - Naive RAG: Basic chunking and semantic retrieval without filtering. - Enhanced RAG: Metadata-aware retrieval using medication name recognition and targeted filtering. ## Repository Structure : Enhanced_RAG_system/ ┣ Enhanced_RAG.ipynb → Full code notebook for the enhanced RAG system ┣ ragas_dataset_enhanced_RAG.csv → Dataset used in RAGAS evaluation ┗ ragas_results.csv → RAGAS metric results for the enhanced system Naive_RAG_system/ ┣ Naive_RAG.ipynb → Full code notebook for the naive RAG system ┣ ragas_input_data_naive_rag.csv → Input dataset for RAGAS evaluation ┗ ragas_naive_results.csv → RAGAS metric results for the naive system data_scraping.ipynb → Web scraping notebook used to collect medication leaflets (PDFs) from PharmNet-DZ data_preprocessing.ipynb → Preprocessing pipeline notebook: file classification, OCR, Docling conversion test_cases.py → Python script with test queries and expected answers README.md → Project summary, instructions, and structure ## Folder and File Descriptions: 1/ Data Scraping: This notebook contains the complete scraping pipeline used to collect Algerian medication data from PharmNet-DZ. It automatically loops through all letters A–Z, extracts medicine names, corresponding detail pages, and available leaflets links. The resulting dataset is saved as a CSV file, and all available notice PDFs are downloaded and stored in a local folder. This data serves as the foundation for further preprocessing and implementation of the two RAG systems. 2/ Data Preprocessing: This notebook handles the full preprocessing pipeline appli …