MoaazSalter/egypt_drug_database

Scrapes and processes Egypt's registered drug listings from the EDA (Egyptian Drug Authority) portal into a clean, categorized CSV database. # Egypt Drug Database Scrapes Egypt's registered drug listings from the EDA (Egyptian Drug Authority) portal and processes them into a clean, categorized CSV database — covering ~12,000 registered drugs across 33 standardized dosage form categories. ## Overview The EDA portal exposes Egypt's official drug registry through a paginated web table. This project automates full extraction of that table via Selenium, then applies a rule-based classification pipeline to normalize the raw, inconsistent dosage form strings into a consistent, query-friendly schema. The end result is a structured, deduplicated drug database ready for use in downstream applications such as pharmacy systems, medical reference tools, or drug lookup APIs. ## Files `eda_scraper.ipynb` — Paginates through the EDA search portal using Selenium, extracting trade names and raw dosage form strings from each page until all records are collected. Output: `drugs_info.csv`. `process_drugs.ipynb` — Reads `drugs_info.csv` and applies a keyword-based classification pipeline to map free-text dosage descriptions into standardized categories. Also cleans trade name encoding artifacts (e.g. `Â`, `®`), removes duplicates, sorts alphabetically, and writes the final `Final_Drug_DB.csv`. ## How It Works ### Scraping The scraper navigates the EDA portal's paginated GridView table, targeting trade name and dosage form columns by XPath. It handles pagination by locating the next page anchor dynamically and iterates until no further pages exist. Because the portal requires a CAPTCHA before the listing table becomes accessible, the scraper is designed with a manual intervention point between navigation and data collection (see Usage below). ### Classification Pipeline Raw dosage form strings from the EDA portal are noisy and inconsistent — the same drug form may appear as `"Film Coated Tablet"`, `"F.C. Tab."`, or `"Prolonged Release Tablet"`. The classification pipeline normalizes all of these through a priority …