AllanKipchumba/scrape-carrefour

Carrefour Kenya Product Scraper # Carrefour Kenya Product Scraper A web scraper that extracts the product catalogue from Carrefour Kenya and outputs structured inventory data ready for use in e-commerce systems. ## What it does - Navigates Carrefour Kenya's website by selected category and sub-category - Discovers all brands within each sub-category - Scrapes every product per brand, paginating through "Load More" until exhausted - Exports data to `.xlsx` files organised by category → sub-category → brand - Aggregates all categories into a single workbook per category in `aggregated_data/` ## Output Each product record contains: | Field | Description | |---|---| | `name` | Product name | | `image_url` | Product image URL | | `price` | Current selling price (KES) | | `original_price` | Pre-discount price (same as price if no discount) | ### Categories covered Alcohol · Baby Products · Bakery · Beauty & Personal Care · Beverages · Bio & Organic Food · Cleaning & Household · Food Cupboard · Fresh Food · Fruits & Vegetables · Health & Fitness · Stationery & School Supplies ## Setup **Requirements:** Python 3.8+, Google Chrome, ChromeDriver matching your Chrome version. ```bash pip install -r requirements.txt ``` Create a `.env` file with your OpenAI key (used by the optional insights summariser): ``` OPENAI_API_KEY=your_key_here ``` ## Usage 1. Open `main.py` and uncomment the categories you want to scrape. 2. Run the scraper: ```bash python main.py ``` 3. Per-brand `.xlsx` files are saved under `data/ / /`. 4. To merge all brand files into one workbook per category: ```bash python aggregate_categories.py ``` Aggregated files land in `aggregated_data/`. ## For resellers The files in `aggregated_data/` are the most convenient starting point — one Excel workbook per product category, each row being a single SKU with its name, image, and price. Import them directly into Shopify, WooCommerce, or any system that accepts a CSV/XLSX product feed. ## Notes - The scraper uses `selenium-s …