JosiahJoseph1777/autochek-cars-scraper
Record type:
software
Creator:
Jos
Host:
This project is a web scraper designed to extract car prices and related data from autochek africa website. The scraper automates the data collection process stores the data incrementally for further analysis. The project includes monitoring and logging to track the scraping process and handle any errors or exceptions.
# Car Scraping Project
## Project Overview
This project is a web scraper designed to extract car prices and related data from the Autochek Africa website. The scraper automates the data collection process, runs daily at 1 AM local time, and stores the data incrementally for further analysis. The project includes monitoring and logging to track the scraping process and handle any errors or exceptions.
## Table of Contents
- Project Overview
- Setup Instructions
- Prerequisites
- Virtual Environment Setup
- Installing Dependencies
- Usage Instructions
- Running `cron_job.py` Manually
- Setting Up the Batch File and Scheduled Task
- Scheduling Details
- Configuring Windows Task Scheduler
- Monitoring and Alerts
- Logging
- Dependencies
- Python Packages
- System Dependencies
- Troubleshooting
- Common Issues and Solutions
- Contributing
- License
---
## Setup Instructions
### Prerequisites
- **Operating System**: Windows 10 or later
- **Python**: Version 3.7 or later
- **Google Chrome Browser**: Latest version
- **ChromeDriver**: Corresponding version to your Chrome browser
### Virtual Environment Setup
1. **Clone the Repository**
```bash
git clone
```
2. **Navigate to the Project Directory**
```bash
cd car-scraping-project
```
3. **Create a Virtual Environment**
```bash
python -m venv venv
```
4. **Activate the Virtual Environment**
```batch
venv\Scripts\activate
```
### Installing Dependencies
Install the required Python packages using `pip`:
```bash
pip install -r requirements.txt
```
---
## Usage Instructions
### Running `cron_job.py` Manually
1. **Activate the Virtual Environment**
```batch
venv\Scripts\activate
```
2. **Navigate to the Project Directory**
```batch
cd path\to\car-scraping-project
```
3. **Run the Script**
```bash
python cron_job.py
```
4. **Monitor Output**
- The script logs its progress to `scraping.log` and `cron_job.log`.
- Scraped data is saved to `car_data.csv`.
### Setting Up the Batch File and Scheduled Task …