# Data Warehouse Project – Customer Review Analysis of Banks in Morocco
## General Overview
This project aims to centralize, clean, and analyze Google Maps reviews of Moroccan banking branches to extract useful insights for decision-makers: satisfaction trends, recurring themes, branch rankings, etc. The pipeline relies on a modern and automated stack.
---
## Specifications and Context
- **Subject**: Centralize and leverage Google Maps customer reviews for Moroccan banks.
- **Main Objectives**:
1. Analyze customer satisfaction (sentiment analysis)
2. Extract recurring topics/points (topic modeling)
3. Rank branches according to customer perception
4. Provide interactive dashboards for decision-making
- **Deliverables**: Python scripts, Airflow DAGs, DBT models, PostgreSQL schema, Looker Studio dashboards, comprehensive documentation.
---
## Technical Stack and Versions
| Component | Version |
|---------------|-------------|
| Python | 3.8.10 |
| DBT | 1.8.7 |
| Airflow | 2.8.1 |
| NLP | bert, nltk |
| PostgreSQL | >=12 |
| BI | Looker Studio |
---
## Pipeline & Detailed Approach
### 1. Data Collection (Scraping)
- **Main Script**: `airflow/dags/scrapping.py`
- **Tools**: Playwright, BeautifulSoup, pandas
- **Logic**:
- Automated scraping of Google Maps reviews for each bank/city
- Popup handling, scrolling, extraction of branch details and reviews
- Storage of raw data in JSON/CSV in `data/`
- Ability to schedule scraping via an Airflow DAG
### 2. Database Ingestion (Staging)
- **Script**: `scripts/load_to_staging.py`
- **Tools**: pandas, psycopg2
- **Logic**:
- Loading CSV files into the PostgreSQL table `avis_bancaires`
- Dynamic table creation based on CSV columns
- Connection and error handling
### 3. Cleaning and Semantic Enrichment
- **Script**: `scripts/enrich_reviews.py`
- **Tools**: pandas, transformers, nltk, spacy, torch, datasets
- **Logic**:
- Text cleaning (punctu …