belia-it/facebook-scraper

Facebook Group scraper for Tunisian carpooling posts — FastAPI dashboard + Playwright scraper + SQLite # 🚀 Facebook Group Scraper: Developer Guide This guide provides instructions for setting up, configuring, and running the Facebook Scraper automations. The scraper uses Playwright to intercept GraphQL responses and syncs data to a Google Sheet. --- ## 🏗️ Architecture Overview The following diagram illustrates how the scraper interacts with Facebook and Google Sheets: ```mermaid graph TD A[Cron Job / Manual Run] --> B[Playwright Browser] B --> C{Authentication} C -- Valid Session --> D[Navigate to Group URL] C -- Expired/Modal --> E[Bypass Modal / Manual Refresh] D --> F[Scroll Feed] F --> G[Intercept GraphQL Responses] G --> H[Parse Post Data] H --> I[Check for Uniqueness] I -- New Posts --> J[Gspread API] J --> K[Update Google Sheet] K --> L[Save Refreshed Session] ``` --- ## 🛠️ Prerequisites - **Python 3.8+** - **pip** (Python package manager) - **Google Cloud Account** (for Sheets API) - **VPS** (optional, for 24/7 automation) --- ## 🏗️ 1. Local Installation 1. **Clone the repository** (if applicable) and navigate to the directory: ```bash cd facebook-scraper ``` 2. **Create and activate a virtual environment**: - **Mac/Linux**: ```bash python3 -m venv venv source venv/bin/activate ``` - **Windows (PowerShell)**: ```powershell python -m venv venv .\venv\Scripts\activate ``` 3. **Install dependencies**: ```bash pip install -r requirements.txt ``` 4. **Install Playwright Chromium**: ```bash playwright install chromium ``` --- ## ⚙️ 2. Configuration (`.env`) Create a `.env` file in the root directory with the following variables: ```env GROUP_URL="facebook.com" SHEET_NAME="Your Google Sheet Name" CREDENTIALS_FILE="credentials.json" STORAGE_STATE="facebook_auth.json" TIMEZONE_OFFSET=1 GROQ_API_KEY="your_optional_grok_key" ``` --- ## 📊 3. Google Sheets Setup The scraper uses a Service Account to interact with Google Sheets. 1. Go to the Google Cloud Console. 2. Create a new project. …