oliftadele/Facebook-post-scrapper-for-Afan-Oromo

Modular Afaan Oromoo Facebook scraper using a 2-stage pipeline for speed. # 🚀 Fast Afaan Oromoo Facebook Scraper A modular, high-speed, and intelligent Facebook scraper designed specifically for extracting **Afaan Oromoo** content. Built with **Python**, **Flask**, and **Playwright**, this tool is optimized for NLP research, sentiment analysis, and dataset building. ## ✨ Key Features - **🌐 Web Interface**: Manage your scraping tasks easily through a modern, responsive web dashboard. - **⚡ Two-Stage Architecture**: Decouples URL discovery from content extraction for maximum speed and reliability. - **🧠 Intelligent Filtering**: Built-in Afaan Oromoo language detection ensures you only collect high-quality, relevant data. - **🛡️ Anti-Detection**: Implements human-like interactions (scrolling, clicking, delays) and custom user agents to stay under the radar. - **💾 NLP-Ready Output**: Saves data in a flat, type-based **JSONL** format, perfect for training machine learning models. - **🔑 Session Management**: Easily import/export Facebook login sessions to avoid repetitive logins and maintain persistency. ## 🏗️ Architecture The project is split into two distinct processes to maximize efficiency: 1. **URL Discovery (Stage 1)**: Rapidly scans Facebook pages/profiles to harvest post URLs. 2. **Data Extraction (Stage 2)**: Iterates through discovered URLs to extract post text and comments, applying language filters in real-time. ## 📊 Data Schema (JSONL) Data is stored in a clean, flat structure: ```json {"type": "post", "post_text": "Oduu gabaasa guutuu...", "url": "facebook.com..."} {"type": "comment", "comment_id": "cmt_a452ec923019", "comment_text": "Baayee nama gaddisiisa"} ``` ## 🚀 Getting Started ### Prerequisites - Python 3.8+ - Node.js (for Playwright browser installation) ### Installation 1. **Clone the repository**: ```bash git clone github.com cd facebook-scrapper ``` 2. **Install dependencies**: ```bash pip install -r requirements.txt ``` 3. **Inst …