93Chidiebere/financial-scraper-agent

Scrapes financials from the websites of publicly listed companies in Nigeria # AI Financial Scraper Agent An intelligent, open-source AI agent designed to scrape and extract financial statement tables (Balance Sheet, Income Statement, Cash Flow) directly from the websites of listed companies. Built using **Python**, **Playwright** (for web automation), **pdfplumber** (for table extraction), and the **Groq API** (for LLM reasoning). --- ## 🚀 Features * **Autonomous Web Crawling:** Navigates company websites to locate investor relations pages and download links. * **Robust PDF Downloading:** Combines Playwright download event handlers with a standard `urllib` fallback to handle all types of document servers. * **Automated Table Extraction:** Scans downloaded PDF reports for key financial headers and automatically extracts statement grids into CSV files. * **Think-Act-Observe Loop:** Powered by Groq's tool-calling models (e.g., `llama-3.3-70b-versatile`) to handle varied website structures dynamically. --- ## 🛠️ Setup Instructions ### 1. Clone or Copy the Project Ensure your project files are structured as follows: ```text financial-scraper-agent/ ├── .venv/ # Python virtual environment (ignored by git) ├── downloads/ # Downloaded financial PDFs (ignored by git) ├── extracted_data/ # Extracted financial CSVs (ignored by git) ├── .env # Your private Groq API key (ignored by git) ├── .env.example # Template for environment variables ├── .gitignore # Tells git what to ignore ├── agent.py # Main agent orchestrator ├── browser_tool.py # Browser & download automation tools ├── pdf_extractor_tool.py # PDF table parser ├── README.md # Project documentation └── requirements.txt # Package dependencies ``` ### 2. Set Up Virtual Environment Open PowerShell or Command Prompt in the project folder and run: ```powershell # Create virtual environment python -m venv .venv # Activate virtual environment (Windows PowerSh …