93Chidiebere/financial-scraper-agent
Domain:
socioeconomicRecord type:
software
Creator:
93C
Host:
Scrapes financials from the websites of publicly listed companies in Nigeria
# AI Financial Scraper Agent
An intelligent, open-source AI agent designed to scrape and extract financial statement tables (Balance Sheet, Income Statement, Cash Flow) directly from the websites of listed companies.
Built using **Python**, **Playwright** (for web automation), **pdfplumber** (for table extraction), and the **Groq API** (for LLM reasoning).
---
## 🚀 Features
* **Autonomous Web Crawling:** Navigates company websites to locate investor relations pages and download links.
* **Robust PDF Downloading:** Combines Playwright download event handlers with a standard `urllib` fallback to handle all types of document servers.
* **Automated Table Extraction:** Scans downloaded PDF reports for key financial headers and automatically extracts statement grids into CSV files.
* **Think-Act-Observe Loop:** Powered by Groq's tool-calling models (e.g., `llama-3.3-70b-versatile`) to handle varied website structures dynamically.
---
## 🛠️ Setup Instructions
### 1. Clone or Copy the Project
Ensure your project files are structured as follows:
```text
financial-scraper-agent/
├── .venv/ # Python virtual environment (ignored by git)
├── downloads/ # Downloaded financial PDFs (ignored by git)
├── extracted_data/ # Extracted financial CSVs (ignored by git)
├── .env # Your private Groq API key (ignored by git)
├── .env.example # Template for environment variables
├── .gitignore # Tells git what to ignore
├── agent.py # Main agent orchestrator
├── browser_tool.py # Browser & download automation tools
├── pdf_extractor_tool.py # PDF table parser
├── README.md # Project documentation
└── requirements.txt # Package dependencies
```
### 2. Set Up Virtual Environment
Open PowerShell or Command Prompt in the project folder and run:
```powershell
# Create virtual environment
python -m venv .venv
# Activate virtual environment (Windows PowerSh …