dmatekenya/mse-api-assignment

Assignment repository for building a REST API to serve Malawi Stock Exchange data. Extract historical data from PDF reports, store in PostgreSQL, create API endpoints, with optional daily web scraping for real-time updates. Includes starter code and setup instructions. # Malawi Stock Exchange API Assignment This assignment is one of the assessments in the **Data Science Capacity Building (DSCBI) Initiative** course, delivered through a collaboration between AIMS, NISR, and CENFRI in Kigali, Rwanda. For more information about the course, visit the course repository. ## Overview Build a comprehensive REST API to serve Malawi Stock Exchange (MSE) historical data. You'll process PDF reports spanning 2017-present, handle varying data formats, store data in PostgreSQL, and create API endpoints for data access. ## About Malawi Stock Exchange - **Website:** mse.co.mw - **Daily Reports:** The website provides daily reports in PDF format here. - **Listed Companies:** You can find a complete list of all companies (also referred to as counters) listed on the exchange here. Please ensure you collect information for all listed companies by navigating through all pages. ## Timeline (4 Weeks) ### Week 1: Data Extraction & Processing (Sept 22-28) **Focus**: Master PDF-to-CSV pipeline - [ ] Analyze PDF report formats from 2017-present - [ ] Update `mse_pdf2csv.py` to handle all report formats - [ ] Process all historical reports to extract daily price data into CSV files - [ ] Generate clean, standardized CSV files **Deliverable**: Working extraction notebook + processed CSV files. ### Week 2: Database Design & Implementation (Sept 29 - Oct 5) **Focus**: PostgreSQL setup and data management - [ ] Design optimal database schema for historical data - [ ] Create tables and relationships - [ ] Implement data loading mechanisms **Deliverable**: Database with all historical data loaded ### Week 3: API Development (Oct 6-12) **Focus**: Build REST API endpoints - [ ] Implement all 3-5 required API endpoints - [ ] Add input validation using Pydantic models - [ ] Add query parameters for filtering **Deliverable**: Complete working API with all endpoints ### Week 4: Documentation, Testing & Polish (Oct 13-19) **Focus**: Professional finishi …