BloomTech-Labs/Sauti-Africa-Market-Monitoring-DS

# Sauti-Africa-Market-Monitoring-DS # data-science TEAM DOCS: - Product Vision Document Github Links: - DATA-SCIENCE - FRONT-END - BACK-END ## **Contributers** |Jesús Caballero |Jing Qian |Taylor Curran | | :-----------------------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------------------: | | | | | | | | | | | | ## **Description** Project description goes here ## **DS Roles** 1. ### Data Engineers + describe the work of the data engineers here 2. ### Machine Learning Engineers + describe the work of the ML engineers here ## Repo Guide - verify_conn.py verify the connection with the database create schema and creates the whole schema of the database. - functions_and_classes.py a script with a handful set of functions used on this project. Global Methodology - It explains the main methodology. ### Data Flow 1. We pull the raw data from the Stakeholder database. There are typos or misspelling words. The first script (aws_collect_data) tries to correct that with the help of dictionaries and lists script. If some products couldn't be corrected, those logs will be dropped in the error logs table. - **Also, we don't manipulate the numerical data, so anyone could try to drop the outliers or numerical typos the better way they considered.** - aws_collect_data.py 2. The second script (split_bc_drop) will try to correct decimal point misplaced, drop outliers, and prices at zero making no sense. It a …