ziedbrah-source/monitor-road-traffic-BigData

The Road Monitoring System is a real-time software application that utilizes big data technologies to monitor and analyze vehicular data on Tunisian roads. It provides insights into vehicle locations, movements, and real-time analytics. The system offers an effective solution for monitoring cars conditions and detecting potential issues promptly. # Road Monitoring System The Road Monitoring System is a software application developed for real-time monitoring and analysis of vehicular data on Tunisian roads. It utilizes big data technologies such as Kafka, Flask, MongoDB, Spark (batch and streaming), and Node.js to provide a comprehensive solution for road traffic monitoring. ## Features - Real-time monitoring of vehicle locations and movements on a map. - Real-time alerts for specific events or conditions. - Real-time data analytics, including average alerts and average speed for last X minutes (you chose it). - Batch processing to calculate percentages of different device types for vehicles. - Visualization of batch processing results using Chart.js. ## Technologies Used - Kafka - HDFS ( HADOOP FILE SYSTEM ) - Flask - MongoDB - Spark - Node.js( TypeScript ) - Java ## Flow Description The Road Monitoring System follows the following flow: 1. Vehicle sensors installed in vehicles continuously gather data and send it to Kafka, utilizing the "sensorsData" topic. This ensures a continuous stream of data from multiple vehicles on Tunisian roads (generated accurately by us for presentation purpose for now). 2. A java server receives the data from Kafka. It processes the incoming data and writes it to HDFS (Hadoop Distributed File System). HDFS provides a scalable and fault-tolerant storage solution for large-scale data. 3. Spark Batch Processing takes over and retrieves the data from HDFS. It performs batch processing operations on the data, calculating the percentages of each device type for the vehicles. These percentages provide valuable insights into the distribution of different device types among the vehicles.(on demand by the administrator) 4. The monitoring server, built using Flask, fetches the batch processing results from Spark Batch Processing. It uses Chart.js to visualize the data in the form of charts and graphs. This allows users to easily understand and interpret the distribution of devi …