MongoDB-Hadoop Distributed and Scalable Framework for Spatio-Temporal Hazardous Materials Data Warehousing

International audience Today the demand for Carriage of Dangerous Goods is experiencing significant increase on Moroccan market. Each year, more than 15 million tons of dangerous goods are transported by road in Morocco. The transport of dangerous goods is regulated by a legal framework in line with international standards; including the European Agreement concerning the International Carriage of Dangerous Goods by Road (ADR) came into effect in Morocco in June 2003. With the aim to facilitate the deployment of some ADR guidelines, this project offers essential IT solutions for its application at the regional and national scale. In this context, the project involves development of software components for calculating safer time-dependent routes, spatial analysis of Voronoi network and establishment of a decisional database to capture HAZMAT (Hazardous Materials) shipments and occurring incidents or accidents. The framework that we propose assumes three major software components. The first is dedicated to the processing of time-dependent routes that considers risk and traffic conditions. The second is developing the transport network partitioning using Voronoi graph diagrams. This component is used for the purposes of management interventions. Finally, the last component provides a NoSQL database for the storage of HAZMAT events and shipping data. Other supports components are provided for collecting and visualizing of data and spatio-temporal events related to HAZMAT. All given components are integrated into an interoperable software infrastructure respecting intelligent transport systems architecture. This infrastructure is distributed and based on a service-oriented architecture. It is also scalable by integration of MongoDB with Hadoop for large-scale distributed data processing. In this work, we also give an assessment of the performance, scalability and fault-tolerance of using MongoDB with Hadoop, towards the goal of identifying the right architecture and software environment for HAZMAT spatio-temporal data analytics.