Sensor Data Management and Stream Data Management: A Conceptual Framework and Its Application in Weather Data Monitoring

With the development of highly intelligent technology and computing, sensors have generated a stream of data that must be effectively collected, processed, and stored to provide an analytical foundation for crucial decisions that affect the global well-being of humans. This study introduces a comprehensive conceptual framework designed to address the challenges associated with these high-velocity data streams and provides an understanding of sensor data management operations like data ingestion, retrieval, queries, storage, and analytics. The focus was on applying atmospheric weather data using the Campbell Scientific weather station to monitor weather parameters. The Atmospheric Weather Station (AWS) used for this study has sensor data and is managed by the Centre for Atmospheric Research, National Space Research and Development Agency, Anyigba. The database designed for this atmospheric data has a twelve-column record where the sensor data has nine columns for the nine weather parameters (rainfall, air temperature, relative humidity, solar radiation, soil volumetric water, soil temperature, wind speed, wind direction, and barometric pressure) and the equipment data has three records. This work also demonstrated that machine learning models are used to train, test, and predict the sensor data collected thereafter, visualize the predicted weather parameters in real time and the results of this prediction would assist both government policymakers and individual decision-makers in planning and socio-economic growth in Nigeria.