Handling Near Real-Time Sensor Data in a Marine Research Data Infrastructure
Further developing the pan-European infrastructure for marine and ocean data management
National oceanographic data centers and major research institutes from 34 countries have developed the pan-European SeaDataNet infrastructure. It connects more than 100 marine data centers that provide data discovery and access functionality for researchers. Furthermore, SeaDataNet is a key element supporting the European Marine Observation and Data network (EMODnet), which was initiated by the EU DG-MARE. It also complements the Copernicus Marine Environmental Monitoring Service.
In order to provide more efficient and faster data access for researchers, several upgrades and enhancements to reflect recent technological developments are necessary. These comprise aspects, such as new types of sensors, as well as international IT concepts (e.g., cloud infrastructures) and interoperability standards (e.g., INSPIRE guidelines for observation data). Consequently, SeaDataCloud aims to improve the current SeaDataNet infrastructure by adopting cloud and high-performance computing technology. SeaDataCloud cooperates with EUDAT, a network of computing infrastructures that develops and operates a common framework for managing scientific data across Europe.
The SeaDataCloud project output results in improved access to research data. A comprehensive collection of services and tools developed to cover researchers’ specific needs is available to support marine research and enable the generation of added value products.
Approaches to handle near real-time observation data
52°North investigates new approaches to enhance the SeaDataNet infrastructure with capabilities for handling near real-time observation data. This comprises two main tasks: the development of a SWE Ingestion Service and a SWE Viewing Service.
- The SWE Ingestion Service: In order to facilitate the publication of observation data streams, we develop a concept and prototypical implementation of a Web service architecture that comprises a plug-and-play approach for sensor data publication. It includes the description of sensor interfaces, sensor data streams (e.g., delivered through Internet of Things protocols such as MQTT) and other sensor data repositories (e.g., CSC-based data stores). This is complemented by tools for editing such sensor data stream descriptions (including the semantics of the data) and an interpreter relying on the descriptions for pushing the available data into the SeaDataCloud infrastructure.
- The SWE Viewing Service: We explore new approaches for a user-friendly visualization of the previously published sensor data streams. This service covers not only the data display itself, but also new ideas on improving the discoverability of the broad range of published sensor data streams.
In addition, we participate in networking activities to promote and further test the SeaDataCloud developments and provide feedback to international standardization activities.
In 2020, our team further refined the SWE Ingestion Service and the SWE Viewing Service. We also addressed the question of how to improve sensor descriptions to facilitate the (semi-)automatic processing of incoming sensor data streams by formally describing their structure in SensorML.