Handling Near Real-Time Sensor Data in a Marine Research Data Infrastructure
Further developing the pan-European infrastructure for marine and ocean data management
The pan-European SeaDataNet infrastructure has been developed by national oceanographic data centers and major research institutes from 34 countries. It connects more than 100 marine data centers providing data discovery and access functionality for researchers. Furthermore, SeaDataNet is a key element supporting the European Marine Observation and Data network (EMODnet), initiated by the EU DG-MARE. It also complements the Copernicus Marine Environmental Monitoring Service.
In order to provide more efficient and faster data access for researchers, several upgrades and enhancements to reflect recent technological developments are needed. These comprise aspects, such as new types of sensors, as well as international IT concepts (e.g. cloud infrastructures) and interoperability standards (e.g. INSPIRE guidelines for observation data). Consequently, SeaDataCloud aims to improve the current SeaDataNet infrastructure by adopting cloud and high performance computing technology. SeaDataCloud cooperates with EUDAT, a network of computing infrastructures that develop and operate a common framework for managing scientific data across Europe.
The SeaDataCloud project output will result in improved access to research data. A comprehensive collection of services and tools developed to cover researchers’ specific needs will be available to support marine research and enable the generation of added value products.
Approaches to handle near real-time observation data
52°North investigates new approaches to enhancing the SeaDataNet infrastructure with capabilities to handle near real-time observation data. This comprises two main tasks:
- The SWE Ingestion Service: In order to facilitate the publication of observation data streams, 52°North worked on a concept and prototypical implementation of a Web service architecture that comprises a plug-and-play approach for sensor data publication. It includes the description of sensor interfaces, sensor data streams (e.g. delivered through Internet of Things protocols such as MQTT) and other sensor data repositories (e.g. CSC-based data stores) as a first step. This is complemented by tools for editing such sensor data stream descriptions (including the semantics of the data) and an interpreter relying on the sensor data stream descriptions for pushing the available data into the SeaDataCloud infrastructure.
- The SWE Viewing Service: 52°North explores new approaches for visualizing the previously published sensor data streams in a user-friendly manner. This covers not only the display of the data itself, but also new ideas on improving the discoverability of the broad range of published sensor data streams.
In addition, 52°North is involved in networking activities to promote and further test the SeaDataCloud developments and provide feedback to international standardization activities.
Over the course of the year, the focus of work shifted from the SWE Ingestion Service prototype to the development and enhancement of the Helgoland Sensor Web Viewer. 52°North improved the general usability of the Helgoland Sensor Web Viewer by enhancing data export functionalities, improving sensor metadata handling (SensorML), and providing easier access to datasets beyond time series data (i.e. trajectories collected by research vessels and profile measurements).