General requirements for FIXO3

From
Jump to: navigation, search

Context of general requirements in FIXO3

Complete report on general requirements for FIXO3 available at: https://envriplus.manageprojects.com/projects/requirements/notebooks/470/pages/58

Summary of FIXO3 general requirements

Detailed requirements

FixO3 (fixed-point open ocean observatories) is an I3 research project that integrates oceanographic data gathered from a number of ocean observatories and provides open access to that data to academic researchers. From [1]:

The Fixed point Open Ocean Observatory network (FixO3) seeks to integrate European open ocean fixed point observatories and to improve access to these key installations for the broader community. These will provide multidisciplinary observations in all parts of the oceans from the air-sea interface to the deep seafloor … The FixO3 network will provide free and open access to in situ fixed point data of the highest quality. It will provide a strong integrated framework of open ocean facilities in the Atlantic from the Arctic to the Antarctic and throughout the Mediterranean, enabling an integrated, regional and multidisciplinary approach to understand natural and anthropogenic change in the ocean.


Operation

FixO3 is built on largely heterogeneous infrastructure—each institution within the infrastructure operates independently in accordance with its own policies and working practices. Despite this, the basic distributed collection of and access to both data and metadata is already relatively well-realised—the challenge lies with the harmonisation of data formats and protocols, as well as how to provide reliable access to real-time data. FixO3 is working towards open access to all datasets. Currently certain datasets are subject to embargos of different lengths, or are simply unavailable for technical reasons.

Technically speaking, the autonomous nature of the various institutions independently deploying observatories and hosting data means a significant heterogeneity of software and technologies adopted for data management, access and processing.

As an I3 project, FixO3 provides transnational access (TNA), by which academics can apply for usage time on observatories in response to formal calls. Technically access to deployed resources is limited to academia rather than industry, as industry may impose data requirements not acceptable to the principles of open access, but this is not a hard restriction.

EMSO is expected to host moorings of FixO3.

FIXO3 provides some expert training to its community in the form of (so far) two workshops: one aimed at junior researchers on the use of marine data infrastructures to acquire data; the other one hardware-related, concerning marine monitoring hardware and procedures.


Data and computation

Most observatories contribute data to the MyOcean/Copernicus Marine Environment Monitoring Service. Some data is also contributed to EMECO (the European Marine Ecosystem Observatory). Institutions gather data and links to the data are made available online to researchers. Many observatories store their own data independently of any dedicated data infrastructure; each has its own data management, data access services (typically via FTP).

FixO3 uses SWE (sensor web enablement) and SOS (sensor observation service) technologies where appropriate to provide access to sensor data.

A goal of FixO3 is to harmonise data curation and access, while averting the tendency for individual institutions to revert to idiosyncratic working practices after any particular harmonisation project has finished. FixO3 has no plans for infrastructure-side data processing.

Standards used for data include NetCDF and ODV (Ocean Data View). Use of SWE standards is being encouraged. Metadata generally complies with ISO standards and an extended version of Dublin-Core. It is still necessary however to sometime perform internal conversions of data to be hosted on various services, for example from ISO 19139. FixO3 provides open source reformatting software (such as PanFMP) for data not in the desired formats, but this is intended for internal use by individual institutions contributing data.

FixO3 has established a common data policy that all partners should adhere to, and have a metadata catalogue accessible from the website, a standards and service registry, and is linked to GEOSS. The number of possible formats that institutions can use to deliver datasets to the research infrastructure has been restricted, and all contributors need a plan for long term preservation of data.

It is difficult to normalise data management costs—the independent nature of different institutional nodes leads to distinct differences in the characteristics and emphasis given to data management in general. In terms of volume, the data handled by both research infrastructures is not large (in Big Data terms), though certain data streams (such as provided by subsea camera systems) could be significantly larger if all raw data was retained (which is not currently planned).

FixO3 does not collect processing data (from user activities).

Based on the ENVRI prototype, OpenSearch is being considered as a means to access distributed data products. OAI is being used internally for data and for data stored in PANGAEA.

Although the general policy is for data access to be open and free, it is currently the case that Copernicus data is password-protected, so a means to bypass this may be required to support open access.

Generally secured/privileged access to data is not a concern of either project due to their umbrella open access policies, however some accounting of data retrieval is a necessary precondition of some institutions contributing their data, and FixO3 requires a single sign-on process before any data is accessed.

There is a notable overlap between EMSO and FixO3 data (i.e. some FixO3 data is provided within the EMSO infrastructure).

FixO3 requires better mechanisms for ensuring harmonisation of datasets across their distributed networks. Heterogeneous data formats make life difficult for researchers. Improved search is also desirable; currently expert knowledge is required, for example to be able to easily discover data stored in the MyOcean environment.


References:

  1. FixO3—Fixed-point open ocean observatories, November 2013. FixO3 Project Introduction Pack. Retrieved September 16th 2015 from http://www.fixo3.eu/project-introduction-pack/.

Formalities (who & when)

Go-between
Yin Chen, Paul Martin
RI representative
Andree Behnken and Robert Huber
Period of requirements collection
Status