General requirements for IAGOS
Context of general requirements in IAGOS
IAGOS, http://www.iagos.org/, is a new European Research Infrastructure conducting long-term observations of atmospheric composition, aerosol and cloud particles on a global scale. Data collection is carried out via instruments onboard commercial aircraft of cooperating internationally operating airlines.
Summary of IAGOS general requirements
The main purpose of IAGOS information system is to support users to search for metadata then download data with PID which can be cited in publication.
The picture describes the data flow in IAGOS: After an aircraft is landed, data is automatically transferred into the reception server as LOA data. LOA data will be validated. Validation process can be automatically or manually validated by PIs (within 3 days). Validated data (L1 data) and calibrated data (L2 data) will be stored in a centralised database, from where, end-users can access via a web-based data portal. Data levels are as follows:
- LOA: raw data
- LOB: automatic validation
- L1: validated data (by PI)
- L2: Calibrated data
- L3: process/analysed data
- L4: added value data, eg. correlation with satellite data
Raw data are in text file (ASCII), after validation, PIs will provide text files in NASA Ames or netCDF format. There is on-going work on metadata standard, the plan is to use ISO 19115 and align with INSPIRE.
Since 1994, IAGOS already has 2TB data in store, including >45,000 flights. ~ 500 users.
Currently IAGOS involve 6 institutions (from France and Germany), has 6 aircrafts, and aims to have 20 by 2025.
IAGOS computational environments involve self-developed software, such as FLEXPART, a tool for lagrangian transportation model. Database was postgresql but changed now to be MONGO DB. Sometime use Matlab, but prefer to use open-source software. Data is in NASA Ames and netCDF format, metadata use ISO 19115 standard.
All IAGOS data open to access for research purposes. But registration is required. User access is under password control. However this needs to be improved, for example to use certificate-based approach. User will use data, they do not have particular responsibilities. When they register to use IAGOS data they need to agree to cite the project in any publications.
Objectives for IAGOS in ENVRI+ include
- Improve data discovery
- Metadata standardisation
- Citation and DOI management
It would be useful to integrate with other RI's data. For example to share data with ACTRIS and ICOS, e.g., in a map, users can see ACTRIS and ICOS platform. IAGOS is interested in Data discovery, provenance, data identification & citation, data curation & cataloguing, data processing. They expect ENVRI+ to provide services for Citation, Cataloguing, Provenance.
Although amount of IAGOS data is not large, to process data based on user given parameters can be complicated since it may involve many files open/read process. How to improve the performance of data processing/generation is at the moment a big challenge.
There are the maintenance costs, which are mainly supported by AERIS (the French Atmospheric Data Center) which is a collaboration among many French organisation (CNRS, CNES, Meteo-France, etc.)
Formalities (who & when)
|Damien Boularnger <firstname.lastname@example.org>
is the Manager of the IAGOS Information system
Period of requirements collection