General requirements of EISCAT-3D

From
Jump to: navigation, search

Context of general requirements in EISCAT-3D[edit]

Complete report on EISCAT-3D general requirements available at: https://envriplus.manageprojects.com/projects/requirements/notebooks/470/pages/38

Summary of EISCAT-3D general requirements[edit]

Detailed requirements[edit]

EISCAT_3D is a research infrastructure that uses incoherent scatter radar to study the Earth’s ionosphere, contributing to geospatial environmental research. From [1]:

EISCAT_3D aims to establish a system of distributed phased array radars that enable comprehensive three-dimensional vector observations of the atmosphere and ionosphere above Northern Fenno-Scandinavia, a unique location for research into the polar atmosphere. The use of new radar technology, combined with the latest digital signal processing, will achieve ten times higher temporal and spatial resolution than obtained by present radars while simultaneously offering, for the first time, continuous measurement capabilities. The flexibility of the EISCAT_3D system will allow the study of atmospheric phenomena at both large and small scales unreachable by the present systems. The new system will be implemented for a wide range of users and applications. It will allow studies to close the gap between space research and environmental research. The continuous data coverage will facilitate the inclusion of detailed incoherent scatter radar data into climate and Earth system modelling.

EISCAT_3D is being constructed by EISCAT, the European Incoherent Scatter Scientific Association, which conducts research into the atmosphere and ionosphere at various levels using incoherent scatter radar.


Operation

The EISCAT_3D radar apparatus can run in a number of different ‘modes’ based on the requirements of users, focusing on different regions and altitudes of the ionosphere. High-power modes are run for limited periods on a campaign basis. Specific modes are chosen based on proposals made by researchers. Otherwise, EISCAT_3D runs continuously in low-power mode, providing a general survey of the upper atmosphere.

Time on EISCAT’s instruments is divided among the EISCAT associates—the countries that have contributed funding to EISCAT. The further division of that time for specific projects is decided on a country-by-country basis. This arrangement is expected to be used specifically in the case of EISCAT_3D as well. In general, researchers will interact with the radar system by making formal proposals for having the system operate in a certain mode for a certain period.

Researchers can make requests for additional instruments to be deployed at EISCAT_3D sites. The current thinking is that any data collected via additional instruments deployed at EISCAT sites should be open to EISCAT associates. It may be possible to attach additional receivers to radars on request, but this poses additional data control and management issues. Final decisions will be made by EISCAT on a case-by-case basis.

Technically EISCAT_3D’s national funding is for infrastructure and operations, with data management an additional cost included by necessity if not original intent.


Data and computation

Access to low-level (raw) data is restricted to EISCAT associates—more precisely, to researchers operating in any of the countries that contribute to EISCAT_3D. High-level (interpreted) data is openly accessible to the general public, regardless of location.

The EISCAT_3D operations centre (which collects raw data from EISCAT_3D instruments) stores the full data (low and high level) for three months, after which all high-level data and 1% of low-level data is archived within an associated data infrastructure, with the remaining 99% of low-level data discarded due to practical storage constraints. Users can access operations centre data directly.

Actual data access will be via a web portal, which provides access to data held at both the operations centre and in archives (access to low-level data subject to access rights). The web portal will have a machine interface for programmatic interactions as well as direct user interaction.

EISCAT_3D will use HDF5 for packing data for users. An open radar format is being discussed (within the context of the COOPEUS project) that could be extended for EISCAT_3D; the current format only supports one-dimensional radar data, whereas EISCAT_3D will notably produce three-dimensional radar data.

All high-level data (data interpreted to show selected physical parameters) will be open (high-level data represents around 90% of data use by researchers). Low-level data (raw data extracted from instrumentation) will be restricted to EISCAT_3D associates (currently determined by IP location; more sophisticated alternatives are being considered). The modes at which the EISCAT_3D system will operate according to various proposals submitted by researchers will also be made open (no secret modes).

EISCAT provides analysis software for reducing raw data into physical parameters which is open source, albeit based on Matlab (which requires a license to use). Software for visualisation of low-level data is also made available, and is likewise open source.

EISCAT_3D does not have security or privacy issues in general (data is mostly open, with the rest only lightly restricted), but there is one sensitive issue—the incidental detection of satellites in orbit, not all of which are white-listed for public tracking. It is necessary therefore to screen all data on-site before being made fully available so as to remove traces of sensitive satellite movements.

EISCAT_3D is using a fairly standard set of technologies for general computing requirements. For on-site processing (processing raw data as it is acquired), the use of FPGAs is being examined. In line with other infrastructures (such as LOFAR and the Tier-1 storage used within the Nordic e-Infrastructure Collaboration), dCache is used for data movement by Nordic Tier-1 and is likely to be used for EISCAT_3D as well. Any changes in technology likely to be driven by coordination with other data infrastructures like EUDAT (for example, the adoption of iRODS).


Requirements

The big problems for EISCAT_3D regard the search and storage of its data. Resources for developing routines for adjusting modes of operation in real time are also very limited.

In order to ensure that researchers using EISCAT_3D are better able to do their work, the searching of data needs to be implemented intelligently so as to allow relatively complex queries to be conducted efficiently over the (very large) datasets collected by EISCAT_3D. The ability to define routines for adjusting the mode of operation of the EISCAT_3D radar system, allowing certain features to be discovered in real time, may be of help in allowing EISCAT_3D to achieve its operational goals most effectively.

EISCAT are looking into how persistent identifiers can be best used to support identification and citation of datasets; currently this is being discussed with the EUDAT consortium, which is in the process of developing a generic solution for persistent identification of scientific data.

Some processing is to be done on the infrastructure side—primarily for search and visualisation of data. It is planned for such computing to be performed within the data infrastructure used for storing the EISCAT_3D data archives, based on leasing of infrastructure resources (e.g. high-performance compute and cluster access) by EISCAT as part of operational funding (assuming that the final budget will cover this), and additional leasing of resources by individual research users / research collectives who also make use of these data infrastructures.


Other infrastructures and initiatives

EISCAT are collaborating with EGI to develop a portal of current data, which may provide technology or expertise for the future EISCAT_3D-specific portal. This EGI competence centre portal uses Liferay. A proposal for EUDAT pilot project is expected to be submitted, driving possible integration with EUDAT services.

How EISCAT_3D will interact with GEOSS (particularly in the ENVRI+ context) is being considered; a formal linking framework between the EISCAT_3D data standards and GEOSS standards would likely be required, since the current GEOSS framework has been deemed not wholly suitable for EISCAT’s interests.

EISCAT runs courses on the use of their radar systems every year, mainly focused on scientific rather than technical questions. This will continue specifically for EISCAT_3D as well, with training on how to access its data in particular. Beyond that, there are no plans for more elaborate training beyond some local outreach.


References

  1. EISCAT Scientific Association, April 2014. EISCAT_3D: The next generation international atmosphere and geospace research radar, technical description. Retrieved September 1st 2015 from https://eiscat3d.se/content/eiscat3d-technical-description.

Formalities (who & when)[edit]

Go-between
Paul Martin
RI representative
Ingemar Häggström and Anders Tjulin
Period of requirements collection
Status