Difference between revisions of "IC 3 Support EISCAT-3D users to reprocess data using user's algorithms"

From
Jump to: navigation, search
(Created page with "== <span style="color: #BBCE00">Background</span> == === <span style="color: #BBCE00">Short description</span> === To reprocess environmental data with not default input par...")
 
Line 14: Line 14:
 
|'''Contact email'''
 
|'''Contact email'''
 
|-
 
|-
|
+
|''RI-ICT''
|
+
|''Ingemar Häggström''
|
+
|''EISCAT''
|
+
|[mailto:Ingemar.haggstrom@eiscat.se ''Ingemar.haggstrom@eiscat.se'']
 
|-
 
|-
|
+
|e-Infrastructure_ICT
|
+
|Leonardo Candela
|
+
|''CNR''
|
+
|[mailto:leonardo.candela@isti.cnr.it ''leonardo.candela@isti.cnr.it'']
 
|-
 
|-
|
+
|Task 5.3/7.2 leader
|
+
|Zhiming Zhao<br>Paul Martin
|
+
|University of Amsterdam (UvA)
|
+
|[mailto:z.zhao@uva.nl <span style="color: #222222">z.zhao@uva.nl</span>]<br>[mailto:P.W.Martin@uva.nl <span style="color: #222222">P.W.Martin@uva.nl</span>]
 
|}
 
|}
 +
 +
=== <span style="color: #BBCE00">Use case type</span> ===
 +
 +
''Implementation case''
 +
 +
=== <span style="color: #BBCE00">Scientific domain and communities</span> ===
 +
 +
'''<span style="color: #BBCE00">Scientific domain</span>'''
 +
 +
''Atmosphere''
 +
 +
'''<span style="color: #BBCE00">Community</span>'''
 +
 +
''Data Service Provision'' | ''Data Usage'' [http://confluence.envri.eu:8090/display/ERM/SV+Communities {_}{+}][http://confluence.envri.eu:8090/display/ERM/SV+Communities+_ <span style="color: #222222">http://confluence.envri.eu:8090/display/ERM/SV+Communities+_</span>] for detailed explanation of those communities.>
 +
 +
'''<span style="color: #BBCE00">Behavior</span>'''
 +
 +
== <span style="color: #BBCE00">Detailed description</span> ==
 +
 +
'''<span style="color: #BBCE00">Objective and Impact</span>'''
 +
 +
Most environmental RIs store data in different levels, and between them there is some sort of process. Most users are interested in the derived physical parameters, being the highest level of data. However, these normally depends on default input parameters for the processing of the data. There is often a need to reprocess the basic data with other input parameters, in the simple case to get a different spatial or temporal resolution, for details or statistical reasons. Depending on the data there can be a large number of more parameters to set and possibly alter.
 +
 +
'''<span style="color: #BBCE00">Challenges</span>'''
 +
 +
The challenge is to set up workflows how to select data and the process to do on it.
 +
 +
'''<span style="color: #BBCE00">Detailed scenarios</span>'''
 +
 +
Select data and process. Get a list of the default parameters, modify them as wanted. Run the process with the updated parameter list. This involves staging lower level data, processing it and delivering the results.
 +
 +
'''<span style="color: #BBCE00">Technical status and requirements</span>'''
 +
 +
The data and processes exists as such. One need to provide the basic tools for selection and setting up parameters in a generic way. It's needed to set up storage, several TB, to provide computation capabilities with basic software, here mainly Matlab.
 +
 +
'''<span style="color: #BBCE00">Implementation plan and timetable</span>'''
 +
 +
M0-3
 +
* decide the site for data, data transfer
 +
* discussions and defining workflows/techniques
 +
M3-9
 +
* Setting up the services with interfaces to the data/metadata
 +
M9-12
 +
* Testing, refining
 +
 +
'''<span style="color: #BBCE00">Expected output and evaluation of output</span>'''
 +
 +
A basic portal with human/machine interface, that can deliver updated physical parameters for a restricted dataset.
 +
 +
== <span style="color: #BBCE00">External Links</span> ==
 +
# IC_3 notebook: [https://envriplus.manageprojects.com/projects/wp9-service-validation-and-deployment-1/notebooks/631 {+}][https://envriplus.manageprojects.com/projects/wp9-service-validation-and-deployment-1/notebooks/631+ <span style="color: #222222">https://envriplus.manageprojects.com/projects/wp9-service-validation-and-deployment-1/notebooks/631+</span>]
  
 
[[Category:Use Case List]]
 
[[Category:Use Case List]]

Revision as of 13:31, 18 March 2020

Background

Short description

To reprocess environmental data with not default input parameters

Contact

Background Contact Person Organization Contact email
RI-ICT Ingemar Häggström EISCAT Ingemar.haggstrom@eiscat.se
e-Infrastructure_ICT Leonardo Candela CNR leonardo.candela@isti.cnr.it
Task 5.3/7.2 leader Zhiming Zhao
Paul Martin
University of Amsterdam (UvA) z.zhao@uva.nl
P.W.Martin@uva.nl

Use case type

Implementation case

Scientific domain and communities

Scientific domain

Atmosphere

Community

Data Service Provision | Data Usage {_}{+}http://confluence.envri.eu:8090/display/ERM/SV+Communities+_ for detailed explanation of those communities.>

Behavior

Detailed description

Objective and Impact

Most environmental RIs store data in different levels, and between them there is some sort of process. Most users are interested in the derived physical parameters, being the highest level of data. However, these normally depends on default input parameters for the processing of the data. There is often a need to reprocess the basic data with other input parameters, in the simple case to get a different spatial or temporal resolution, for details or statistical reasons. Depending on the data there can be a large number of more parameters to set and possibly alter.

Challenges

The challenge is to set up workflows how to select data and the process to do on it.

Detailed scenarios

Select data and process. Get a list of the default parameters, modify them as wanted. Run the process with the updated parameter list. This involves staging lower level data, processing it and delivering the results.

Technical status and requirements

The data and processes exists as such. One need to provide the basic tools for selection and setting up parameters in a generic way. It's needed to set up storage, several TB, to provide computation capabilities with basic software, here mainly Matlab.

Implementation plan and timetable

M0-3

  • decide the site for data, data transfer
  • discussions and defining workflows/techniques

M3-9

  • Setting up the services with interfaces to the data/metadata

M9-12

  • Testing, refining

Expected output and evaluation of output

A basic portal with human/machine interface, that can deliver updated physical parameters for a restricted dataset.

External Links

  1. IC_3 notebook: {+}https://envriplus.manageprojects.com/projects/wp9-service-validation-and-deployment-1/notebooks/631+