CV Brokered Data Import

Revision as of 19:20, 4 April 2020 by ENVRIwiki (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Importing data from sources other than the acquisition network requires that the import be brokered by the publishing subsystem before data can be delivered into the data curation subsystem.

Brokered Data Import

A virtual laboratory allows researchers to upload new data into a research infrastructure. All requests are validated by the AAAI service via its authorise action interface. The laboratory provides an interface to an external resource (this might take the form of a URI and a preferred data transfer protocol) and submits a request to a data broker in the data publishing subsystem via its data request interface. The data broker will translate any valid requests into actions; in this scenario, a data transfer request is sent to the data transfer service within the data curation subsystem.

The data transfer service will configure and deploy a data importer, the importer will open a data-flow from an external resource to one or more suitable data stores within the infrastructure and update records within those stores as appropriate. The importer is responsible for the annotation and registration of imported datasets – this generally entails obtaining a global persistent identifier for any new datasets and updating the catalogues used by the research infrastructure to identify and sort its data inventory. As such, the importer can invoke the catalogue service to update catalogues and invoke any community-used PID service to acquire identifiers.