The SAEON Open Data Platform (ODP) standard operating procedures (SOPs) document covers practical measures that fall outside the content covered by the SAEON Data Policy and SAEON Preservation Policy.

The conditions of publishing in the SAEON ODP are laid out in the SAEON Data Policy. All projects and data agreements will be filtered through either the SAEON Managing Director, the Data Management Committee or the uLwazi Node Manager. Small data submissions, such as a few datasets from a data provider that falls within SAEON’s collection policy, will be handled by the uLwazi Node’s Curation team.

Prior to data being submitted, data provisioning will be organised based off of the project specifications in the data agreement or from data management plans submitted to the Curation team and sent to the Infrastructure team if additional provisioning is needed.

The steps for making data available in the SAEON ODP are currently executed by members of the data curation team because user registration has not been fully implemented. In future, the system will allow data providers to automatically execute the ingest steps. In the current iteration of the ODP the curation and publication of the data will follow this workflow:

A curator will work with the data provider to receive the Submission Information Package (SIP), either over email or via FTP if the submission is too large to send via email, they will then assess the SIP and ensure that the data is in the correct format, can be opened and is formatted according to Tidy Data principles if it is tabular data and that the data provider has filled in the correct metadata standard and provided enough information for the metadata to be compliant. More information will be requested from the data provider if necessary and the data will be sent back for editing if there are any changes that need to be made to it.

The curator will then make any necessary edits on the metadata, such as adding a DOI if one was requested, adding administrative metadata and small edits such as making the title more descriptive or adding additional keywords. They will also log in to the Repository and upload the SIP, create an Archival Information Package (AIP) and add the link to the data to the metadata. Only curators have permissions to create folders and upload to the Repository.

When the metadata is complete the curator will then convert the record into JSON and log in to the ODP Administrative Interface where they can upload the record and then validate it against its metadata standard, either SAEON DataCite 4.3 or ISO19115 SAEON Profile. If the record does not validate against the standard it cannot be published. A second curator will then quality assure the metadata record, checking the DIP too, before changing its workflow state to Published. Only curators have permissions to upload metadata records to the ODP.

Once the record is published the first curator will then send the data provider the DOI, or link to the metadata record if no DOI was requested.

Minor changes to an existing metadata record, such as adjusting a bounding box or fixing a spelling error can be done in the ODP by a curator if a data provider requests it. Major changes, such as adding an additional creator or changing the lineage statement which would affect how users of the data would either cite or process the data, would require a new version of the metadata record.

Any changes to the data itself will require a new version of that data to be archived and a new metadata record to be produced, which can then be linked through related identifiers to its previous version.

We discourage the removal of archived and published datasets but if a data provider needs to remove a dataset then the metadata record will be updated to indicate that that has happened.