saeon_preservation_policy

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
saeon_preservation_policy [2022/07/28 09:21] – [Data Management] leosaeon_preservation_policy [2022/09/16 10:44] (current) – lindsay
Line 19: Line 19:
 **AIP** - Archival Information Package: This is a package containing data and the metadata that describes it. It is created by a data curator from the Submission Information Package (SIP) supplied by the data provider, with the addition of any necessary format migrations or additional information added to the metadata. **AIP** - Archival Information Package: This is a package containing data and the metadata that describes it. It is created by a data curator from the Submission Information Package (SIP) supplied by the data provider, with the addition of any necessary format migrations or additional information added to the metadata.
  
-**DOI** - Digital Object Identifier: A Digital Object Identifier (DOI) is a unique persistent identifier assigned to an object. This links to the metadata record for the object as well as to a digital location, where details about the object can be found. SAEON makes use of DataCite’s DOI system.  +**DOI** - Digital Object Identifier: A Digital Object Identifier (DOI) is a unique persistent identifier assigned to an object. This links to the metadata record for the object as well as to a digital location, where details about the object can be found. SAEON makes use of DataCite’s DOI system.
  
 **Data curators**: The uLwazi team members responsible for the management of data throughout its lifecycle, from ingestion to dissemination and long-term archiving. **Data curators**: The uLwazi team members responsible for the management of data throughout its lifecycle, from ingestion to dissemination and long-term archiving.
  
-**Data providers**: The people and organisations who are submitting data to be archived and published in the SAEON ODP. +**Data providers**: The people and organisations who are submitting data to be archived and published in the SAEON ODP.
  
 **DSI** - Department of Science and Innovation: A South African government department whose mission is to provide leadership, an enabling environment, and resources for science, technology and innovation in support of South Africa’s development. **DSI** - Department of Science and Innovation: A South African government department whose mission is to provide leadership, an enabling environment, and resources for science, technology and innovation in support of South Africa’s development.
Line 33: Line 33:
 **OAIS** - Open Archival Information System Reference Model: SAEON makes use of the Reference Model for an Open Archival Information System (OAIS) , developed by The Consultative Committee for Space Data Systems (CCSDS), as a best-practice standard to work towards. **OAIS** - Open Archival Information System Reference Model: SAEON makes use of the Reference Model for an Open Archival Information System (OAIS) , developed by The Consultative Committee for Space Data Systems (CCSDS), as a best-practice standard to work towards.
  
-**QA** - Quality Assurance: In this context quality assurance is performed by data curators to ensure that the SIPs provided by the data providers contain data that falls within SAEON’s collection policy in an acceptable format and that sufficient metadata has been provided to describe the data. +**QA** - Quality Assurance: In this context quality assurance is performed by data curators to ensure that the SIPs provided by the data providers contain data that falls within SAEON’s collection policy in an acceptable format and that sufficient metadata has been provided to describe the data.
  
-**QC** - Quality Control:  Quality control is performed by data curators to check that all the necessary quality assurance steps were taken. +**QC** - Quality Control:  Quality control is performed by data curators to check that all the necessary quality assurance steps were taken.
  
 **SAEON** - South African Environmental Observation Network: The South African Environmental Observation Network (SAEON) is a business unit of the NRF and serves as a national platform for detecting, translating and predicting environmental change through scientifically designed observation systems and research. SAEON also captures and makes long-term datasets freely accessible, and runs an education outreach programme. SAEON has six nodes dispersed geographically across the country. **SAEON** - South African Environmental Observation Network: The South African Environmental Observation Network (SAEON) is a business unit of the NRF and serves as a national platform for detecting, translating and predicting environmental change through scientifically designed observation systems and research. SAEON also captures and makes long-term datasets freely accessible, and runs an education outreach programme. SAEON has six nodes dispersed geographically across the country.
Line 78: Line 78:
   * File System for managing text files, images, video, audio - for any other digital object or unstructured data.   * File System for managing text files, images, video, audio - for any other digital object or unstructured data.
  
-At this stage the SIP may be reassigned to a data curator with the relevant domain expertise. The purpose of this QA step is to check if any information is needed from the data provider prior to publishing the dataset. If no further data management actions are required, an Archival Information Package (AIP) is generated and passed on to anotherdata curator for Quality Control (QC) and publication.+At this stage the SIP may be reassigned to a data curator with the relevant domain expertise. The purpose of this QA step is to check if any information is needed from the data provider prior to publishing the dataset. If no further data management actions are required, an Archival Information Package (AIP) is generated and passed on to another data curator for Quality Control (QC) and publication.
  
 The Archival Information Package (AIP) generation is initiated by uploading the data in the SIP into the correct data store. If the data are not in the correct format for long-term preservation, additional curation steps such as format migration, further updates to metadata and additional quality assurance are added to the workflow. See Table 1 below for preferred preservation file formats.\\ The Archival Information Package (AIP) generation is initiated by uploading the data in the SIP into the correct data store. If the data are not in the correct format for long-term preservation, additional curation steps such as format migration, further updates to metadata and additional quality assurance are added to the workflow. See Table 1 below for preferred preservation file formats.\\
Line 122: Line 122:
 The version control of the data is conducted through metadata using the DataCite schema, which makes use of related identifiers to link versions that are derived from the AIP, or which provide major or minor versions of the original AIP. Changes in datasets will trigger a major version whereas additional details about the dataset will trigger a minor version. The version control of the data is conducted through metadata using the DataCite schema, which makes use of related identifiers to link versions that are derived from the AIP, or which provide major or minor versions of the original AIP. Changes in datasets will trigger a major version whereas additional details about the dataset will trigger a minor version.
  
-=== Data migration ===+=== Data transformation ===
  
-All Data providers are required to comply with the SAEON Data Policy which grants SAEON staff the necessary rights to migrate data archived in the ODP to new formats when the need arises. For institutional data providers there are Service Level Agreements (SLA) in place that allow for format migration.+All Data providers are required to comply with the SAEON Data Policy which grants SAEON staff the necessary rights to convert data archived in the ODP to new formats when the need arises. For institutional data providers there are Service Level Agreements (SLA) in place that allow for format migration.
  
 === Data retention === === Data retention ===
Line 161: Line 161:
  
 The data users are able to browse and search the metadata records; make use of the data services; and download the metadata, data and supplementary information that is available. There is a user feedback form that allows them to comment on data downloads or provide general feedback. The data users are able to browse and search the metadata records; make use of the data services; and download the metadata, data and supplementary information that is available. There is a user feedback form that allows them to comment on data downloads or provide general feedback.
- 
  
 ==== Continuity of Access to Data Holdings ==== ==== Continuity of Access to Data Holdings ====
  • saeon_preservation_policy.1659000069.txt.gz
  • Last modified: 2022/07/28 09:21
  • by leo