Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
saeon_netcdf_guidance [2022/07/11 15:00] – lindsay | saeon_netcdf_guidance [2022/07/28 12:23] (current) – lindsay | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== Multidimensional Data Guidance Overview | + | ===== Multidimensional Data Guidance Overview ===== |
The SAEON Open Data Platform allows for the storage, discoverability and download of multidimensional scientific data. Network Common Data Format (netCDF) is a file format designed for storing multidimensional data in the form of arrays, and is widely used in the atmospheric and oceanographic communities. The netCDF library has been developed by the [[https:// | The SAEON Open Data Platform allows for the storage, discoverability and download of multidimensional scientific data. Network Common Data Format (netCDF) is a file format designed for storing multidimensional data in the form of arrays, and is widely used in the atmospheric and oceanographic communities. The netCDF library has been developed by the [[https:// | ||
- | ===== NetCDF File Structure | + | ==== NetCDF File Structure ==== |
There are certain types of discrete sampling in atmospheric/ | There are certain types of discrete sampling in atmospheric/ | ||
- | ==== Global Attributes | + | === Global Attributes === |
This section contains metadata that pertains to the overall netCDF file. The following table contains global attributes derived from CF and ACDD conventions required by the SAEON ODP for multidimensional data in netCDF format. For additional resources you can refer directly to the official documentations of each convention. | This section contains metadata that pertains to the overall netCDF file. The following table contains global attributes derived from CF and ACDD conventions required by the SAEON ODP for multidimensional data in netCDF format. For additional resources you can refer directly to the official documentations of each convention. | ||
Line 52: | Line 52: | ||
| lineage (RECOMMENDED) | | lineage (RECOMMENDED) | ||
- | ==== Dimensions | + | === Dimensions === |
NetCDF dimensions define the shape, grid or coordinate system of a variable. A dimension has both a name and a length. A dimension length is an arbitrary positive integer, except for cases where one dimension, at most, is UNLIMITED and can grow along that dimension. Once the data feature type has been identified, the US National Oceanographic Data Center [[https:// | NetCDF dimensions define the shape, grid or coordinate system of a variable. A dimension has both a name and a length. A dimension length is an arbitrary positive integer, except for cases where one dimension, at most, is UNLIMITED and can grow along that dimension. Once the data feature type has been identified, the US National Oceanographic Data Center [[https:// | ||
Line 60: | Line 60: | ||
Something to keep in mind in terms of naming dimensions and variables is the netCDF coordinate variable. This is when a variable has the exact same name as a dimension and is most commonly used for coordinates such as lat, lon, depth and/or time. This may not necessarily have special meaning to the netCDF library, however visualisation software treats this in a special way. | Something to keep in mind in terms of naming dimensions and variables is the netCDF coordinate variable. This is when a variable has the exact same name as a dimension and is most commonly used for coordinates such as lat, lon, depth and/or time. This may not necessarily have special meaning to the netCDF library, however visualisation software treats this in a special way. | ||
- | ==== Variables | + | === Variables === |
Variables in a netCDF contain the parameters measured by an instrument. Variables in netCDF files can be one of six types (char, byte, short, int, float or double). Each variable has a data type, name and a shape which is defined by its specified dimensions. | Variables in a netCDF contain the parameters measured by an instrument. Variables in netCDF files can be one of six types (char, byte, short, int, float or double). Each variable has a data type, name and a shape which is defined by its specified dimensions. | ||
Line 70: | Line 70: | ||
Note: The // | Note: The // | ||
- | ==== Variable Attributes | + | === Variable Attributes === |
- | === Time coordinates | + | == Time coordinates == |
Time data in a netCDF is represented as an integer representing an interval from some reference time. Time variables must include a clear units attribute, as there is no default value. The units attribute for a time variable is a string value in the format recommended by UDUNITS. Commonly these strings include “// | Time data in a netCDF is represented as an integer representing an interval from some reference time. Time variables must include a clear units attribute, as there is no default value. The units attribute for a time variable is a string value in the format recommended by UDUNITS. Commonly these strings include “// | ||
Line 90: | Line 90: | ||
| comment (RECOMMENDED) | “....” | Miscellaneous information about the data, that cannot be described in any of the other available attributes. | | | comment (RECOMMENDED) | “....” | Miscellaneous information about the data, that cannot be described in any of the other available attributes. | | ||
- | === Horizontal coordinates | + | == Horizontal coordinates == |
SAEON ODP uses the WGS84 coordinate reference system in decimal degrees when describing latitude and longitude. | SAEON ODP uses the WGS84 coordinate reference system in decimal degrees when describing latitude and longitude. | ||
Line 107: | Line 107: | ||
| comment (RECOMMENDED) | “....” | Miscellaneous information about the data, that cannot be described in any of the other available attributes. | | | comment (RECOMMENDED) | “....” | Miscellaneous information about the data, that cannot be described in any of the other available attributes. | | ||
- | === Vertical coordinates | + | == Vertical coordinates == |
Depth or height variables should be measured in SI units, most commonly “metres”. Pressure measurements should not be labelled as a depth or height variable, but as a separate parameter. | Depth or height variables should be measured in SI units, most commonly “metres”. Pressure measurements should not be labelled as a depth or height variable, but as a separate parameter. | ||
Line 115: | Line 115: | ||
//All attributes are MANDATORY to include in your netCDF unless stated: CONDITIONAL= include these fields if the information exists, or RECOMMENDED = it is recommended to include these fields but not mandatory.// | //All attributes are MANDATORY to include in your netCDF unless stated: CONDITIONAL= include these fields if the information exists, or RECOMMENDED = it is recommended to include these fields but not mandatory.// | ||
- | | Attribute | Example | Description | | + | | **Attribute** | **Example** | **Description** | |
| standard_name | “depth” | A description of the variable’s content from the CF standard name table. | | | standard_name | “depth” | A description of the variable’s content from the CF standard name table. | | ||
| long_name | “depth” | A descriptive name that indicates a variable’s content. | | | long_name | “depth” | A descriptive name that indicates a variable’s content. | | ||
Line 125: | Line 125: | ||
| comment (RECOMMENDED) | “....” | Miscellaneous information about the data, that cannot be described in any of the other available attributes. | | | comment (RECOMMENDED) | “....” | Miscellaneous information about the data, that cannot be described in any of the other available attributes. | | ||
- | === Geophysical parameters | + | == Geophysical parameters == |
These variables contain the data collected by an instrument or sensor. | These variables contain the data collected by an instrument or sensor. | ||
Line 147: | Line 147: | ||
| comment (RECOMMENDED) | “....” | Miscellaneous information about the data, that cannot be described in any of the other available attributes. | | | comment (RECOMMENDED) | “....” | Miscellaneous information about the data, that cannot be described in any of the other available attributes. | | ||
- | === Quality control flags (CONDITIONAL) | + | == Quality control flags (CONDITIONAL) == |
If applicable to the platform, quality control flags can be self describing variables that show some sort of assessment to identify possible errors in the data. | If applicable to the platform, quality control flags can be self describing variables that show some sort of assessment to identify possible errors in the data. | ||
Line 163: | Line 163: | ||
| comment (RECOMMENDED) | “....” | Miscellaneous information about the data, that cannot be described in any of the other available attributes. | | | comment (RECOMMENDED) | “....” | Miscellaneous information about the data, that cannot be described in any of the other available attributes. | | ||
- | + | === NetCDF manipulation tools === | |
- | ==== NetCDF manipulation tools ==== | + | |
* [[https:// | * [[https:// |