⇐ ⇒

[CF-metadata] Groups in netCDF4 files and CF

From: Jim Biard <jim.biard>
Date: Mon, 20 May 2013 11:58:54 -0400

Andreas,

CF has yet to address the new netCDF-4 features. My personal opinion is that CF needs to be extended to embrace the new netCDF-4 features (and I am currently using netCDF-4 with groups), but interoperability / backward compatibility concerns have prevented this. As a result, I'm not surprised to find that PanoplyJ doesn't support your file. I'm not sure, but I think it might be that there is no viewer application that will properly handle your file.

Since there are no applicable CF conventions, what I'm about to say has no authority whatsoever, but I am going to share my thoughts and opinions about file design using groups as a possible step in the direction of developing such conventions.

I think the best way to consider groups within a netCDF-4 file is to treat each one as a file in its own right (with certain extensions). The group concept in netCDF-4 is a form of scope control. Placing coordinates in a different group than measurements would then imply that the two sets of variables are not visible to one another. If there are dimensions or coordinate variables that are common to data variables that have been placed in different groups, they could be reasonably be placed at a common level above the groups (such as the root). Dimensions declared in a group (including the root) are visible to all child groups, so extension to coordinate variables would not be unreasonable.

There is another way to think about groups, which is to consider them as nothing more than ways to make hierarchical names for attributes, dimensions, and variables. In this view, your file contents are effectively

> dimensions:
> lon = 5760 ;
> nv = 2 ;
> lat = 2880 ;
> time = UNLIMITED ; // (204 currently)
> variables:
> double coordinates/lon(lon) ;
> coordinates/lon:standard_name = "longitude" ;
> coordinates/lon:units = "degrees_east" ;
> coordinates/lon:bounds = "coordinates/lon_bnds" ;
> double coordinates/lon_bnds(lon, nv) ;
> double coordinates/lat(lat) ;
> coordinates/lat:standard_name = "lat" ;
> coordinates/lat:units = "degrees_north" ;
> coordinates/lat:bounds = "coordinates/lat_bnds" ;
> double coordinates/lat_bnds(lat, nv) ;
> int coordinates/time(time) ;
> coordinates/time:standard_name = "time" ;
> coordinates/time:units = "hours since 1994-01-01 00:00:00 +0:00" ;
> coordinates/time:calendar = "gregorian" ;
> coordinates/time:bounds = "coordinates/time_bnds" ;
> int coordinates/time_bnds(time, nv) ;
> double data/sciamachy_vcdtrop-no2_v4(time, lon, lat) ;
> data/sciamachy_vcdtrop-no2_v4:_FillValue = NaN ;
> data/sciamachy_vcdtrop-no2_v4:least_significant_digit = 5L ;


(I have used '/' characters to concatenate the group names with the variable names to make the "flattened" names. You could as easily use '_' characters instead.)

There is nothing overtly "wrong" about variables with names such as these. In fact, there is a certain attraction to making the distinction between measurement and coordinate variables clear through what amounts to a naming convention. But (You knew that was coming, didn't you?), I have reservations about a convention that starts by splitting the world into coordinate and measurement domains at the top level. I think this approach will lead to unnecessary limitations.

Grace and peace,

Jim

Jim Biard
Research Scholar
Cooperative Institute for Climate and Satellites
Remote Sensing and Applications Division
National Climatic Data Center
151 Patton Ave, Asheville, NC 28801-5001

jim.biard at noaa.gov
828-271-4900



Follow us on Facebook!

On May 19, 2013, at 11:50 AM, Andreas Hilboll <lists at hilboll.de> wrote:

> Dear Dr. Schmunk, dear CF experts,
>
> I'm wondering if CF concentions say anything about the grouping within a
> netCDF4 file, and if PanoplyJ can handle files with different groups? I
> couldn't find anything in the documentation, but wanted to make sure.
>
> I'm having issues using PanoplyJ
> (http://www.giss.nasa.gov/tools/panoply/) with a file where data and
> coordinate variables are in different groups. For example, the data
> variable of this file cannot be plotted with Panoply:
>
> ---8<-------
> dimensions:
> lon = 5760 ;
> lat = 2880 ;
> time = UNLIMITED ; // (204 currently)
> nv = 2 ;
>
> group: coordinates {
> variables:
> double lon(lon) ;
> lon:standard_name = "longitude" ;
> lon:units = "degrees_east" ;
> lon:bounds = "lon_bnds" ;
> double lon_bnds(lon, nv) ;
> double lat(lat) ;
> lat:standard_name = "lat" ;
> lat:units = "degrees_north" ;
> lat:bounds = "lat_bnds" ;
> double lat_bnds(lat, nv) ;
> int time(time) ;
> time:standard_name = "time" ;
> time:units = "hours since 1994-01-01 00:00:00 +0:00" ;
> time:calendar = "gregorian" ;
> time:bounds = "time_bnds" ;
> int time_bnds(time, nv) ;
> } // group coordinates
>
> group: data {
>
> group: VCDtrop_NO2 {
>
> group: SCIAMACHY {
> variables:
> float sciamachy_vcdtrop-no2_v4(time, lon, lat) ;
> sciamachy_vcdtrop-no2_v4:_FillValue = NaNf ;
> sciamachy_vcdtrop-no2_v4:least_significant_digit = 5L ;
> } // group SCIAMACHY
> } // group VCDtrop_NO2
> } // group data
> }
> ---8<-------
>
> However, when collapsed so that all variables are in the rootgrp, the
> file is plotted correctly:
>
> ---8<-------
> dimensions:
> lon = 5760 ;
> nv = 2 ;
> lat = 2880 ;
> time = UNLIMITED ; // (204 currently)
> variables:
> double lon(lon) ;
> lon:standard_name = "longitude" ;
> lon:units = "degrees_east" ;
> lon:bounds = "lon_bnds" ;
> double lon_bnds(lon, nv) ;
> double lat(lat) ;
> lat:standard_name = "lat" ;
> lat:units = "degrees_north" ;
> lat:bounds = "lat_bnds" ;
> double lat_bnds(lat, nv) ;
> int time(time) ;
> time:standard_name = "time" ;
> time:units = "hours since 1994-01-01 00:00:00 +0:00" ;
> time:calendar = "gregorian" ;
> time:bounds = "time_bnds" ;
> int time_bnds(time, nv) ;
> double sciamachy_vcdtrop-no2_v4(time, lon, lat) ;
> sciamachy_vcdtrop-no2_v4:_FillValue = NaN ;
> sciamachy_vcdtrop-no2_v4:least_significant_digit = 5L ;
> ---8<-------
>
> So my question is: is this a bug in PanoplyJ, or is Panoply doing the
> right thing and I don't understand the CF conventions correctly?
>
> Your help is greatly appreciated!
> Andreas.
>
>
> -- Andreas.
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20130520/b482c11b/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: CicsLogoTiny.png
Type: image/png
Size: 15784 bytes
Desc: not available
URL: <http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20130520/b482c11b/attachment-0001.png>
Received on Mon May 20 2013 - 09:58:54 BST

This archive was generated by hypermail 2.3.0 : Tue Sep 13 2022 - 23:02:41 BST

⇐ ⇒