[CF-metadata] CF Conventions and netCDF-4 enhanced model

From: stephen.pascoe at stfc.ac.uk <stephen.pascoe>
Date: Fri, 12 Sep 2014 09:04:05 +0000

Hi all,


I think the CF convention document urgently needs updating to acknowledge the existence of NetCDF4 features and take a stance on the interpretation of CF within a NetCDF4 file. I see this as a first step before we consider whether to use NetCDF4 features explicitly in the CF model. At the BADC data is being provided, sometimes unknowingly, in NetCDF4 which is not strictly in the classic model but is thought to be CF compliant. We need to clarify what CF compliance means in these case.

I suggest a few concrete things to clarify in the spec.
1. Clarify that, in the context of NetCDF4, a "NetCDF4 Dataset" doesn't have to be a file. It can be a group within a file.

2. Clarify the scope of coordinate/auxiliary variables applicable to a data variable within a hierarchal file -- John's suggestion that variables in the current group or a parent could be considered as in scope sounds reasonable.

3. Acknowledge the existence of the String type. In my view the spec should make clear that the type of string attributes in CF can be [char], String or [String]. What is the interpretation of [String] when length>1?

These 3 things still keep the CF model essentially flat whilst explaining how it should map onto the hierarchal NetCDF4 model with a string type. This would maintain backward compatibility for the time being.

I'm happy to participate in the discussion, although I'll probably find it difficult to keep up.

Cheers,
Stephen.

---
Stephen Pascoe  +44 (0)1235 445980
Centre of Environmental Data Archival
STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
From: Russ Rew [mailto:russ at unidata.ucar.edu]
Sent: 12 September 2014 04:55
To: John Caron
Cc: cf-metadata at cgd.ucar.edu
Subject: Re: [CF-metadata] CF Conventions and netCDF-4 enhanced model
I'd also like to participate in a working group developing updated CF conventions that take advantage of the netCDF-4 enhanced data model.
On Wed, Sep 10, 2014 at 12:34 PM, John Caron <caron at ucar.edu<mailto:caron at ucar.edu>> wrote:
Hi Karl and all:
NetCDF-4 compression and chunking are transparent to the user, and are compatible with the "classic data model".
I think we should be gathering experiences with  the enhanced data model, and start a CF-2.X convention draft document that uses the enhanced model. It would also be a good time to remove deprecated features and in general not require backwards compatibility. Perhaps if there are 5-6 people we could start a working group to discuss.
John
On Wed, Sep 10, 2014 at 10:35 AM, Corey Bettenhausen <corey.bettenhausen at ssaihq.com<mailto:corey.bettenhausen at ssaihq.com>> wrote:
Tim,
There was a discussion of this last year. See the archives:
http://mailman.cgd.ucar.edu/pipermail/cf-metadata/2013/author.html
Particularly, the thread "Towards recognizing and exploiting hierarchical groups":
http://mailman.cgd.ucar.edu/pipermail/cf-metadata/2013/056827.html
Cheers,
-Corey
On Sep 10, 2014, at 9:53 AM, Timothy Patterson wrote:
> Is it correct to say that, although they don't explicitly state it, the CF conventions (1.6 and the draft 1.7) restrict compliant netCDF products to be either netCDF-3 or netCDF-4 in classic format? There are no conventions for the enhanced features such as groups and user-defined types like enumerated variables, and Section 2.2, as an example, bars the use of unsigned integer variables or string variables (which are even stated not to exist, again implying classic-model only).
>
> There are some features of the enhanced model we want to use for our future datasets (such as groups) and some features which would make life easier but could be worked around if it led to CF compliance (enumerated types, unsigned integers, string types, etc.). Are there any plans to introduce conventions for the use of these enhanced features at some point in the future or would non-classic model datasets always be seen as non-compliant?
>
> Thanks for your insights on this issue!
>
> Regards,
>
> Tim Patterson
>
>
>
> ---------------------
>
> Dr. Timothy Patterson
> Instrument Data Simulation
> Product Format Specification
>
> EUMETSAT, Eumetsatallee 1, D-64295 Darmstadt, Germany
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
--
Corey Bettenhausen
Science Systems and Applications, Inc
NASA Goddard Space Flight Center
301 614 5383<tel:301%20614%205383>
corey.bettenhausen at ssaihq.com<mailto:corey.bettenhausen at ssaihq.com>
_______________________________________________
CF-metadata mailing list
CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
_______________________________________________
CF-metadata mailing list
CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20140912/5f7902b8/attachment-0001.html>

Received on Fri Sep 12 2014 - 03:04:05 BST