⇐ ⇒

[CF-metadata] CF upgrade to netCDF variable names

From: Mike McCann <mccann>
Date: Wed, 15 Jan 2014 11:36:51 -0800

I have two use cases:
MBARI has data from an underwater vehicle that contains hundreds of engineering variables that are automatically logged using the onboard software's names for the variables. Those variables include the '.' character. We tried to use our existing NetCDF TDS/Hyrax infrastructure to handle these data but ran into several frustrating inconsistencies in how various packages handled the '.'. Unfortunately, we are not using the infrastructure for these data.
The ESIP Federation documentation group discussed creating a flattened object serialization convention for hierarchical metadata and wanted to use '.' as a delineator but needed to abandon that consideration to stay CF compliant.
-Mike

--
Mike McCann
Software Engineer
Monterey Bay Aquarium Research Institute
7700 Sandholdt Road
Moss Landing, CA 95039-9644
Voice: 831.775.1769  Fax: 831.775.1736 http://www.mbari.org
On Jan 15, 2014, at 10:28 AM, John Graybeal wrote:
>> Perhaps a reference to the IETF RFC 2119 [1] (which defines these and a few other related terms) should be added to the CF spec
> 
> Yes please, since discussion on this thread has already varied in its understanding/application of those terms.
> 
> The ambiguity in the sentence "No variable or dimension names are standardized by this convention." is also relevant. It could mean "This convention defines no requirements about variable or dimension names." or "This convention does not specify any particular variable or dimension names." The former meaning obviously reinforces the interpretation that 'should' is not a requirement.
> 
> While the arguments pushing for the restrictive naming convention (_ as the only special character) are perhaps not strong, for my own use I don't have a compelling use case on the need for more characters either. Mostly this is a matter of personal taste -- I like being able to use . and - to help with visual parsing and + and _at_ for semantic reasons, and they help reduce the number of likely prefix collisions (which a single separator doesn't help with at all). 
> 
> There is also a social benefit from relaxing the CF almost-standard: on-boarding. We want to encourage netCDF users to transition to CF. Minimizing the number of inconsistencies seems practical and forward-thinking. Forcing a netCDF user (which may include lots of HDF users too, these days) to abandon established attribute names is a significant cost for the affected users, now and going forward.
> 
> John
> 
> 
> On Jan 15, 2014, at 10:00, Ethan Davis <edavis at unidata.ucar.edu> wrote:
> 
>> Hi all,
>> 
>> The use of "should" may, by many, be interpreted as a recommendation
>> rather than as a requirement.
>> 
>> Though the terms "must", "should", and "may" are used throughout the CF
>> spec, I am not finding any text that defines those terms.
>> 
>> Perhaps a reference to the IETF RFC 2119 [1] (which defines these and a
>> few other related terms) should be added to the CF spec. Though it seems
>> that might require a fairly full review of the uses in CF of the terms
>> defined in RFC 2119.
>> 
>> Ethan
>> 
>> [1] http://www.ietf.org/rfc/rfc2119.txt
>> 
>> On 1/15/2014 10:46 AM, Karl Taylor wrote:
>>> All,
>>> 
>>> Yes, that statement seems quite definitive and unambiguous, and for the
>>> reasons stated in other emails, I support retaining it.
>>> 
>>> regards,
>>> Karl
>>> 
>>> On 1/15/14 9:37 AM, Steve Hankin wrote:
>>>> 
>>>> On 1/15/2014 9:24 AM, Jim Biard wrote:
>>>>> Chris,
>>>>> 
>>>>> The point is, the Conventions themselves state that there is *no
>>>>> standard*.  People are all the time trying to add meaning to variable
>>>>> names, but the standard actually states that the meaning is to reside
>>>>> in the attributes.  The variable names are just keys for
>>>>> differentiating the variables.  (I could name all my variables
>>>>> ?vNNNNNNNNNN?, where N is a digit, and I would be completely valid
>>>>> according to the standard.)  The long_name and standard_name
>>>>> attributes are the places where descriptors of the variable content
>>>>> are to be found.
>>>>> 
>>>>> So I?m raising a question. _ Is there actually anything other than
>>>>> sentiment (i.e., an actual rule) that anyone can point to that
>>>>> prevents someone from using ?new? characters in their variable names?_
>>>> 
>>>> How about the lines from the CF document that you cut-pasted (thank you):
>>>> 
>>>>   /Variable, dimension and attribute names should begin with a
>>>>   letter and be composed of letters, digits, and underscores. Note
>>>>   that this is in conformance with the COARDS conventions, but is
>>>>   more restrictive than the netCDF interface which allows use of the
>>>>   hyphen character. The netCDF interface also allows leading
>>>>   underscores in names, but the NUG states that this is reserved for
>>>>   system use./
>>>> 
>>>>   - Steve
>>>>> 
>>>>> Grace and peace,
>>>>> 
>>>>> Jim
>>>>> 
>>>>> CICS-NC <http://www.cicsnc.org/>Visit us on
>>>>> Facebook <http://www.facebook.com/cicsnc> 	*Jim Biard*
>>>>> *Research Scholar*
>>>>> Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
>>>>> North Carolina State University <http://ncsu.edu/>
>>>>> NOAA's National Climatic Data Center <http://ncdc.noaa.gov/>
>>>>> 151 Patton Ave, Asheville, NC 28801
>>>>> e: jbiard at cicsnc.org <mailto:jbiard at cicsnc.org>
>>>>> o: +1 828 271 4900
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> On Jan 15, 2014, at 12:00 PM, Chris Barker <chris.barker at noaa.gov
>>>>> <mailto:chris.barker at noaa.gov>> wrote:
>>>>> 
>>>>>> On Wed, Jan 15, 2014 at 7:39 AM, jbiard <jbiard at mail.cicsnc.org
>>>>>> <mailto:jbiard at mail.cicsnc.org>> wrote:
>>>>>> 
>>>>>>   I don't think we should use ease of mapping variable names to a
>>>>>>   programming language as a reason for allowing (or not allowing)
>>>>>>   any particular character in variable names. 
>>>>>> 
>>>>>> Why not? maybe not a compelling reason, but I can't imagine a
>>>>>> compelling reason to have more flexible naming conventions, either.
>>>>>> 
>>>>>>   CF has, as I understood it, considered variable names as
>>>>>>   completely up to the producer, relying on attributes to provide
>>>>>>   meaning.  So, I can name a temperature variable "fluffy_bunny"
>>>>>>   if I want to, and it is completely valid.
>>>>>> 
>>>>>> valid yes, a good idea? probably not.
>>>>>> 
>>>>>>   Section 1.3 of the Conventions states, "No variable or dimension
>>>>>>   names are standardized by this convention." 
>>>>>> 
>>>>>> so there are no standard variable names -- that's not the same as
>>>>>> standards for variable names....
>>>>>> 
>>>>>> Personally, I wish there were standards for variable names, it would
>>>>>> make it easier to code against -- but that cat's out of the bag. But
>>>>>> this cat isn't: the restiricitons have been there for a long time,
>>>>>> so the question now is:
>>>>>> 
>>>>>> what are the reasons for easing those restrictions?
>>>>>> 
>>>>>> and
>>>>>> 
>>>>>> what are the reasons for keeping those restrictions?
>>>>>> 
>>>>>> we've given a few reasons for keeping them (maybe not all  that
>>>>>> compeling toyou, but reasons none the less) -- what are the reasons
>>>>>> for relaxing them, other than "I like this naming convention that is
>>>>>> currently not allowed" ?
>>>>>> 
>>>>>> I'm not convinced that "fluffy-bunny" is any more readable or
>>>>>> anything else than "fluffy_bunny"
>>>>>> 
>>>>>> -Chris
>>>>>> 
>>>>>> 
>>>>>> -- 
>>>>>> 
>>>>>> Christopher Barker, Ph.D.
>>>>>> Oceanographer
>>>>>> 
>>>>>> Emergency Response Division
>>>>>> NOAA/NOS/OR&R            (206) 526-6959   voice
>>>>>> 7600 Sand Point Way NE   (206) 526-6329   fax
>>>>>> Seattle, WA  98115       (206) 526-6317   main reception
>>>>>> 
>>>>>> Chris.Barker at noaa.gov <mailto:Chris.Barker at noaa.gov>
>>>>>> _______________________________________________
>>>>>> CF-metadata mailing list
>>>>>> CF-metadata at cgd.ucar.edu <mailto:CF-metadata at cgd.ucar.edu>
>>>>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>>>>> 
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> CF-metadata mailing list
>>>>> CF-metadata at cgd.ucar.edu
>>>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> CF-metadata mailing list
>>>> CF-metadata at cgd.ucar.edu
>>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> CF-metadata mailing list
>>> CF-metadata at cgd.ucar.edu
>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>>> 
>> 
>> -- 
>> Ethan Davis                                       UCAR Unidata Program
>> edavis at unidata.ucar.edu                    http://www.unidata.ucar.edu
>> _______________________________________________
>> CF-metadata mailing list
>> CF-metadata at cgd.ucar.edu
>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> 
> ------------------------------------
> John Graybeal
> Marine Data Manager
> 
> M +1 408 675-5445
> skype: graybealski
> Marinexplore
> 920 Stewart Drive
> Sunnyvale 94085
> California, USA
> www.marinexplore.com
> 
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20140115/77a753cd/attachment-0001.html>
Received on Wed Jan 15 2014 - 12:36:51 GMT

This archive was generated by hypermail 2.3.0 : Tue Sep 13 2022 - 23:02:41 BST

⇐ ⇒