Hi Jim,
I recoil away from using the units field for two reasons. First, is that the likes of Steve Emerson have convinced me that unit descriptions shouldn't carry semantics outside SI. Secondly, CF units are coupled to a standard that CF has adopted from elsewhere (UDUNITS) and suspect that the UDUNITS governance might not be too happy about adding an ISO8601 description to their list. If I'm wrong about this then problem solved, but I suspect not. That leaves us with the option of breaking the link with UDUNITS and establishing a units vocabulary under CF control, which has been mooted previously, but thankfully not taken up. I know from experience what CF-type governance can do to a units vocabulary. I inherited one and it includes such horrors as 'mg/g dry sediment' that I'm still struggling to excise from legacy data.
My suggestion for using long name was based on the assumption that an ISO8601 time variable would never be permitted in CF without an accompanying 'time elapsed since event marker' variable, the former being for human interpretation and the latter for machines. Having one with units of seconds and the other dimensionless should allow machines to easily identify what they need. If the variable is for human consumption (covers both readability and preservation of original data use cases) then information delivery through the long name will work. Note, I apologise if the decision to allow ISO8601 alone has been made - but I've sort of lost track of this thread.
Which brings me to the point that this discussion is getting to the stage where we need to be thinking about how we can bring it to a conclusion.
Cheers, Roy.
________________________________
From: Jim Biard [jim.biard at noaa.gov]
Sent: 28 March 2013 14:49
To: cf-metadata at cgd.ucar.edu
Cc: Lowry, Roy K.
Subject: Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?)
Hi.
The format of the string is not what is being described. That can be described by the documentation (be it CF, ISO, or a combination). So what is it that we are trying to describe, apart from questions of format? (Expanding on Chris's previous mention, a user-defined type that was a structure with elements for year, month, day, hour, minute, and second could be used instead of a string.)
It seems to me that we are trying to figure out how to denote that a variable contains a "non-arithmetic" expression of time, similar to "degree minute second hemisphere" representations of latitude and longitude. (Non-arithmetic may be a poor way of expressing what I mean. I'm trying to say that you can't just take two values and add or subtract them in an atomic operation.) You can represent such values in strings, but you can also represent them by packing them into long integers (to millisecond accuracy). The question of whether or not this is a wise thing to do is something else altogether.
I see no reason to exclude the use of the units attribute to denote that the values are expressions of time in which the time since the epoch has been diced up into years, months, days, hours, minutes, and seconds (with varying precision indicated by omission of finer resolution elements). Our current use of the units attribute for time does more than just specify the units (days vs hours, etc). What are the units for such a non-arithmetic time value? They are complex. We could specify something like "years months days" (in the case of a variable that contained dates only), or we could specify something like "datetime". When you went to the units table to find out datetime means, you would find a description.
As far as that goes, I can see a valid argument for declaring a new standard name to use for such variables. If we had a standard name "date" or "datetime", we could use this to differentiate between arithmetic and non-arithmetic time expressions. The units attribute could then express which elements were present in the representation, or such variables could be considered to have no units. We could also specify that variables with a standard name of "date" (for example) must be of string type. (This also has a side benefit - at least to some - of preventing such variables from being used as time axes.)
In all these cases, the calendar attribute is critical to placing the values into a reference frame, and must be included.
Regarding Roy's alternatives, I get serious heartburn when considering 1) and 2). The long name is not supposed to be a place where machines would go to get information about how to interpret the contents of a variable. Everybody seems to want to encroach on it lately. Similarly, the calendar attribute has a specific role, which is to identify the reference frame for the time information. Adding type/units information to this attribute just muddies the water even further.
As far as alternative 3 goes, I have no problem with adding one or more attributes to such variables if it helps clarify something for posterity, but I think we must still resolve what to do with standard name and units for such variables.
Having thought through all of that, I am leaning towards using a standard name of "date" or "datetime" (and use of units, etc as described above) if we are going to add non-arithmetic expressions of time to CF. I would prefer that we stick with the current restriction that the storage format for times be numeric (that is, in essence, what we currently have), and leave the question of representation formats up to other layers, but I understand the desire to have a way to store human-readable dates/times that would be consistent across files.
I've had many headaches maintaining a proprietary legacy software base (not netCDF-related) that didn't separate storage and representation formats because of the amount of code that was needed handle all of the cross-conversions.
Grace and peace,
Jim
Jim Biard
Research Scholar
Cooperative Institute for Climate and Satellites
Remote Sensing and Applications Division
National Climatic Data Center
151 Patton Ave, Asheville, NC 28801-5001
jim.biard at noaa.gov<mailto:jim.biard at noaa.gov>
828-271-4900
On Mar 28, 2013, at 5:48 AM, "Lowry, Roy K." <rkl at bodc.ac.uk<mailto:rkl at bodc.ac.uk>> wrote:
Dear All,
I think Chris has hit the nail on the head here. In my view neither the Standard Name nor the units of measure are the way to describe what is in essence the format of a string. So, what other options are there open to us? I can see three alternatives:
1) Use the long name to describe the string format (not just the standard used but the profile)
2) Use the existing calendar attribute
3) Specify a suitable extension to CF to do the job.
These are roughly in my order of preference.
Cheers, Roy.
________________________________________
From: CF-metadata [cf-metadata-bounces at cgd.ucar.edu<mailto:cf-metadata-bounces at cgd.ucar.edu>] On Behalf Of Chris Barker - NOAA Federal [chris.barker at noaa.gov<mailto:chris.barker at noaa.gov>]
Sent: 27 March 2013 15:56
Cc: cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>
Subject: Re: [CF-metadata] New standard name: datetime_iso8601 (standard_name or units?)
On Wed, Mar 27, 2013 at 8:05 AM, Steve Hankin <steven.c.hankin at noaa.gov<mailto:steven.c.hankin at noaa.gov>> wrote:
ISO date-time strings are a way of encoding the physical quantity
that we know as TIME. So TIME is the "right" standard_name for ISO
date-time strings per the definition quoted above.
Now, it may be that there is a compelling argument to violating the normal
definition of standard_name for the case of ISO date-time strings. Or on
the other hand is it preferable to use the units attribute to indicate the
use of an ISO date-time string?
An ISO string for a datetime is not a name (it's still time), but it
is not a unit either.
What it is is a data type -- more akin to a float or integer -- i.e. a
particular way to translate bytes to a value. The bytes are a char
array, and the value is the datetime itself.
I don't know if thinking about it this way is helpful, as we are
building on netcdf, and I don't now that netcdf allows you to define
new data types, but food for thought.
Also, of course, all the other data types in netcdf (and CF) are
direct translations to commonly used binary formats in computers, and
this one is not.
hmm -- a quick peak at the netcdf4 docs says:
"The richer enhanced model supports user-defined types and data structures"
So maybe this could be a user defined type?
Having said that, I don't support using ISO strings to define
datetimes in CF. I understand particular use-cases, like keeping the
original time stamp from a data collection system and the like, but
then maybe it's really just arbitrary auxiliary text information, in
which case maybe we don't need a standard name or custom data types at
all.
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker at noaa.gov<mailto:Chris.Barker at noaa.gov>
_______________________________________________
CF-metadata mailing list
CF-metadata at cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system.
_______________________________________________
CF-metadata mailing list
CF-metadata at cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
________________________________
This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20130328/33f9ba44/attachment-0001.html>
Received on Thu Mar 28 2013 - 11:15:54 GMT
