Randall, are you lurking out there? :-)
On 4/29/15 9:37 AM, Arctur, David K wrote:
> Please pardon this digression, but is xkcd following this thread?
> http://xkcd.com/1514/ - just came out last week? ;-)
>
> Cheers,
> David Arctur
> ?
>
>
> On Apr 28, 2015, at 4:41 PM, Jim Biard <jbiard at cicsnc.org
> <mailto:jbiard at cicsnc.org>> wrote:
>
> Jonathan,
>
> As I've mentioned in other emails, I don't think TAI and GPS are
> calendars, but I understand where you are coming from.
>
> When you are working in the model domain you are generating your own
> timestamps, so you are self-consistent and have no problems. As far as
> that goes, there are models that use no_leap or all_leap calendars,
> and they are again self-consistent and have no problems. In the "real"
> world, we often start with UTC timestamps that have leap seconds
> accounted for, yet convert them to elapsed times using calculators
> that don't account for leap seconds. This can actually lead to elapsed
> time values that encode a time discontinuity and cannot be counted on
> to produce accurate differences between every pair of values. I'm not
> sure what we should call that, but I don't like the idea of naming it
> with a calendar.
>
> It is true that a large majority of files "in the wild" probably
> either use time at a resolution that is coarse enough that none of
> this matters or managed to avoid having any leap seconds creep in to
> their elapsed time values.
>
> I'm not suggesting that we add anything to the units attribute. I am
> suggesting that we need to do two things. One is to more precisely
> define what sorts of times can be used in the time reference part of
> the units attribute. I just reread section 4.4, and it actually says
> that the time is UTC or a time zone offset from it. I think it should
> stay that way and the wording strengthened to make it clearer. Since
> there are no leap seconds defined before the TAI epoch date, I think
> there would be no problem with saying that times in years before Jan
> 1, 1958 are assumed to have no leap seconds. (Call it proleptic UTC if
> you like.)
>
> The other thing I think we need to do is provide a way to indicate
> that the elapsed times in a time variable are true elapsed times that
> are certain to be free of leap second discontinuities, or are possibly
> contaminated with leap second discontinuities. In connection with this
> we would need to add clarifying language in the CF conventions to
> educate people on the importance of using time calculators that are
> aware of leap seconds when moving between UTC timestamps and elapsed
> time values. This could be handled by adding a modifier to a calendar
> name in the calendar attribute, or it could be handled by adding a new
> attribute to hold this information. I think that coming up with one or
> more new calendar names is a more confusing and less useful way to
> accomplish this.
>
> It seems to me that the historical default for existing datasets is
> that the leap second handling has to be "unknown". For most (maybe
> all) of the model datasets, the actual situation is "no leap second
> discontinuities". (Do you actually have any models where time in the
> output has a resolution fine enough and span wide enough for it to
> matter anyway?) For many "real world" datasets, the actual situation
> is "possibly contaminated with leap second discontinuities", since the
> times started as UTC timestamps and were converted to elapsed times
> using calculators that didn't consider leap seconds. (But as I
> mentioned, most of them probably aren't affected by the error.) We
> have two communities that have different defaults, so we shouldn't
> pick one over the other.
>
> Grace and peace,
>
> Jim
>
> On 4/28/15 12:52 PM, Jonathan Gregory wrote:
>> Dear Jim
>>
>> I agree that TAI and GPS aren't distinct CF calendars if they differ only
>> because of the epoch. In CF and udunits, the reference time is part of the
>> the units, as you know; it's not a property of the calendar. I referred to
>> GPS calendar just to contrast it with UTC.
>>
>> Unlike you, I don't live only in the real world. :-) Many models use the
>> default CF calendar (called standard or gregorian) as an approximation to
>> the real world. In these models there are no leap seconds, and it would not
>> be correct to call this UTC, or to include the leap seconds in the encoding
>> and decoding of time cooordinates. In any case, as we understand, most software
>> doesn't allow for leap seconds. For those two reasons, I propose that we
>> more precisely define the default (standard, gregorian) calendar to say
>> explicitly that it does not include leap seconds i.e. every day is 86400 s
>> long exactly. This is probably correct for most of the data which has been
>> written with this calendar.
>>
>> I call UTC a calendar because it affects the encoding, since it implies leap
>> seconds. You suggest, I think, that the treatment of leap seconds should be
>> indicated in the reference time in the units string. This doesn't seem quite
>> right to me because the units doesn't say anything about the encoding. We
>> use the same format of units string for all calendars. udunits does mention
>> UTC in respect of the format of this string because of wanting to talk about
>> time zones, not because of leap seconds. Time zones could be used in models
>> too (though I don't know of a case). Hence I still propose that we should
>> regard UTC as a calendar, if it's correct that the leap seconds in use in the
>> real world are part of the definition of UTC. This means we should define a
>> a new calendar, which is not the default, in which time units have just the
>> same format as usual i.e. udunits, but the encoding and decoding of values
>> is done including leap seconds according to UTC. The same value, with the
>> same units string, may translate into a different date-time (by a number of
>> seconds) according to whether the calendar is default (standard, gregorian)
>> or utc. It wouldn't make sense to use this calendar for dates before the
>> introduction of UTC, so it should be illegal to do so. We could do this by
>> prohibiting reference times earlier than the start of UTC and negative values
>> of time in this calendar.
>>
>> Maybe I haven't grasped your point properly?
>>
>> Best wishes
>>
>> Jonathan
>>
>> ----- Forwarded message from Jim Biard <jbiard at cicsnc.org-----
>>
>> Date: Tue, 28 Apr 2015 11:35:16 -0400
>> From: Jim Biard<jbiard at cicsnc.org>
>> To:cf-metadata at cgd.ucar.edu
>> Subject: Re: [CF-metadata] How to define time coordinate in GPS?
>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0)
>> Gecko/20100101 Thunderbird/31.6.0
>>
>> Jonathan,
>>
>> I see where you are coming from, and there's validity in that line
>> of thought. Leap seconds represent a finer-grained adjustment to the
>> overall date/time system being used. I still think it makes good
>> sense to add a new attribute to declare whether or not leap second
>> handling was used or strengthen our standards for time variables so
>> that problems are averted.
>>
>> From a human understandability perspective, a calendar attribute of
>> "GPS" or "UTC" will be somewhat confusing. In my experience, people
>> don't speak of the UTC calendar, it's UTC time. Further, TAI and GPS
>> time don't really concern themselves with anything but counts of
>> seconds since an epoch date & time. People convert the counts to
>> time stamps for convenience, but they are actually more equivalent
>> to the Julian Day Number (JDN) than they are to the Gregorian or
>> Julian calendar. TAI and GPS time have different epochs, and TAI is
>> more accurate, but they both are running counts of seconds that
>> aren't tied to the motions of the Earth. As a result, I think that
>> it's improper to talk about a GPS calendar or TAI calendar.
>>
>> What is being exposed by this discussion is the reality that any of
>> us (myself included) have often ignored or been unaware of the fact
>> that the time calculators (time handling software) we used when
>> filling our time variables with elapsed times weren't giving us true
>> counts of seconds since the epochs we wrote into our units
>> attributes. If you are working at a resolution of days or hours,
>> this will probably never cause a problem. If you are working at a
>> resolution of minutes or less and working over a time span of
>> greater than two years, it may well have caused at least occasional
>> small problems. If you are working with full-resolution
>> polar-orbiting satellite data, one second represents ~7 km of
>> satellite motion, so such errors can produce significant geolocation
>> errors.
>>
>> A set of elapsed seconds since a Gregorian/UTC epoch that were
>> calculated from Gregorian/UTC time stamps without regard for leap
>> seconds and which crossed a leap second boundary are not "GPS"
>> seconds. Nor are they "UTC" seconds. They are, strictly speaking,
>> elapsed times into which one or more step errors have been
>> introduced. As I mentioned in a previous email, as long as you use
>> the same time calculator to extract time stamps as you did to get
>> elapsed times from input time stamps, you won't notice anything. You
>> may notice a problem if you are taking differences between elapsed
>> times and a leap second boundary gets involved.
>>
>> As I've considered all of this more, I'm tending to favor the second
>> option I suggested.
>>
>> We could also be more strict, and say the epoch time stamp in the units
>> attribute must always be in UTC. The question would then be reduced to
>> whether or not leap seconds were counted into the elapsed times stored
>> in the time variable. In this case, we could add a "leap_seconds"
>> attribute which would have a value of "UTC" if UTC leap seconds were
>> counted into the elapsed times, and "none" if not. This would also allow
>> for some other system of leap seconds to be used. (I don't know if there
>> are others.) For backward compatibility, considering history, the
>> default value for this attribute would probably be "UTC".
>>
>> Clearly, having epoch time stamps with time zone offsets from UTC,
>> as described in the CF conventions, would be OK as well. I'm also
>> open to other namings for the new attribute and for its possible
>> values. The leap seconds only become an issue in certain rather
>> specific instances, so I think that such an attribute, along with a
>> bit of discussion in the document, would likely be sufficient to
>> warn those people that may find themselves negatively affected by
>> improper leap second handling.
>>
>> Grace and peace,
>>
>> Jim
>>
>>
>> On 4/27/15 1:07 PM, Jonathan Gregory wrote:
>>> Dear Jim
>>>
>>>> I don't think calendars are the right place to encode this. We could
>>>> add a new "time_system" attribute where you would declare whether
>>>> your time stamps and elapsed times were based on UTC, GPS, TAI, etc.
>>>> If we take this route, we should require the elapsed times to encode
>>>> leap seconds if the time system is UTC, and state that the default
>>>> time system is UTC.
>>> I think this is a calendar issue because the calendar is the set of rules
>>> which translate between components of time (YMDhms) and elapsed time (in
>>> fixed time units) since the reference time. Your later email seems to me
>>> to be consistent with that. In the real world, the elapsed interval (expressed
>>> e.g. as the number of seconds) between the ref YMDhms and the actual YMDhs
>>> depends on whether your calendar includes leap seconds (UTC) or not (GPS).
>>> It seems that GPS is the calendar likely to have been assumed in existing
>>> CF datasets, so it would be logical to say that the default is the real-
>>> world calendar without leap seconds. Have I misunderstood something? If we
>>> regard this as a property of the calendar, we don't need a new attribute.
>>>
>>> Best wishes
>>>
>>> Jonathan
>>> _______________________________________________
>>> CF-metadata mailing list
>>> CF-metadata at cgd.ucar.edu
>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>
> --
> <CicsLogoTiny.png> <http://www.cicsnc.org/>Visit us on
> Facebook <http://www.facebook.com/cicsnc> *Jim Biard*
> *Research Scholar*
> Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
> North Carolina State University <http://ncsu.edu/>
> NOAA National Centers for Environmental Information
> <http://ncdc.noaa.gov/>
> /formerly NOAA?s National Climatic Data Center/
> 151 Patton Ave, Asheville, NC 28801
> e: jbiard at cicsnc.org <mailto:jbiard at cicsnc.org>
> o: +1 828 271 4900
>
> /We will be updating our social media soon. Follow our current
> Facebook (NOAA National Climatic Data Center
> <https://www.facebook.com/NOAANationalClimaticDataCenter> and NOAA
> National Oceanographic Data Center
> <https://www.facebook.com/noaa.nodc>) and Twitter (_at_NOAANCDC
> <https://twitter.com/NOAANCDC> and _at_NOAAOceanData
> <https://twitter.com/NOAAOceanData>) accounts for the latest
> information./
>
>
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu <mailto:CF-metadata at cgd.ucar.edu>
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>
--
CICS-NC <http://www.cicsnc.org/> Visit us on
Facebook <http://www.facebook.com/cicsnc> *Jim Biard*
*Research Scholar*
Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
North Carolina State University <http://ncsu.edu/>
NOAA National Centers for Environmental Information <http://ncdc.noaa.gov/>
/formerly NOAA?s National Climatic Data Center/
151 Patton Ave, Asheville, NC 28801
e: jbiard at cicsnc.org <mailto:jbiard at cicsnc.org>
o: +1 828 271 4900
/We will be updating our social media soon. Follow our current Facebook
(NOAA National Climatic Data Center
<https://www.facebook.com/NOAANationalClimaticDataCenter> and NOAA
National Oceanographic Data Center <https://www.facebook.com/noaa.nodc>)
and Twitter (_at_NOAANCDC <https://twitter.com/NOAANCDC> and @NOAAOceanData
<https://twitter.com/NOAAOceanData>) accounts for the latest information./
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20150429/a22f1a7e/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: CicsLogoTiny.png
Type: image/png
Size: 15784 bytes
Desc: not available
URL: <http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20150429/a22f1a7e/attachment-0001.png>
Received on Wed Apr 29 2015 - 07:56:07 BST
