Please pardon this digression, but is xkcd following this thread?
http://xkcd.com/1514/ - just came out last week? ;-)
Cheers,
David Arctur
?
On Apr 28, 2015, at 4:41 PM, Jim Biard <jbiard at cicsnc.org<mailto:jbiard at cicsnc.org>> wrote:
Jonathan,
As I've mentioned in other emails, I don't think TAI and GPS are calendars, but I understand where you are coming from.
When you are working in the model domain you are generating your own timestamps, so you are self-consistent and have no problems. As far as that goes, there are models that use no_leap or all_leap calendars, and they are again self-consistent and have no problems. In the "real" world, we often start with UTC timestamps that have leap seconds accounted for, yet convert them to elapsed times using calculators that don't account for leap seconds. This can actually lead to elapsed time values that encode a time discontinuity and cannot be counted on to produce accurate differences between every pair of values. I'm not sure what we should call that, but I don't like the idea of naming it with a calendar.
It is true that a large majority of files "in the wild" probably either use time at a resolution that is coarse enough that none of this matters or managed to avoid having any leap seconds creep in to their elapsed time values.
I'm not suggesting that we add anything to the units attribute. I am suggesting that we need to do two things. One is to more precisely define what sorts of times can be used in the time reference part of the units attribute. I just reread section 4.4, and it actually says that the time is UTC or a time zone offset from it. I think it should stay that way and the wording strengthened to make it clearer. Since there are no leap seconds defined before the TAI epoch date, I think there would be no problem with saying that times in years before Jan 1, 1958 are assumed to have no leap seconds. (Call it proleptic UTC if you like.)
The other thing I think we need to do is provide a way to indicate that the elapsed times in a time variable are true elapsed times that are certain to be free of leap second discontinuities, or are possibly contaminated with leap second discontinuities. In connection with this we would need to add clarifying language in the CF conventions to educate people on the importance of using time calculators that are aware of leap seconds when moving between UTC timestamps and elapsed time values. This could be handled by adding a modifier to a calendar name in the calendar attribute, or it could be handled by adding a new attribute to hold this information. I think that coming up with one or more new calendar names is a more confusing and less useful way to accomplish this.
It seems to me that the historical default for existing datasets is that the leap second handling has to be "unknown". For most (maybe all) of the model datasets, the actual situation is "no leap second discontinuities". (Do you actually have any models where time in the output has a resolution fine enough and span wide enough for it to matter anyway?) For many "real world" datasets, the actual situation is "possibly contaminated with leap second discontinuities", since the times started as UTC timestamps and were converted to elapsed times using calculators that didn't consider leap seconds. (But as I mentioned, most of them probably aren't affected by the error.) We have two communities that have different defaults, so we shouldn't pick one over the other.
Grace and peace,
Jim
On 4/28/15 12:52 PM, Jonathan Gregory wrote:
Dear Jim
I agree that TAI and GPS aren't distinct CF calendars if they differ only
because of the epoch. In CF and udunits, the reference time is part of the
the units, as you know; it's not a property of the calendar. I referred to
GPS calendar just to contrast it with UTC.
Unlike you, I don't live only in the real world. :-) Many models use the
default CF calendar (called standard or gregorian) as an approximation to
the real world. In these models there are no leap seconds, and it would not
be correct to call this UTC, or to include the leap seconds in the encoding
and decoding of time cooordinates. In any case, as we understand, most software
doesn't allow for leap seconds. For those two reasons, I propose that we
more precisely define the default (standard, gregorian) calendar to say
explicitly that it does not include leap seconds i.e. every day is 86400 s
long exactly. This is probably correct for most of the data which has been
written with this calendar.
I call UTC a calendar because it affects the encoding, since it implies leap
seconds. You suggest, I think, that the treatment of leap seconds should be
indicated in the reference time in the units string. This doesn't seem quite
right to me because the units doesn't say anything about the encoding. We
use the same format of units string for all calendars. udunits does mention
UTC in respect of the format of this string because of wanting to talk about
time zones, not because of leap seconds. Time zones could be used in models
too (though I don't know of a case). Hence I still propose that we should
regard UTC as a calendar, if it's correct that the leap seconds in use in the
real world are part of the definition of UTC. This means we should define a
a new calendar, which is not the default, in which time units have just the
same format as usual i.e. udunits, but the encoding and decoding of values
is done including leap seconds according to UTC. The same value, with the
same units string, may translate into a different date-time (by a number of
seconds) according to whether the calendar is default (standard, gregorian)
or utc. It wouldn't make sense to use this calendar for dates before the
introduction of UTC, so it should be illegal to do so. We could do this by
prohibiting reference times earlier than the start of UTC and negative values
of time in this calendar.
Maybe I haven't grasped your point properly?
Best wishes
Jonathan
----- Forwarded message from Jim Biard <jbiard at cicsnc.org<mailto:jbiard at cicsnc.org>-----
Date: Tue, 28 Apr 2015 11:35:16 -0400
From: Jim Biard <jbiard at cicsnc.org><mailto:jbiard at cicsnc.org>
To: cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>
Subject: Re: [CF-metadata] How to define time coordinate in GPS?
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0)
Gecko/20100101 Thunderbird/31.6.0
Jonathan,
I see where you are coming from, and there's validity in that line
of thought. Leap seconds represent a finer-grained adjustment to the
overall date/time system being used. I still think it makes good
sense to add a new attribute to declare whether or not leap second
handling was used or strengthen our standards for time variables so
that problems are averted.
>From a human understandability perspective, a calendar attribute of
"GPS" or "UTC" will be somewhat confusing. In my experience, people
don't speak of the UTC calendar, it's UTC time. Further, TAI and GPS
time don't really concern themselves with anything but counts of
seconds since an epoch date & time. People convert the counts to
time stamps for convenience, but they are actually more equivalent
to the Julian Day Number (JDN) than they are to the Gregorian or
Julian calendar. TAI and GPS time have different epochs, and TAI is
more accurate, but they both are running counts of seconds that
aren't tied to the motions of the Earth. As a result, I think that
it's improper to talk about a GPS calendar or TAI calendar.
What is being exposed by this discussion is the reality that any of
us (myself included) have often ignored or been unaware of the fact
that the time calculators (time handling software) we used when
filling our time variables with elapsed times weren't giving us true
counts of seconds since the epochs we wrote into our units
attributes. If you are working at a resolution of days or hours,
this will probably never cause a problem. If you are working at a
resolution of minutes or less and working over a time span of
greater than two years, it may well have caused at least occasional
small problems. If you are working with full-resolution
polar-orbiting satellite data, one second represents ~7 km of
satellite motion, so such errors can produce significant geolocation
errors.
A set of elapsed seconds since a Gregorian/UTC epoch that were
calculated from Gregorian/UTC time stamps without regard for leap
seconds and which crossed a leap second boundary are not "GPS"
seconds. Nor are they "UTC" seconds. They are, strictly speaking,
elapsed times into which one or more step errors have been
introduced. As I mentioned in a previous email, as long as you use
the same time calculator to extract time stamps as you did to get
elapsed times from input time stamps, you won't notice anything. You
may notice a problem if you are taking differences between elapsed
times and a leap second boundary gets involved.
As I've considered all of this more, I'm tending to favor the second
option I suggested.
We could also be more strict, and say the epoch time stamp in the units
attribute must always be in UTC. The question would then be reduced to
whether or not leap seconds were counted into the elapsed times stored
in the time variable. In this case, we could add a "leap_seconds"
attribute which would have a value of "UTC" if UTC leap seconds were
counted into the elapsed times, and "none" if not. This would also allow
for some other system of leap seconds to be used. (I don't know if there
are others.) For backward compatibility, considering history, the
default value for this attribute would probably be "UTC".
Clearly, having epoch time stamps with time zone offsets from UTC,
as described in the CF conventions, would be OK as well. I'm also
open to other namings for the new attribute and for its possible
values. The leap seconds only become an issue in certain rather
specific instances, so I think that such an attribute, along with a
bit of discussion in the document, would likely be sufficient to
warn those people that may find themselves negatively affected by
improper leap second handling.
Grace and peace,
Jim
On 4/27/15 1:07 PM, Jonathan Gregory wrote:
Dear Jim
I don't think calendars are the right place to encode this. We could
add a new "time_system" attribute where you would declare whether
your time stamps and elapsed times were based on UTC, GPS, TAI, etc.
If we take this route, we should require the elapsed times to encode
leap seconds if the time system is UTC, and state that the default
time system is UTC.
I think this is a calendar issue because the calendar is the set of rules
which translate between components of time (YMDhms) and elapsed time (in
fixed time units) since the reference time. Your later email seems to me
to be consistent with that. In the real world, the elapsed interval (expressed
e.g. as the number of seconds) between the ref YMDhms and the actual YMDhs
depends on whether your calendar includes leap seconds (UTC) or not (GPS).
It seems that GPS is the calendar likely to have been assumed in existing
CF datasets, so it would be logical to say that the default is the real-
world calendar without leap seconds. Have I misunderstood something? If we
regard this as a property of the calendar, we don't need a new attribute.
Best wishes
Jonathan
_______________________________________________
CF-metadata mailing list
CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
--
<CicsLogoTiny.png> <http://www.cicsnc.org/> Visit us on
Facebook <http://www.facebook.com/cicsnc> Jim Biard
Research Scholar
Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
North Carolina State University <http://ncsu.edu/>
NOAA National Centers for Environmental Information <http://ncdc.noaa.gov/>
formerly NOAA?s National Climatic Data Center
151 Patton Ave, Asheville, NC 28801
e: jbiard at cicsnc.org<mailto:jbiard at cicsnc.org>
o: +1 828 271 4900
We will be updating our social media soon. Follow our current Facebook (NOAA National Climatic Data Center<https://www.facebook.com/NOAANationalClimaticDataCenter> and NOAA National Oceanographic Data Center<https://www.facebook.com/noaa.nodc>) and Twitter (_at_NOAANCDC<https://twitter.com/NOAANCDC> and @NOAAOceanData<https://twitter.com/NOAAOceanData>) accounts for the latest information.
_______________________________________________
CF-metadata mailing list
CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20150429/589bd35d/attachment-0001.html>
Received on Wed Apr 29 2015 - 07:37:03 BST
