Jonathan,
We are definitely getting much closer to full agreement. I continue to
think that a separate time_system (or some such) attribute would be a
much better way to handle this than modifying the calendar attribute,
and that space-separated calendar modifiers would be next best after
that, but I will bow to the apparent majority and agree to your proposal
for modified definitions of the general reference time and Gregorian
calendar, the addition of a new gregorian_utc calendar, etc, as you just
outlined.
There are two remaining things that I would like to see.
1. A section that explains the importance for data producers and
consumers of using the right time handling routines for the input
time data and the calendar chosen if your time resolution makes you
sensitive to errors on the order of 1-30 seconds, pointing out (for
example) that using the standard *nix time functions with sets of
correct UTC timestamps will produce incorrect elapsed time values
under the right conditions. Such a section would also point out to
data consumers that if errors on this level are significant to them,
they shouldn't assume that existing observational datasets are free
of these errors.
2. A standard mechanism for data producers to indicate that they have,
in fact, taken the extra care with their time calculations,
whichever calendar they may have chosen - that is, the elapsed time
values are guaranteed to be free of any leap second related offsets
or discontinuities. This would give data consumers greater
confidence in cases where such errors matter.
Grace and peace,
Jim
On 5/14/15 11:49 AM, Jonathan Gregory wrote:
> Dear Jim
>
> I think I completely agree with what you have written here! Thanks. Therefore
> I wonder what we are discussing now, and like Chris I'm unsure of the status
> of the proposal. So I will take the liberty of repeating the essential
> points of what I have suggested before, and if you disagree with it again, in
> the light of this new understanding maybe we will both understand why :-)
>
> * Don't mention "UTC" in the description of time units. Instead say the time
> zone of the Greenwich meridian, without summmer/daylight-saving time.
>
> * Clarify that in the CF convention the choice of "calendar" implies the
> particular set of rules that is used to convert between date-times (YYYY-MM-DD
> hh:mm:ss i.e. sets of six numbers) and time coordinates in units of elapsed
> time since a reference time. The calendar is identified by the calendar att of
> of the time coordinate variable. It's a property of the time coord variable.
>
> * Require the calendar to be specified i.e. no default, and abolish the
> "standard" calendar (currently a synonym for the default). These are backward-
> incompatible changes for data-writers, but of course they do not invalidate any
> data written with old CF versions.
>
> * Redefine the "gregorian" calendar to mean that leap seconds are not included
> in the time conversions. All days have 86400 seconds in this calendar. This
> might not be correct for some existing data written with calendar="gregorian"
> but we cannot know. For new data, writers should take care they choose the
> right calendar, and that's the reason for not allowing a default, so they have
> to consider the issue.
>
> * Introduce a new calendar "gregorian_utc" which does include leap seconds
> in the time conversions according to UTC.
>
> I've omitted the Julian/Gregorian issues. If we can agree on the UTC issue
> first, we can come back to that.
>
> What do you and all think?
>
> Best wishes
>
> Jonathan
>
>
> ----- Forwarded message from Jim Biard <jbiard at cicsnc.org> -----
>
>> Date: Wed, 13 May 2015 13:35:17 -0400
>> From: Jim Biard <jbiard at cicsnc.org>
>> To: cf-metadata at cgd.ucar.edu
>> Subject: Re: [CF-metadata] How to define time coordinate in GPS?
>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0)
>> Gecko/20100101 Thunderbird/31.6.0
>>
>> Jonathan,
>>
>> I'd say this is a lot of where the misunderstanding lies.
>>
>> I agree that if the calendar and time system are specified as
>> Gregorian/NLS, then you should not involve leap seconds in your
>> calculations going between times represented as HH:MM:SS and elapsed
>> times since your epoch. But if the calendar and time system are
>> specified as Gregorian/UTC, then you should involve leap seconds in
>> those same calculations. But in either case you *should* end up with
>> a set of elapsed times in your time variable that don't have any
>> leap second offsets or discontinuities.
>>
>> Let's say I have a machine that counts the number of total seconds
>> and 86,400 second periods (standard days) that have gone by since I
>> started it, which was on Jan 1, 1970. (A UNIX system with a
>> super-accurate CPU clock.) When I have a leap day, I don't add or
>> subtract anything from the counts the machine has. What I do is use
>> the algorithms specified by the Gregorian calendar to represent
>> February as having one more day than usual so that my YYYY-MM-DD
>> calendar date stays more closely aligned with Earth's position in
>> its orbit. Leap seconds are just like that. I don't add or subtract
>> anything from the counts my machine has. I use the algorithms
>> specified by the UTC time system to represent June 30 (in a year
>> when I have a leap second) as having one more second in the last
>> minute of the last hour so that my HH:MM:SS clock time stays more
>> closely aligned with Earth's position in its rotation.
>>
>> In the particular case of *nix systems using the Network Time
>> Protocol, getting the system to display the correct UTC time
>> actually is accomplished by subtracting a second from the elapsed
>> time being counted by the system, thus ensuring that any long term
>> recording of elapsed times since the computer was started does, in
>> fact, include leap second discontinuities. This can add a whole
>> other facet to the general problem of how to get all of this right
>> as a data producer if I'm getting my time from my *nix machine. I
>> don't know for sure how Windows boxes handle it, but I don't think
>> it's actually any better.
>>
>> Grace and peace,
>>
>> Jim
>>
>> On 5/13/15 12:54 PM, Jonathan Gregory wrote:
>>> Dear Jim
>>>
>>> I think your last email (which didn't go to the email list) explains why we
>>> have not been understanding each other about your points 2 and 3:
>>>
>>>> There shouldn't be (or there is no) good reason to use a different
>>>> calendar to encode and decode, but this is exactly what has been
>>>> done with regard to times. As a netCDF file producer, if you have a
>>>> set of observations that have attached correct Gregorian/UTC date &
>>>> time strings and you use a POSIX calculator to encode your times
>>>> since your correct Gregorian/UTC reference date & time string, then
>>>> you are at risk of producing incorrect elapsed time values. (Whether
>>>> you actually do or not depends on factors mentioned in previous
>>>> emails.) The contents of time variables *should* always be true
>>>> counts of time that has elapsed since the reference epoch and
>>>> *should* never, ever encode leap second induced discontinuities or
>>>> offsets.
>>> That's not how I understand what we've been discussing. I think that if
>>> calendar="gregorian[_nls]" then the encoding/decoding should be done without
>>> leap seconds (86400 s per day), but if calendar="gregorian_utc" the encoding/
>>> decoding should be done with leap seconds according to UTC. This why I see
>>> the issue as being one that relates to the calendar, and not to the units
>>> or the reference time, which are calendar-neutral. Is this what we disagree on?
>>>
>>> I can see there's a difficulty with the UTC calendar that if a second is taken
>>> out there will be two consecutive seconds with the same timestamp. This is
>>> a problem for encoding - the data-writer would have to be careful that he
>>> chose the right one. It's not a problem for using the encoded time coords,
>>> which will be monotonic and run forward at 1 second per second, nor for
>>> decoding, which will correctly produce the repeated timestamp. Inserting a
>>> leap second is the same kind of discontinuity as the change from Julian to
>>> Gregorian calendar. It means that there is a range of date-times which cannot
>>> be encoded because they are not valid, but decoding is no problem. Similarly
>>> there are dates which are valid in some calendars but not others, such as
>>> 30 Feb being valid in the 360_day calendar.
>>>
>>> I think my view is consistent with the general treatment of calendars by CF.
>>> The encoding/decoding algorithm differs, but the same units string could
>>> always be used.
>>>
>>> Best wishes
>>>
>>> Jonathan
>>> _______________________________________________
>>> CF-metadata mailing list
>>> CF-metadata at cgd.ucar.edu
>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>> --
>> CICS-NC <http://www.cicsnc.org/> Visit us on
>> Facebook <http://www.facebook.com/cicsnc> *Jim Biard*
>> *Research Scholar*
>> Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
>> North Carolina State University <http://ncsu.edu/>
>> NOAA National Centers for Environmental Information <http://ncdc.noaa.gov/>
>> /formerly NOAA?s National Climatic Data Center/
>> 151 Patton Ave, Asheville, NC 28801
>> e: jbiard at cicsnc.org <mailto:jbiard at cicsnc.org>
>> o: +1 828 271 4900
>>
>> /We will be updating our social media soon. Follow our current
>> Facebook (NOAA National Climatic Data Center
>> <https://www.facebook.com/NOAANationalClimaticDataCenter> and NOAA
>> National Oceanographic Data Center
>> <https://www.facebook.com/noaa.nodc>) and Twitter (_at_NOAANCDC
>> <https://twitter.com/NOAANCDC> and _at_NOAAOceanData
>> <https://twitter.com/NOAAOceanData>) accounts for the latest
>> information./
>>
>>
>> _______________________________________________
>> CF-metadata mailing list
>> CF-metadata at cgd.ucar.edu
>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>
> ----- End forwarded message -----
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
--
CICS-NC <http://www.cicsnc.org/> Visit us on
Facebook <http://www.facebook.com/cicsnc> *Jim Biard*
*Research Scholar*
Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
North Carolina State University <http://ncsu.edu/>
NOAA National Centers for Environmental Information <http://ncdc.noaa.gov/>
/formerly NOAA?s National Climatic Data Center/
151 Patton Ave, Asheville, NC 28801
e: jbiard at cicsnc.org <mailto:jbiard at cicsnc.org>
o: +1 828 271 4900
/We will be updating our social media soon. Follow our current Facebook
(NOAA National Climatic Data Center
<https://www.facebook.com/NOAANationalClimaticDataCenter> and NOAA
National Oceanographic Data Center <https://www.facebook.com/noaa.nodc>)
and Twitter (_at_NOAANCDC <https://twitter.com/NOAANCDC> and @NOAAOceanData
<https://twitter.com/NOAAOceanData>) accounts for the latest information./
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20150518/802fc6d7/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: CicsLogoTiny.png
Type: image/png
Size: 15784 bytes
Desc: not available
URL: <http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20150518/802fc6d7/attachment-0001.png>
Received on Mon May 18 2015 - 11:35:32 BST
