Dear Jim,
Yes, there are definitely cases like the one you mention where one
shouldn't use the simple (approx.) gregorian calendar. Any measurements
taken during the leap second would have to be either shifted in time or
dropped. When that would be unacceptable one should use either
greogrian_utc or gregorian_gps (and in converting from wall-clock time
to elapsed time with the utc option, one would have to use a formula
(and look-up table) to include all leap seconds).
best regards,
Karl
On 7/20/15 6:25 AM, Jim Biard wrote:
> Jonathan, Karl,
>
> I'm generally OK with what Karl has said. There is one more twist to
> Karl's gregorian calendar. I think it is definitely an edge case, but
> I feel it is worth noting.
>
> If you have a set of time measurements that start as UTC time stamps
> and span a leap second event (from June 1, 2015 to July 31, 2015 for
> example, since a leap second was added after 2015-06-30 23:59:59), a
> discontinuous shift will be introduced into the time values. The shift
> becomes more noticeable as the measurement precision and frequency
> increase, and becomes most noticeable when the precision is +-1 second
> or better and the frequency is 1/second or higher.
>
> The leap second added at the end of June produced a UTC timestamp of
> 2015-06-31 23:59:60. An elapsed time calculated from that timestamp
> with an algorithm that doesn't recognize leap seconds would have the
> same value as an elapsed time calculated from 2015-07-01 00:00:00. If
> your data had a measurement frequency of 1/second, you would have two
> elapsed times with the same value. This would violate the requirement
> that coordinate variables be monotonic. If your data had a measurement
> frequency of 10/second, you would have 10 elapsed times that repeated
> the previous 10.
>
> If you are measuring once per minute and your precision is +-1 second,
> you would likely be able to find an unhandled leap second event
> through analysis, but it wouldn't likely affect calculations being
> done with the elapsed times. If you are measuring 10 times per second
> and your precision is +-1 microsecond (e.g. satellite data), an
> unhandled leap second event would generate significant errors.
>
> Grace and peace,
>
> Jim
>
> On 7/17/15 9:53 AM, Jonathan Gregory wrote:
>> Dear Karl
>>
>> I agree with your summary of the situation, thank you. This whole discussion
>> began (as the title reminds us) because of a use-case of specifying that GPS
>> time had been used, and not UTC time, so I expect there will be use of these
>> new calendars, but that most data will continue be written with gregorian,
>> which we will define more precisely when we make this change.
>>
>> Best wishes
>>
>> Jonathan
>>
>> ----- Forwarded message from Karl Taylor<taylor13 at llnl.gov> -----
>>
>>> Date: Wed, 15 Jul 2015 13:30:52 -0700
>>> From: Karl Taylor<taylor13 at llnl.gov>
>>> To:cf-metadata at cgd.ucar.edu
>>> Subject: Re: [CF-metadata] How to define time coordinate in GPS?
>>> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:31.0)
>>> Gecko/20100101 Thunderbird/31.7.0
>>>
>>> Hi Jonathan and Jim,
>>>
>>> Thanks for bearing with me as I educate myself and think about this
>>> stuff. I rethought further about what I said yesterday and
>>> realized that maybe we were still making this too complicated.
>>> Jonathan's discussion helped me to come up with the following:
>>>
>>> Starting with the following claims:
>>>
>>> 1) All model data stored under CF has been generated by models with
>>> strict 86400 second mean sidereal days (and no leap seconds are
>>> necessary)
>>>
>>> 2) Up to now, the vast majority of all observational data stored
>>> under CF has had reference times that can be associated with the UTC
>>> calendar (or its predecessor GMT) and the elapsed times have been
>>> recorded omitting leap seconds.
>>>
>>> I make the second claim because, for example, folks doing daily
>>> measurements or even 3-hourly measurements collected for synoptic
>>> weather prediction record that data at certain times of day as
>>> dictated by clocks that follow UTC. For example a measurement might
>>> be done every day at 0Z and the person performing the measurement is
>>> probably not even aware that every once in a while he had to wait 1
>>> second longer than usual because a leap second was inserted. When
>>> these measurements are recorded as time series, I'd be surprised if
>>> anyone has inserted a leap second in computing elapsed time.
>>>
>>> This "sloppiness" in recording elapsed time actually is a blessing
>>> to those of us comparing climate model output to observations. If
>>> we want to compare monthly means, for example, we can specify
>>> exactly the same time bounds for both observed data and model output
>>> and we will extract the data of interest. If instead leap seconds
>>> had been accounted for in calculating elapsed time under the UTC
>>> system, then the time bounds would be different from models,
>>> certainly giving us headaches and likely leading to mistakes.
>>>
>>> I would therefore advocate to interpret "gregorian" as follows:
>>>
>>> This calendar is based on the assumption that the length of the mean
>>> solar day is exactly 86400 seconds. For observations, the
>>> assumption is not in fact exactly correct. To facilitate
>>> comparisons between model results and climate and weather
>>> observations, an adjustment in the true elapsed time is required
>>> when calendar="gregorian". The reference time should be specified
>>> according to the UTC time system, but leap seconds should be omitted
>>> in converting from wall-clock time to elapsed time. This means that
>>> elapsed time values will only be approximately correct, but this is
>>> generally desirable because: 1) simple algorithms can be used to
>>> convert from wall-clock time to elapsed time, 2) the mismatch
>>> between the model's mean solar day and the actual mean solar day
>>> (about .002 seconds) can be ignored in comparing models and
>>> observations; observed elapsed times are effectively modified to
>>> yield a time series that stays synced with the diurnal cycle we
>>> would find if the earth's day were exactly 86400 seconds long. Note
>>> that although elapsed times stored in the CF-conforming file will be
>>> only approximate, they will differ by less than 17 seconds (as of
>>> July 2015) from the true elapsed time.
>>>
>>> I think this definition means that there is no need for gregorian_utc_nls.
>>>
>>> Although I can see why the two new calendars (gregorian_utc and
>>> gregorian_gps) could be used for special cases, I would hope most
>>> providers of climate and weather data will continue to use a
>>> "gregorian" calendar as defined above. This will make it much
>>> easier for modelers to compare their output to observations.
>>>
>>> best regards,
>>> Karl
>>>
>>>
>>> On 7/15/15 1:34 AM, Jonathan Gregory wrote:
>>>> PS. First, to repeat myself:
>>>>
>>>> So my conclusion (at present - but I suspect I haven't understood something
>>>> you have said) is that gregorian is fine and sufficient (with the addition of
>>>> gregorian_utc and gregorian_gps for the well-defined real-world systems) if
>>>> we define it to mean that 86400-seconds are assumed to be used for conversion
>>> >from clock time to elapsed time and elapsed time to clock time, that it is not
>>>> defined whether the reference time and clock times are GPS and UTC, and that
>>>> the elapsed times may not be exactly correct for the real world (due to the
>>>> neglect of leap seconds).
>>>>
>>>> I also said, "This is then the same as gregorian_utc_nls, so we would not need
>>>> that either, which was your final conclusion in your previous post." Actually
>>>> it's not quite the same as gregorian_utc_nls, which asserts that clock times,
>>>> if real-world, are UTC. Otherwise it's the same. If my conclusion above is
>>>> correct, then I don't mind whether we introduce gregorian_utc_nls or not. No-
>>>> one has definitely asked for it, as far as I remember; we discussed it because
>>>> of anticipating the need, I believe. We could therefore follow the usual CF
>>>> principle of omitting it until it's asked for.
>>>>
>>>> Cheers
>>>>
>>>> Jonathan
>>>>
>>>> ----- Forwarded message from Jonathan Gregory <j.m.gregory at reading.ac.uk-----
>>>>
>>>> Date: Wed, 15 Jul 2015 08:01:04 +0100
>>>> From: Jonathan Gregory<j.m.gregory at reading.ac.uk>
>>>> To:cf-metadata at cgd.ucar.edu
>>>> Subject: [CF-metadata] How to define time coordinate in GPS?
>>>> User-Agent: Mutt/1.5.21 (2010-09-15)
>>>>
>>>> Dear Karl
>>>>
>>>> I think the meaning of "gregorian" up to CF 1.6 is actually not clear, because
>>>> we had not thought about these distinctions earlier. If we change the calendar
>>>> definitions, it does not affect the interpretation of any existing data written
>>>> with previous versions. However I agree that it would be inconvenient for data-
>>>> users if they had to process "gregorian" times differently according to the
>>>> setting of the Conventions version.
>>>>
>>>> I thought that the current proposal was that gregorian would be vague in
>>>> future, like you wrote in your second option:
>>>>
>>>> 1) might or might not account for leap seconds, 2) might or might not assume
>>>> the length of the solar day is exactly 86400 seconds long, and 3) might express
>>>> the reference time according to either UTC or GPS
>>>>
>>>> and that is exactly the reason why, after agreeing with Jim last week, I
>>>> changed my mind to argue that we needed gregorian_nls for model data, to
>>>> preserve the exactness for times which really are always 86400-second days.
>>>> You agree we would need gregorian_nls in that case. I agree that it means that
>>>> models would use gregorian_nls in future, not gregorian.
>>>>
>>>> However in your first option you don't think this is necessary:
>>>>
>>>> Under the "gregorian" calendar the length of the solar day can be assumed to be
>>>> exactly 86400 seconds long (i.e., there are no leap seconds). This means that
>>>> for models where this assumption almost invariably is valid, conversion from
>>>> elapsed time to clock time is straight-forward and exact, whereas for
>>>> observations, conversion to clock time may introduce errors as large as 16
>>>> seconds because it is unknown whether the UTC or GPS time system has been used
>>>> in specifying the reference times (appearing in the time units attribute), and
>>>> it is also unknown whether leap seconds have been properly accounted for in
>>>> converting UTC clock times to elapsed time.
>>>>
>>>> I'm sorry, but I can't see what is the difference. In both cases you say it
>>>> is unknown whether leap seconds have been included in converting from clock
>>>> time to elapsed time. Is the crucial difference about the length of the day?
>>>> Does "assume the solar day to be exactly 86400 seconds" mean "assume all days
>>>> are 86400 seconds in converting elapsed time to clock time"?
>>>>
>>>> If that's what you mean, I think your first and preferred option is fine, and
>>>> then I agree we do not need gregorian_nls. To spell it out, I think this is
>>>> acceptable if "gregorian" means that you will recover the clock times (time-
>>>> stamps) exactly by using an algorithm that assumes constant 86400-second days
>>>> (i.e. with no leap seconds). If these clock times refer to the real world, it
>>>> is undefined whether the reference time is GPS or UTC, and consequently it is
>>>> undefined whether the decoded clock times are GPS or UTC.
>>>>
>>>> But my statement "you will recover the clock times (time-stamps) exactly by
>>>> using an algorithm that assumes constant 86400-second days" is inconsistent
>>>> with your statement "conversion to clock time may introduce errors ... because
>>>> it is unknown whether ... leap seconds have been properly accounted for in
>>>> converting UTC clock times to elapsed time". According to my statement,
>>>> gregorian means that leap seconds were ignored when converting clock times to
>>>> elapsed times i.e. 86400-second days were used. If that's not the case, you
>>>> can't decode accurately. Thus the elapsed times may not be quite right for the
>>>> real world. This is then the same as gregorian_utc_nls, so we would not need
>>>> that either, which was your final conclusion in your previous post.
>>>>
>>>> So my conclusion (at present - but I suspect I haven't understood something
>>>> you have said) is that gregorian is fine and sufficient (with the addition of
>>>> gregorian_utc and gregorian_gps for the well-defined real-world systems) if
>>>> we define it to mean that 86400-seconds are assumed to be used for conversion
>>> >from clock time to elapsed time and elapsed time to clock time, that it is not
>>>> defined whether the reference time and clock times are GPS and UTC, and that
>>>> the elapsed times may not be exactly correct for the real world (due to the
>>>> neglect of leap seconds).
>>>>
>>>> Best wishes
>>>>
>>>> Jonathan
>>>> _______________________________________________
>>>> CF-metadata mailing list
>>>> CF-metadata at cgd.ucar.edu
>>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>>> _______________________________________________
>>> CF-metadata mailing list
>>> CF-metadata at cgd.ucar.edu
>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>> ----- End forwarded message -----
>> _______________________________________________
>> CF-metadata mailing list
>> CF-metadata at cgd.ucar.edu
>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>
> --
> CICS-NC <http://www.cicsnc.org/> Visit us on
> Facebook <http://www.facebook.com/cicsnc> *Jim Biard*
> *Research Scholar*
> Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
> North Carolina State University <http://ncsu.edu/>
> NOAA National Centers for Environmental Information
> <http://ncdc.noaa.gov/>
> /formerly NOAA?s National Climatic Data Center/
> 151 Patton Ave, Asheville, NC 28801
> e: jbiard at cicsnc.org <mailto:jbiard at cicsnc.org>
> o: +1 828 271 4900
>
> /Connect with us on Facebook for climate
> <https://www.facebook.com/NOAANCEIclimate> and ocean and geophysics
> <https://www.facebook.com/NOAANCEIoceangeo> information, and follow us
> on Twitter at _at_NOAANCEIclimate <https://twitter.com/NOAANCEIclimate>
> and _at_NOAANCEIocngeo <https://twitter.com/NOAANCEIocngeo>. /
>
>
>
>
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20150720/3be50f5e/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 15784 bytes
Desc: not available
URL: <
http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20150720/3be50f5e/attachment-0001.png>
Received on Mon Jul 20 2015 - 10:51:56 BST
