Jonathan,
I still think there's
On 5/10/15 3:29 AM, Jonathan Gregory wrote:
> Dear all
>
> A postscript to my last message. I am not myself convinced that the backward
> incompatibility (for data-writing) that I suggested below is really worth the
> pain it would cause! I think it might well be OK to retain the existing
> calendar name of "gregorian" from CF1.7, but give it a more precise definition,
> in order to eliminate the ambiguity about leap seconds. (Of course, there is
> nothing we can do to remedy this with existing data.)
>
> With all the arguments otherwise unaltered, that would make my proposal:
>
> * We redefine "gregorian", and introduce a new calendar "gregorian_utc", for
> the real-world calendar, without and with UTC leap seconds respectively.
>
> * We abolish the "standard" calendar (since one thing this discussion has
> shown is that there is not a single standard!) and we require the calendar
> always to be specified (no default).
>
> * We state that all the other calendars have fixed-length days with no leap
> seconds.
>
> My further suggestions, about dealing with the Julian/Gregorian transition
> and negative years in better ways, are unaffected (except to retain the name
> "gregorian"). Those are an optional extra, dealing with a different subject,
> but it would be opportune to do both at once.
>
> Cheers
>
> Jonathan
>
> ----- Forwarded message from Jonathan Gregory <j.m.gregory at reading.ac.uk-----
>
> Dear Jim
>
> Yes, I'm glad we appear to be approaching an agreement!
>
>>> "In order to calculate a new date and time given a base date, base time and a
>>> time increment one must know what calendar to use."
>>> and I think that is the sense in which I am using "calendar".
>> I agree that this is a CF-consistent usage of the word calendar, but
>> it runs against natural usage, and I think it's worth keeping that
>> in mind.
> OK. Perhaps we can clarify what we mean in the CF convention.
>
>> I agree that leap seconds haven't been carefully considered before.
>> I disagree that nearly all existing time values have been encoded
>> without leap seconds. I'd say that nearly all existing time values
>> that were derived from true UTC timestamps are at risk of having
>> leap second discontinuities encoded into the set of values.
> All right. In that case I think we may have to take more serious steps to
> avoid future problems. I'll come back to that.
>
>> There are three issues here, so let's not conflate them. They are:
>>
>> 1. What to call the time system that is like UTC in overall form
>> (Greenwich meridian, etc) but doesn't include leap seconds.
>> 2. How to indicate which actual time system is being used for the time
>> part of the reference time in the units attribute.
>> 3. How to indicate whether or not the elapsed times in the time
>> variable are certain to be free of leap second induced discontinuities.
> This is a useful classification, thanks, but I don't think the situation is
> quite as complicated as that.
>
> I'm not sure (2) is something we need to be concerned about - but it may
> be I've missed a point. In my understanding, there is no calendar implied by
> the reference time, because it's given as a timestamp YYYY-MM-DD hh:mm:ss
> [+-hh:mm], where the [] is the optional time-zone. A timestamp is calendar-
> neutral; it can be interpreted in any calendar, except that some dates will be
> illegal in some calendars, and some times might be illegal if leap seconds are
> in effect. The reference date-time is used for encoding and decoding in the
> calendar specified, with or without leap seconds, but itself implies nothing
> about the time system. We could state it as a requirement of CF that the
> reference date and time must be legal in the encoding which applies to the time
> coordinate.
>
> Regarding part of (1), we need a name for the time-zone which applies at the
> Greenwich meridian without summer/daylight-saving time. We shouldn't use "UTC",
> as the CF standard currently does (quoting from udunits(3)), because that's
> confusing. Maybe it should just be stated explicitly as I have done!
I agree that it's important to write an explicit definition of the
timestamp that doesn't reference UTC. I don't see timestamps as being
time system neutral any more than datestamps are calendar neutral. They
may be interpretable by more than one system, but that's not the same
thing in my mind as neutrality.
You can read a Gregorian date as a Julian date, but you are going to be
off by a number of days. You can read a UTC time as a traditional time,
but you are going to be off by a number of seconds. The new calendars
you propose would address the question of what systems (calendar and
time) were used for all parts of the time reference in the units
attribute. That is what they accomplish, they address point (2). They
don't really address point (3) at all.
> To address point (3), I continue to favour a subspecies of calendar name,
> rather than a modifier or separate attribute, because this distinction only
> applies to the real-world calendars. A decomposition of metadata is cumbersome
> if it isn't generally relevant. Suppose X can take values X1 or X2; if it's X1
> then Y can be Y1 or Y2, whereas Y is irrelevant if it's X2. In that situation I
> would have a single attribute with possible values X1_Y1 X1_Y2 and X2. A single
> attribute is easier for scanning a dataset, less work to write and read, and
> more likely to be correct because it's less likely that Y will not be coded if
> relevant or will be coded if irrelevant.
>
> I suggest that the leap seconds are needed only in the Gregorian calendar i.e.
> the real-world one. I think it's unlikely the proleptic Gregorian calendar will
> be used with leap-seconds; you only need this calendar if you're going back
> more than several centuries, in which case it's probably not a dataset which
> has UTC precision in time, and it's most often used with models, which do not
> have leap seconds. (However, a leap-second variety could be introduced if I am
> wrong and it is needed.) Leap seconds are not needed in the noleap, all_leap,
> 360_day and none calendars, which are all for model worlds. The julian calendar
> is used astronomically and might be used in models, but the web page you
> cited (http://www.ucolick.org/~sla/leapsecs/timescales.html) points out that's
> not a good idea to try to use it with leap seconds since it's based on units
> of day (=86400 s).
Point (3) relates to the question of which calculator (implementation of
a calendar and/or time system as software for converting date and/or
time stamps to elapsed times since an epoch) was used to create the
elapsed time values found in a given time variable. This question is
largely independent of the question of which systems are represented by
the reference date and time in the units attribute. An example in terms
of dates alone may help clarify the issue.
I have a set of datestamps that are based on the Gregorian calendar. I
calculate elapsed days from the reference date stored in my units
attribute on my time variable and populate my time variable with the
values. I use a calculator that is based on the rules of the Julian
calendar to get my elapsed times.
As long as the span of dates from the reference date to the last
datestamp in my set don't cross any year where the two calendars differ
on whether or not to add a leap day, the values stored in my time
variable will be correct. But a 1-day discontinuity will be encoded into
my elapsed day values each time I do cross a place where the Julian and
Gregorian calendars differ.
If I turn my elapsed dates back into datestamps using the same Julian
date calculator, no one will notice. If I instead use a Gregorian date
calculator to recover datestamps and one or more discontinuities were
encoded, I will find that I don't get back the same set of datestamps I
started with. If I try to take differences between time variable values
and one or more discontinuities were encoded, I will find that the
results will contain an error if the difference is taken across the
location of a discontinuity.
This may sound a bit silly when speaking of Gregorian vs Julian
calendars, but this is exactly what has been happening on the time
system level when people have received UTC timestamps and naively used
the *nix time functions to create elapsed time values to store into time
variables. More explicitly defining the time system used for the time
part of the reference date and time in the units attribute via an
expanded calendar definition does not tell you how the elapsed times
stored in the time variable were calculated.
> If this is the case, then I would propose that in the next version of CF:
>
> * We introduce two new calendars, "gregorian_noleaps" and "gregorian_utc", for
> the real-world calendar, without and with UTC leap seconds respectively. I
> suggest "noleaps" instead of "traditional", which you put forward, because
> "noleaps" is more self-explanatory, I think. I agree with you that "POSIX" is
> not so good because it implies a reference time.
I agree with Ben Hetland that if we were to go with new calendars to
address point (2) we should use a name like gregorian_noleapseconds or
gregorian_noleapsec. I disagree with Ben about the question of how
difficult it is to parse a fixed "calender [time-system [encoding]]"
sequence.
I think that backward compatibility, among other things, argues in favor
of adding trailing space-separated modifiers to existing calendar names.
Even if we end up going with some version of your proposed new calendar
names, it's important to understand that those new names and definitions
don't solve the issue in point (3).
> * We abolish the "gregorian" and "standard" calendars, and we require the
> calendar always to be specified (no default). This would be quite a radical
> step, but forcing the noleaps/utc property to be explicitly stated is only way
> I can see to avoid in future the ambiguity about whether elapsed times have
> been encoded with or without leap seconds. This does not invalidate existing
> data that adheres to CF1.6 or earlier, but it would invalidate existing
> data-writing software that does not write the calendar attribute, and it would
> require data-reading software to recognise the new calendars (although if it
> assumed the existing default for them that would not be too bad).
>
> * We state that all the other calendars have fixed-length days with no leap
> seconds.
>
> What do you think? Is it worth the pain?
>
> Although it's going to complicate this discussion, if we decided to abolish the
> existing default, we could take this opportunity to deal with problems with the
> mixed Julian/Gregorian calendar, as in http://cf-trac.llnl.gov/trac/ticket/96,
> opened by Dave Allured, but never concluded. To do that I would further propose
> (copying some text from that ticket):
>
> * The gregorian_noleaps and gregorian_utc calendars can be used only to encode
> dates since 1582-10-15, and must not have reference dates earlier than that.
> This means they cannot cross the Julian/Gregorian transition. This is desirable
> because there are ambiguities introduced by assuming different dates for the
> change in calendar. By this rule, the common choice of 1-1-1 as a reference
> date would be disallowed in these calendars, for example.
>
> * We introduce a new calendar "mixed_gregorian_julian", which is the calendar
> of udunits, with no leap seconds. However we make it stricter than currently,
> in these ways: (1) The reference date is not allowed to be any of the dates in
> the transitional period 1582-10-5 to 1582-10-14 inclusive. (2) Neither the
> reference date nor any date which is encoded with this calendar is allowed to
> be a negative year. (3) Year 0 is interpreted as climatological time in this
> calendar, following COARDS, but this is deprecated in favour of the CF
> conventions of Sect 7.4.
>
> * Because of problems caused by the discontinuity, it is recommended that the
> mixed_gregorian_julian calendar be used only in datasets with real-world
> historical dates which span the change of calendar from Julian to Gregorian. In
> datasets with real-world historical dates that all precede the change of
> calendar, the julian calendar should be used. In datasets with real-world
> historical dates that all follow the change of calendar, and in simulated
> datasets in which there is no change of calendar, the proleptic_gregorian
> calendar should be used.
>
> * We disallow dates to be encoded or reference dates to be used in year zero or
> negative years for the julian calendar, because it's ambiguous whether this
> calendar has a year 0.
>
> * We state that year 0 is valid in the proleptic_gregorian, noleap, all_leap,
> 360_day and none calendars.
>
> If these further proposals complicate the previous discussion, we can defer
> them until we've reached agreement on leap seconds!
>
> Best wishes
>
> Jonathan
>
> ----- End forwarded message -----
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Grace and peace,
Jim
--
CICS-NC <http://www.cicsnc.org/> Visit us on
Facebook <http://www.facebook.com/cicsnc> *Jim Biard*
*Research Scholar*
Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
North Carolina State University <http://ncsu.edu/>
NOAA National Centers for Environmental Information <http://ncdc.noaa.gov/>
/formerly NOAA?s National Climatic Data Center/
151 Patton Ave, Asheville, NC 28801
e: jbiard at cicsnc.org <mailto:jbiard at cicsnc.org>
o: +1 828 271 4900
/We will be updating our social media soon. Follow our current Facebook
(NOAA National Climatic Data Center
<https://www.facebook.com/NOAANationalClimaticDataCenter> and NOAA
National Oceanographic Data Center <https://www.facebook.com/noaa.nodc>)
and Twitter (_at_NOAANCDC <https://twitter.com/NOAANCDC> and @NOAAOceanData
<https://twitter.com/NOAAOceanData>) accounts for the latest information./
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20150511/1b2f327b/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: CicsLogoTiny.png
Type: image/png
Size: 15784 bytes
Desc: not available
URL: <http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20150511/1b2f327b/attachment-0001.png>
Received on Mon May 11 2015 - 08:35:49 BST
