Chris,
While I agree that the elapsed time *should* be correct, I think it's
highly likely that there are files "in the wild" that don't have correct
elapsed times. I know that until I started digging into this I would
have naively used the *nix timegm function to produce elapsed times from
UTC timestamps.
What I'm advocating is a way to allow the large majority of data
producers who have time resolutions that are coarse enough that leap
seconds don't matter to continue to work without having to do anything
extra, while providing a positive mechanism for those who have fine time
resolutions to signal to their users that they can trust the elapsed
time values. I'd say that what is special about this one is that it
involves a coordinate variable, as opposed to a data variable, and that
the number and kinds of pitfalls in moving between timestamps and
elapsed times warrant the extra attention.
The more I think about 'gregorian_nls' though, the more I think it is
not useful. If you have GPS time values (perhaps as elapsed times since
the GPS epoch of 1980-01-06 00:00:00 UTC) as your input, and you wanted
to use a GPS timestamp in your reference time (which for today is
currently 16 seconds off from UTC), you would need to have a way to
state that the timestamp is in the GPS time system. Similarly, if you
have TAI time values (elapsed times since the TAI epoch of 1958-01-01
00:00:00 UTC) and wanted to use a TAI timestamp in your reference time
(which for today is currently 35 seconds off from UTC), you would need
to have a way to state that the timestamp is in the TAI time system. The
gregorian_nls calendar is too generic. And then there's the Loran time
system (and others).
As an example, the date and time stamps as I'm writing this in the
different systems are:
* TAI 2015-05-20 20:21:00
* GPS 2015-05-20 20:21:16
* UTC 2015-05-20 20:21:35
How about this?
Redefine 'gregorian' to remove reference to UTC as Jonathan has
described, and state that presence or absence of leap second artifacts
in the reference timestamp and elapsed time values is not known for a
time variable that names this calendar. This is backward compatible.
Define 'gregorian_utc' as using the Gregorian calendar and the UTC time
system in the reference date and time in the units attribute. Specify
that the elapsed times *must* be free of leap second offsets or
discontinuities. Data producers that have GPS times as inputs (for
example) must be sure to obtain a proper UTC timestamp, but are
otherwise unconstrained. Data producers that have UTC timestamps as
inputs must be sure to properly handle conversion from UTC timestamps to
elapsed times so that leap second problems are not introduced. Data
consumers, if they read the section on time in the CF Conventions, will
understand how to make proper use of the time variable reference
timestamp and elapsed time values.
And we could also define 'gregorian_gps', 'gregorian_tai', etc. Or we
could define just one of them (it doesn't have to be gregorian_utc) and
say that this is what you use with CF. For folks with input time data
that is free of leap seconds (either time stamps or elapsed times), the
most they will have to do is convert the reference time to the chosen
time system. It's going to be more work for people with UTC timestamps
as inputs no matter what.
Grace and peace,
Jim
On 5/20/15 11:44 AM, Chris Barker wrote:
> On Wed, May 20, 2015 at 7:43 AM, John Graybeal
> <jbgraybeal at mindspring.com <mailto:jbgraybeal at mindspring.com>> wrote:
>
> Without fully appreciating _all_ the particulars (sorry!), I think
> Jonathan's diagnostic (that people would tend to keep using
> gregorian) is correct.
>
>
> agreed.
>
> I like the idea of a warning against that practice, with a
> recommendation to use gregorian_nls if that's appropriate (and of
> course a pointer to the relevant sections of the standard).
>
>
> But I'm not sure that we need to lighten the spec. Let's say that
> "gregorian" in no CF-1.7+ compliant. Yes, people will still use it, so
> they will have slightly non-compliant files, but folks will still be
> able to use them.
>
> But people are more likely to do the right thing if is officially
> non-=compliant than if it is simply against recommendations.
>
> If the goal is to get as many files as possible to use gregorian_nls
> (when appropriate) then it probably should be the standard.
>
> Note that I'm still a bit confused about the particulars here:
>
> It seems to me that the calendar describes what the epoch in a time
> variable means.
>
> the elapsed time _should_ be correct. period. If it says X seconds
> since the epoch, then it should be X seconds since the epoch.
>
> I understand that someone may have made a mistake/used an
> inappropriate library, etc, such that there may be an off
> by-a-couple-seconds error in the elapse time. But are we really trying
> to include something in the standard to accommodate that?
>
> There could be errors / lack of precision in ANY variable in a file --
> what's so special about this one?
>
> Or maybe I'm mis-interpreting the plan here.
>
> -Chris
>
>
>
>
>
>
>
>
> john
>
>
> On May 20, 2015, at 07:16, Jonathan Gregory
> <j.m.gregory at reading.ac.uk <mailto:j.m.gregory at reading.ac.uk>> wrote:
>
> > Dear Jim
> >
> > If instead we decided not to redefine gregorian, but to replace
> it with
> > gregorian_nls, that would be a more definite indication that
> action had been
> > taken to use the right code, wouldn't it. Would you be content
> with that?
> >
> > This isn't my favourite option, as you know. While it doesn't
> seem onerous,
> > I'm not sure it's realistic. I suspect that people might continue to
> > code calendar="gregorian" anyway, even if the CF checker
> reported it as an
> > error, but perhaps I am too pessimistic! A compromise would be
> to *recommend*
> > coding gregorian_nls, meaning the same as (redefined) gregorian,
> but indicating
> > more definitely that there were no leap seconds in the encoding,
> in order to
> > reassure the user it had been done correctly. If we took that
> course, the CF
> > checker would give a warning if gregorian was coded, and
> recommend that it
> > should be changed to gregorian_nls. This is a bit like your idea
> of having an
> > extra attribute, but it's implied by the existing attribute. I
> would be
> > comfortable with this compromise. What do you and others think?
> >
> > Best wishes
> >
> > Jonathan
> >
> > ----- Forwarded message from Jim Biard <jbiard at cicsnc.org
> <mailto:jbiard at cicsnc.org>> -----
> >
> >> Date: Wed, 20 May 2015 09:15:01 -0400
> >> From: Jim Biard <jbiard at cicsnc.org <mailto:jbiard at cicsnc.org>>
> >> To: cf-metadata at cgd.ucar.edu <mailto:cf-metadata at cgd.ucar.edu>
> >> Subject: Re: [CF-metadata] How to define time coordinate in GPS?
> >> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0)
> >> Gecko/20100101 Thunderbird/31.7.0
> >>
> >> Jonathan,
> >>
> >> Given the direction we have taken of using a new calendar name to
> >> capture the other aspect, we are limited in our options. We could
> >> specify an attribute that, if present, could be taken as a
> >> "guarantee" (although people admittedly could set the attribute and
> >> still do it wrong). The attribute name could be time_encoding with
> >> values of 'true_elapsed' and 'unknown'. The value of 'unknown'
> would
> >> be the default, so if the attribute was not specified, the
> >> assumption would be that you can't be sure. Most data producers
> >> would have no need to specify the attribute, since their time
> >> resolutions are not so fine as to make them sensitive to the
> >> potential problems.
> >>
> >> Other than your suggestion about 1.7 implying greater care taken,
> >> that's what comes to mind.
> >>
> >> Grace and peace,
> >>
> >> Jim
> >>
> >> On 5/19/15 5:23 PM, Jonathan Gregory wrote:
> >>> Dear Jim
> >>>
> >>>> We are definitely getting much closer to full agreement. I
> continue
> >>>> to think that a separate time_system (or some such) attribute
> would
> >>>> be a much better way to handle this than modifying the calendar
> >>>> attribute, and that space-separated calendar modifiers would
> be next
> >>>> best after that, but I will bow to the apparent majority and
> agree
> >>>> to your proposal for modified definitions of the general
> reference
> >>>> time and Gregorian calendar, the addition of a new gregorian_utc
> >>>> calendar, etc, as you just outlined.
> >>> That is good news. I am glad that we understand things in a
> compatible way now
> >>> thanks to this discussion, and I am grateful for your
> flexibility and willing-
> >>> ness to compromise on the implementation.
> >>>
> >>>> There are two remaining things that I would like to see.
> >>>>
> >>>> 1. A section that explains the importance for data producers and
> >>>> consumers of using the right time handling routines for the
> input
> >>>> time data and the calendar chosen if your time resolution
> makes you
> >>>> sensitive to errors on the order of 1-30 seconds, pointing
> out (for
> >>>> example) that using the standard *nix time functions with
> sets of
> >>>> correct UTC timestamps will produce incorrect elapsed time
> values
> >>>> under the right conditions. Such a section would also point
> out to
> >>>> data consumers that if errors on this level are significant
> to them,
> >>>> they shouldn't assume that existing observational datasets
> are free
> >>>> of these errors.
> >>> That's a good idea. I am in favour of it. We can do that in
> the section where
> >>> we describe the calendars.
> >>>
> >>>> 2. A standard mechanism for data producers to indicate that
> they have,
> >>>> in fact, taken the extra care with their time calculations,
> >>>> whichever calendar they may have chosen - that is, the
> elapsed time
> >>>> values are guaranteed to be free of any leap second related
> offsets
> >>>> or discontinuities. This would give data consumers greater
> >>>> confidence in cases where such errors matter.
> >>> Yes, I see the value in that. I wonder how we would do it.
> What comes to mind
> >>> is to suggest that they are making that guarantee by
> specifying CF-1.7 (I hope
> >>> - or whichever version contains this change) rather than any
> earlier version,
> >>> which means in particular that the calendar will no longer
> have a default. If
> >>> they have changed their software to write CF-1.7 in the
> Conventions attribute,
> >>> we can hope that they have also changed it to be consistent
> with the new and
> >>> more precise definition of the calendar attribute. The changes
> made are listed
> >>> in an appendix to the standard (it will be Appendix Z in
> CF-1.7) and the
> >>> implication of this change should be made clear there. What
> else could we do?
> >>>
> >>> Best wishes
> >>>
> >>> Jonathan
> >>> _______________________________________________
> >>> CF-metadata mailing list
> >>> CF-metadata at cgd.ucar.edu <mailto:CF-metadata at cgd.ucar.edu>
> >>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> >>
> >> --
> >> CICS-NC <http://www.cicsnc.org/> Visit us on
> >> Facebook <http://www.facebook.com/cicsnc> *Jim Biard*
> >> *Research Scholar*
> >> Cooperative Institute for Climate and Satellites NC
> <http://cicsnc.org/>
> >> North Carolina State University <http://ncsu.edu/>
> >> NOAA National Centers for Environmental Information
> <http://ncdc.noaa.gov/>
> >> /formerly NOAA?s National Climatic Data Center/
> >> 151 Patton Ave, Asheville, NC 28801
> >> e: jbiard at cicsnc.org <mailto:jbiard at cicsnc.org>
> <mailto:jbiard at cicsnc.org <mailto:jbiard at cicsnc.org>>
> >> o: +1 828 271 4900 <tel:%2B1%20828%20271%204900>
> >>
> >> /We will be updating our social media soon. Follow our current
> >> Facebook (NOAA National Climatic Data Center
> >> <https://www.facebook.com/NOAANationalClimaticDataCenter> and NOAA
> >> National Oceanographic Data Center
> >> <https://www.facebook.com/noaa.nodc>) and Twitter (_at_NOAANCDC
> >> <https://twitter.com/NOAANCDC> and _at_NOAAOceanData
> >> <https://twitter.com/NOAAOceanData>) accounts for the latest
> >> information./
> >>
> >>
> >
> >> _______________________________________________
> >> CF-metadata mailing list
> >> CF-metadata at cgd.ucar.edu <mailto:CF-metadata at cgd.ucar.edu>
> >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> >
> >
> > ----- End forwarded message -----
> > _______________________________________________
> > CF-metadata mailing list
> > CF-metadata at cgd.ucar.edu <mailto:CF-metadata at cgd.ucar.edu>
> > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu <mailto:CF-metadata at cgd.ucar.edu>
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>
>
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R (206) 526-6959 voice
> 7600 Sand Point Way NE (206) 526-6329 fax
> Seattle, WA 98115 (206) 526-6317 main reception
>
> Chris.Barker at noaa.gov <mailto:Chris.Barker at noaa.gov>
>
>
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
--
CICS-NC <http://www.cicsnc.org/> Visit us on
Facebook <http://www.facebook.com/cicsnc> *Jim Biard*
*Research Scholar*
Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
North Carolina State University <http://ncsu.edu/>
NOAA National Centers for Environmental Information <http://ncdc.noaa.gov/>
/formerly NOAA?s National Climatic Data Center/
151 Patton Ave, Asheville, NC 28801
e: jbiard at cicsnc.org <mailto:jbiard at cicsnc.org>
o: +1 828 271 4900
/We will be updating our social media soon. Follow our current Facebook
(NOAA National Climatic Data Center
<https://www.facebook.com/NOAANationalClimaticDataCenter> and NOAA
National Oceanographic Data Center <https://www.facebook.com/noaa.nodc>)
and Twitter (_at_NOAANCDC <https://twitter.com/NOAANCDC> and @NOAAOceanData
<https://twitter.com/NOAAOceanData>) accounts for the latest information./
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20150520/3470282e/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: CicsLogoTiny.png
Type: image/png
Size: 15784 bytes
Desc: not available
URL: <http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20150520/3470282e/attachment-0001.png>
Received on Wed May 20 2015 - 14:43:49 BST
