Hi all,
CF specifies that the value of the units attribute is a string that can be
recognized by UDUNITS. (There are a few variations, e.g., ?as per the
recommendations [in UDUNITS]? and ?formatted as per the udunits.data
file?.) While there are a few recommendations based on how UDUNITS handles
or interprets particular units strings, there is no mention in CF that
UDUNITS must or even should be used for anything beyond validating a units
string.
[Sorry, I haven?t really kept up with the ?Add calendars gregorian_tai
...? conversation. But I see now that Chris mentions the above [1].
Hopefully, I haven?t missed too much else in my skimming of that
conversation.]
It seems that CF should be more explicit about how it relies on UDUNITS?
Beyond recognizing valid units strings, is UDUNITS a reference
implementation for how units should be interpreted and values converted. If
so and given that UDUNITS does not handle calendars, it seems clear that
time (at least when a calendar is specified?) should be an exception. Are
there other areas that should be exceptions?
Cheers,
Ethan
[1]
https://github.com/cf-convention/cf-conventions/issues/148#issuecomment-434369818
On Mon, Oct 29, 2018 at 7:48 AM Martin Juckes - UKRI STFC <
martin.juckes at stfc.ac.uk> wrote:
> Hello Klaus,
>
> you are right, "30 days" would be invalid according the penultimate
> paragraph of section 3.1 .... it would, however, be OK according to the
> rules in the conformance document and the CF checker. There are 3 current
> standard names which have invalid canonical units by this rule (of the form
> "1e-3 kg ..."), and several CMIP6 variables. It is a distraction from the
> main topic here ... so I'll start another thread.
>
> I agree with your suggestion that many issues could be resolved by adding
> new units to the "udunits2-common.xml" section of the UDUNITS database. If
> we want to support statements of the form "units = calendar_months since
> 1900-01-01", a little more work would be needed. The "... since ..." format
> is supported by UDUNITS for time and the date is always interpreted in
> terms of the Julian/Gregorian calendar: getting it extended to a new family
> of units may be difficult.
>
> regards,
> Martin
> ________________________________
> From: CF-metadata <cf-metadata-bounces at cgd.ucar.edu> on behalf of Klaus
> Zimmermann <klaus.zimmermann at smhi.se>
> Sent: 29 October 2018 12:27
> To: cf-metadata at cgd.ucar.edu
> Subject: Re: [CF-metadata] [CF-metadar ata] [EXTERNAL] 'months since' and
> 'years since' time units
>
> Hello All,
>
> all in all it seems that there is broad agreement that we need a new way
> to describe coarser time information that has to be separate from the
> time domain as described in udunits and, by extension, CF up to now.
>
> The finest unit there should be a general calendar month, the only other
> one that was mentioned up to now would be some form of calendar year
> that always consists of 12 calendar months.
>
> The relationship to day of year or day of month will depend on external
> choices like the calendar and application, but might even be ill defined
> if, for example, in a climatology data from different calendars is
> combined.
> Consequently, no automatic conversion between the new domain and the
> regular time domain is possible.
>
>
> The open question really is how this should be implemented. It could be
> brought into the udunits database, coded completely separately in an
> attribute other than units, or written in the units attribute, but in a
> special form to mark it as separate from udunits.
>
> Once these choices are made, some names need to be agreed.
>
>
> I want to add some technical remarks. Using "units: 30 days" as proposed
> by Martin would be in violation of CF 3.1 (as I understand it), where it
> says that scale factors as supported by udunits are not allowed in CF.
> Of course, this could be changed and indeed CF seems to be lagging
> behind the current udunits a bit: udunits2 supports ppm, for instance,
> and does not contain a udunits.dat file anymore.
>
> Instead, the database of units is distributed as a set of XML files,
> namely "udunits2.xml" (that merely imports the others),
> "udunits2-base.xml" (that contains the SI base units),
> "udunits2-derived.xml" (that contains the SI derived units),
> "udunits2-prefixes.xml" (that contains the SI prefixes, of course). All
> of the data up to this point comes from the SI governing body, but there
> is one last file, "udunits2-common.xml", that contains a rich set of
> definitions that were deemed useful despite not being officially
> endorsed by the SI, among them ppm.
>
> It seems quite possible to include the new domain into the -common file;
> alternatively one could think to distribute an extension or an extended
> version of the udunits database, while still relying on the udunits
> library for parsing and conversion of all units.
>
> I personally would prefer to either include the new definitions in
> udunits or have them as a completely separate attribute, but not to mix
> a new way of interpretation into the currently udunits only units
> attribute.
>
> Regards
> Klaus
>
>
> On 29/10/18 11:12, Martin Juckes - UKRI STFC wrote:
> > Hello All,
> >
> >
> > There is perhaps a simpler alternative to deal with the problem of
> encoding data from 360 day calendars: the data described by Ryan should be
> encoded with "units: 30 days", which would be valid and accurate.
> >
> >
> > The references to UDUNITS may be a bit of a distraction ... UDUNITS
> inherits all its definitions of units of time, distance, mass, heat and
> current from the SI units. Coincidentally, the governing body for SI units
> will be discussing a change in the definition of the kilogram this month
> (it is to be redefined in terms of he Planck constant). What is happening
> to the SI kg is a bit of a diversion, but I want to make the point that the
> relationship between, for instance, energy, mass, length and time, and the
> way in which that is represented in our units system is fundamental to our
> understanding of many aspects of the physical world. Making a change to the
> interpretation of "time" which destroys this relationship would be a
> serious step.
> >
> >
> > The CDM library that Dave referred to near the start of this
> conversation does make this distinction between SI time and instants of
> historical time explicit with its "datetime" construction.
> >
> >
> > There is another complexity with calendars which hasn't been discussed
> yet: outside CF calendars are understood as different ways of representing
> one unique sequence of days. The Islamic calendar, for instance, has a
> shorter year than the Gregorian calendar, but given any date in the Islamic
> calendar there is a unique corresponding Gregorian date: they are just two
> representations of the same time-line. Hence, in the Islamic calendar, the
> seasons move through the year in a 31 year cycle. When climate model output
> is written with a 360-day calendar, on the other hand, we assume that this
> model is using modified orbital parameters which give a seasonal cycle of
> 360 days. It follows that dates match up, if at all, in a fuzzy sense, e.g.
> 1900-01-01 in the 360 day calendar corresponds to 1900-01-01 in the
> Gregorian calendar, 2000-01-01 with 2000-01-01. The mapping for days is
> ill-defined, as there just aren't enough days in the 360-day calendar. This
> distinction between a model world, in which the solar cycle has been
> coerced to match a convenient number of days, and real world calendars is
> not made explicit at the moment.
> >
> > regards,
> > Martin
> > ________________________________
> > From: B?rring Lars <Lars.Barring at smhi.se>
> > Sent: 26 October 2018 18:04
> > To: Ryan Abernathey; Juckes, Martin (STFC,RAL,RALSP)
> > Cc: cf-metadata at cgd.ucar.edu
> > Subject: SV: [CF-metadata] [CF-metadar ata] [EXTERNAL] 'months since'
> and 'years since' time units
> >
> > CORRECTION IN CAPITALS below:
> >
> >
> > Hello Ryan, all,
> >
> > I fully agree with you that bringing in the real world calendar
> complications of uneven month lengths and leap days has substantially
> broadened the conversation. And when I was trying to read up on earlier
> email threads I, too, learnt quite a lot.
> >
> > As far as understand:
> > the issue at hand for your very specific question/request is that in
> Udunits the unit MONTH is defined as year/12, and an alias to tropical_year
> which is defined to be exactly 365.242198781 days. In that sense the unit
> month is already taken. But I understand you initial request, you ask
> whether unit month for the specific case of the 360 day calendar could be
> redefined to as 1 month = 30 days (which of course make perfect sense for
> this particular calendar.) Basically that makes the unit month a function
> of the calendar. If it is possible it will solve your initial question but
> not the substantially more issues surrounding the uneven month lengths (for
> which the 1 month = year/12 will remain as status quo).
> >
> > _IF_ it is possible to have the month length depend on the calendar,
> then I strongly suggest that we do the same thing for year, i.e. to have
> the length of the year depend on the calendar. This will many issues for
> most calendars, basically turning year to a useful unit in many situations.
> >
> > Kind regards,
> > Lars
> >
> >
> >
> > ________________________________
> > Fr?n: Ryan Abernathey [ryan.abernathey at gmail.com]
> > Skickat: den 26 oktober 2018 18:29
> > Till: martin.juckes at stfc.ac.uk
> > Kopia: rkl at bodc.ac.uk; dblodgett at usgs.gov; jbiard at cicsnc.org; B?rring
> Lars; j.m.gregory at reading.ac.uk; cf-metadata at cgd.ucar.edu
> > ?mne: Re: [CF-metadata] [CF-metadar ata] [EXTERNAL] 'months since' and
> 'years since' time units
> >
> > Hi Everyone,
> >
> > Thanks for this fascinating discussion. I never imagined that this
> question would provoke so much complexity. I have learned a lot.
> >
> > Coming back to the original issue which motivated this discussion: using
> months as a unit for 360 day calendars. In this case, 1 month = 30 days.
> There is no ambiguity or complexity. In this case it is a physical unit.
> Many existing datasets already encode data this way, in violation of the CF
> recommendations.
> >
> > I fear that, in the desire to generalize this issue and solve a much
> broader and more difficult problem (how to deal with all possible ambiguous
> units in all possible calendars), the original problem will be forgotten.
> Given my lack of experience in this field, it is hard for me to tell
> whether the solutions being proposed here will actually impact the original
> issue. I cannot force the data provider in question (IRI data library) to
> re-encode all its data with a new convention. If I could, I would just have
> them re-encode following the existing standard and be done.
> >
> > In any case, thank you all very much for this important work. I
> sincerely appreciate all of your efforts.
> >
> > Best,
> > Ryan
> >
> >
> >
> > On Fri, Oct 26, 2018 at 11:48 AM Martin Juckes - UKRI STFC <
> martin.juckes at stfc.ac.uk<mailto:martin.juckes at stfc.ac.uk>> wrote:
> > Dear All,
> >
> >
> > My apologies to Dave for creating the diversion around the opaqueness of
> UDUNITS. There are two sides to this otherness: on the one hand, it is an
> independent group of people doing good work that we can take advantage of,
> on the other, they express the rules and concepts which they arrive at in
> their own syntax which takes an effort for us to disentangle.
> >
> >
> > Before proposing anything to UDUNITS (if we decide to go that way) I
> think it is worth thinking a bit more about exactly what we want and how it
> fits into other frameworks for managing units. Dave has already mentioned
> some software frameworks which deal with this cleanly, but Lars has pointed
> out that we need an extra step to get something in the standard.
> >
> >
> > UDUNITS ties itself back to the SI standard, which is clearly anchored
> in physical constants and is not going to touch this sort of thing. UDUNITS
> does have some units which are not defined in terms of SI units ... but
> they mainly have a fairly direct linear relationship with SI units.
> >
> >
> > The GML (Geography Markup Language) has some potentially relevant
> formalisms. They have a concept of "ConventionalUnits", which are units
> which are not defined in terms of SI units (though they may have a rough
> conversion factor). They also have "AbstractTime" constructions which
> support the calendar year and months. In the GMLJP2 application profile for
> JPEG the "AbstractTime" concept is implemented for 3 calendars (Gregorian,
> Julian and GPS).
> >
> >
> > Jim has suggested "coarse_time/year/month" and "orbital_time/year/month"
> as alternative labels. The 2nd of these made me think about the option of
> just counting orbits, which is already supported by the UDUNITS unit of
> "cycle", which is a measure of angle. Unfortunately this wouldn't work as
> the calendar year and month not a fixed angular displacements around the
> sun. One of the main motivations is to have a variable which is explicitly
> defined to follow the rules set out by various calendars, so perhaps
> "calendar_time" would be better (as a standard name)? Used together with
> "calendar_month" and "calendar_year", this would take us back to Karl's
> suggestion on the 18th.
> >
> >
> > As Jim has said, if we introduce these new units in association with a
> new standard name, we can make it clear that they do not transform directly
> into the existing time units (though software should be able to this easily
> enough). Karl has suggested that the reference time should also be
> restricted to monthly granularity, e.g. "calendar_months since 2001-01".
> >
> >
> > As Karl has said, the big gain will be in making it easy for people to
> describe simple concepts (such as data for each calendar month) without
> having to work their way around the fact that the UDUNITS month is a fixed
> number of seconds.
> >
> >
> > Jonathan has suggested that we should rely on software to do the
> conversion from the simple view of calendar months into CF metadata and
> back, but that will result in multiple different implementations presenting
> the users with a range of options. On the basis that simple information
> should be stored transparently, I think we should try to accommodate the
> many users who want calendar months directly. This would also make it
> possible to specify an alternative, slightly more transparent, formulation
> for cell_methods for monthly climatological means: rather than "time: mean
> within days time: mean over days time:mean over years" we could have
> "calendar_time: mean within calendar_month calendar_time:mean over
> calendar_year".
> >
> >
> > We have all been talking as if the length of the day is a fixed number
> of seconds .... which is a good enough approximation, but looking up the
> definition of "GPS time" has reminded me that this isn't the case. Some
> days have leap-seconds, most don't (there is about 1 leap second every 1.5
> years). We don't have to worry about this in climate applications, but
> there may be Earth Observation applications that need to be careful about
> these leap seconds. This was discussed on the CF list back in 2015, and the
> conclusion was that there was no need to take it into account, as the
> impact would be negligible for the use cases we want to support. On the
> other hand, if we proposing something to UDUNITS it may be worth
> considering how this fits in.
> >
> >
> > <calendar>_time: A variable measuring the progression of time within a
> calendar system in which the periods of interest have variable clock-time
> lengths (e.g. calendar months vary from 28 to 31 days in length in the
> Gregorian calendar). The standard name "time" should be used if clock time
> is wanted, this term treats each calendar month as an equal unit.
> >
> >
> > The canonical units would be "calendar_year", and the construction
> "calendar_year since YYYY-MM" would be supported.
> >
> >
> > For UDUNITS the proposal would be that, following GML, they allow
> calendar_year and calendar_month (=calendar_year/12) and also calendar_week
> and calendar_day (=calendar_week/7) as units of a calendar system which may
> involve irregular time increments. The calendar_day differs from the day
> when a leap second is introduced, once every 1.5 years on average. The
> calendar_month is a whole number of days, and never equivalent to the month
> time unit. These are not physical based units, but they are widely used in
> societal and environmental applications.
> >
> >
> > I'm sorry that this has turned into a rather disjointed set of ideas
> .... should we start an trac ticket?
> >
> >
> > regards,
> >
> > Martin
> >
> > ________________________________
> > From: Lowry, Roy K. <rkl at bodc.ac.uk<mailto:rkl at bodc.ac.uk>>
> > Sent: 26 October 2018 09:46
> > To: David Blodgett; Juckes, Martin (STFC,RAL,RALSP)
> > Cc: cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>
> > Subject: Re: [CF-metadata] [CF-metadar ata] [EXTERNAL] 'months since'
> and 'years since' time units
> >
> >
> > Dear Dave,
> >
> >
> > I see no problem with CF incorporating external standards such as
> UDUNITS providing that standard is adequately maintained (e.g. addition of
> new concepts) and supported by its governance through the provision of
> tools. In my opinion UDUNITS governance makes the grade. Indeed, it was a
> tool development query by UDUNITS that kicked off this thread and Martin's
> suggestion is to contact UDUNITS governance which I'm sure won't be ignored.
> >
> >
> > The alternative would be for CF to set up its own controlled vocabulary
> for units of measure. Even if the resources to do this could be secured I
> feel it would be a terrible waste.
> >
> >
> > Cheers, Roy.
> >
> >
> > I have now retired but will continue to be active through an Emeritus
> Fellowship using this e-mail address.
> >
> >
> > ________________________________
> > From: CF-metadata <cf-metadata-bounces at cgd.ucar.edu<mailto:
> cf-metadata-bounces at cgd.ucar.edu>> on behalf of David Blodgett <
> dblodgett at usgs.gov<mailto:dblodgett at usgs.gov>>
> > Sent: 26 October 2018 01:31
> > To: Martin Juckes - UKRI STFC
> > Cc: cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>
> > Subject: Re: [CF-metadata] [CF-metadar ata] [EXTERNAL] 'months since'
> and 'years since' time units
> >
> > Dear Martin,
> >
> > I'll leave it to others to argue the merits of various solutions. My aim
> was to point out that this problem is solved by prominent software which
> does basically what you describe. We could quibble about the concept of
> time and date being different, but given that this problem has a clear
> solution, I don't really care to. One way or another we will need to
> introduce the complexity of an additional way of discretizing a temporal
> axis that is based on cyclic calendar days/months/years rather than
> continuous seconds increments of time that add up to days months and years
> in nuanced ways depending on the time of year or leap-cycle.
> >
> > Your suggested path -- work with the UDUNITS community to define this
> new type of timeline unit -- make very good sense.
> >
> > I would like to clarify my point about UDUNITS being relied on. The
> parallel would be reliance on EPSG codes for projection parameters. A
> UDUNITS string is completely opaque to software that ONLY knows CF. By my
> understanding of CF, this breaks the rules by not fully describing the
> concept within the specification. I'm not complaining, I think it was the
> right decision to use an external source for units of measure -- I'm just
> pointing out the inconsistency relative to other aspects of the
> specification.
> >
> > - Dave
> >
> >> On Oct 25, 2018, at 7:37 AM, Martin Juckes - UKRI STFC <
> martin.juckes at stfc.ac.uk<mailto:martin.juckes at stfc.ac.uk>> wrote:
> >>
> >> Dear David, Lars,
> >>
> >>
> >> I'm afraid I do have a couple of reservations, though I understand the
> motivation for trying to support this kind of information in the convention.
> >>
> >>
> >> My main concern is that "time" is a very important concept and it is
> distinct from "date". Time has a very precise meaning which is recognised
> across scientific domains. I recognise that it may be used interchangeably
> with "date" in many contexts, but the CF convention is built on precisely
> defined terms, so I feel it would be counter-productive to broaden our
> concept of "time" to include "date". On the other hand, introducing a new
> CF name of "date" would make it possible to introduce the sort of
> information that you are talking about.
> >>
> >>
> >> My 2nd concern is the suggestion that we should relax the current
> specification of "units", which allows the comprehensive UDUNITS package to
> be used when comparing variables without different units. I'm puzzled by
> your suggestion the UDUNITS is opaque ... it looks like a pretty well
> maintained library to me. Their definition of "year" as a "tropicalyear"
> agrees with the GNU units library. I can't see why the information you want
> has to go in the "units" attribute (I'm sorry if I have missed your
> explanation on this point). There are two alternatives which I would prefer:
> >>
> >>
> >> (1) try to persuade UDUNITS to accept calendar_year and calendar_month
> as units of "date", or perhaps "calendar_time". I don't think it would be
> worth trying to persuade them to add these of units of "time", because, as
> Karl has mentioned, they are not convertible in the same way other time
> units are. As units of "calendar_time", however, they would fit perfectly
> well into the UNIDATA and CF model.
> >>
> >>
> >> (2) place the information in a different attribute. This is clearly a
> new concept for the convention, so it would be reasonably to add a new
> attribute.
> >>
> >>
> >> I think option (1) is worth trying, it would give a cleaner end result,
> >>
> >>
> >> regards,
> >>
> >> Martin
> >>
> >>
> >>
> >>
> >>
> >>
> >> ________________________________
> >> From: CF-metadata <cf-metadata-bounces at cgd.ucar.edu<mailto:
> cf-metadata-bounces at cgd.ucar.edu>> on behalf of David Blodgett <
> dblodgett at usgs.gov<mailto:dblodgett at usgs.gov>>
> >> Sent: 21 October 2018 13:30
> >> To: B?rring Lars
> >> Cc: cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>
> >> Subject: Re: [CF-metadata] [EXTERNAL] 'months since' and 'years since'
> time units
> >>
> >> Dear Lars,
> >>
> >> Yes. I think we are in agreement. It seems odd to me that CF relies so
> heavily on UDUNITS given the emphasis on non-opaque metadata in the spec
> generally.
> >>
> >> So the relevant section is here:
> >>
> http://cfconventions.org/cf-conventions/cf-conventions.html#time-coordinate
> > NetCDF Climate and Forecast (CF) Metadata Conventions<
> http://cfconventions.org/cf-conventions/cf-conventions.html#time-coordinate
> >
> > cfconventions.org<http://cfconventions.org>
> > The CF conventions generalize and extend the COARDS conventions .The
> extensions include metadata that provides a precise definition of each
> variable via specification of a standard name, describes the vertical
> locations corresponding to dimensionless vertical coordinate values, and
> provides the spatial coordinates of non-rectilinear gridded data.
> >
> >
> >
> >>
> >> I think what we are proposing is to replace the cautionary paragraph
> about use of year and month with an addition to the udunits convention that
> introduces "calendar" to date/time units.
> >>
> >> I think the calendar list<
> http://cfconventions.org/cf-conventions/cf-conventions.html#calendar> is
> sufficient given the addition of a non-udunits interpretation of a time
> axis?
> >>
> >> Now the question is, what is the additional description? I think we
> have plenty of food for that below.
> >>
> >> Is there any disagreement with this approach out there? I expect their
> may be.
> >>
> >> Best,
> >>
> >> - Dave
> >>
> >> On Oct 20, 2018, at 8:27 AM, B?rring Lars <Lars.Barring at smhi.se<mailto:
> Lars.Barring at smhi.se><mailto:Lars.Barring at smhi.se<mailto:
> Lars.Barring at smhi.se>>> wrote:
> >>
> >> Dear David,
> >>
> >> It seems that we pretty much agree, the CF Convention should do better
> than just refer to the current Udunits for time units.
> >>
> >> From a CF Conventions perspective is useful to separate the discussion
> of the different year length in various calendars from the issue of uneven
> length of calendar months (and Udunits somewhat draconian way of solving
> this to little relevance for this community I would argue).
> >>
> >> If the solution to both issues is to adopt the solution you refer to,
> this would be perfectly fine with me. But to me the important thing is that
> the underlying rules should be part of the CF Convention, either by
> expressing them in the CF Conventions document, or by making a reference to
> an external document (as is now done for Udunits).
> >>
> >>
> >>
> >> Kind regards,
> >> Lars
> >>
> >>
> >> ________________________________
> >> Fr?n: David Blodgett [dblodgett at usgs.gov<mailto:dblodgett at usgs.gov
> ><mailto:dblodgett at usgs.gov<mailto:dblodgett at usgs.gov>>]
> >> Skickat: den 20 oktober 2018 13:47
> >> Till: B?rring Lars
> >> Kopia: cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu
> ><mailto:cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>>
> >> ?mne: Re: [CF-metadata] 'months since' and 'years since' time units
> >>
> >> Dear Lars,
> >>
> >> Maybe my point is lost in the reference to an outside source. I think
> CF should support more than udunits. It should support an extended units
> attribute for a variable meant to represent a calendar/time axis.
> >>
> >> That extension should be prepending "calendar" to a udunits date
> string. This is the second option in the NetCDF-Java Common Data Model<
> https://www.unidata.ucar.edu/software/thredds/current/netcdf-java/#home>
> which was adopted from the Environmental Data Abstraction Library<
> https://www.semanticscholar.org/paper/The-Environmental-Data-Abstraction-Library-(-Edal-)-Griffiths-Haines/77c1018b473dbcee44068075a6458c5ef2c2f5a4>,
> both of which support CF but also support non-cf data that exists in the
> community.
> >>
> >> If the developers of udunits want to support the extension, that would
> be great -- they would be coming into line with practices implemented by
> others in the community.
> >>
> >> Regarding the analogy to length units. Let's dig in a bit. Units
> declarations define the function that has to be applied to take one
> quantity and transform it to a reference datum or some other arbitrary
> datum. Time is no different -- the function is just a little different.
> There are plenty of "transformed" quantities that require a specialized
> function to go from the units they are stored in to units that make the
> data comparable or useful in some other context (sigma coordinates for
> example)<https://www.phy.ornl.gov/csep/om/node24.html>. In this case,
> what we are saying is that we have a specialized function for time that has
> real utility and isn't supported by the assumptions in udunits. We even
> have implementations of our specialized function and a way to describe it!!
> >>
> >> All the best,
> >>
> >> - Dave
> >>
> >> On Oct 20, 2018, at 5:39 AM, B?rring Lars <Lars.Barring at smhi.se<mailto:
> Lars.Barring at smhi.se><mailto:Lars.Barring at smhi.se<mailto:
> Lars.Barring at smhi.se>>> wrote:
> >>
> >> Hi all,
> >>
> >> _at_Jim, I think that even within the domain of months and years, a month
> is not necessarily a well behaved unit. Think of precipitation height (mm)
> or sunshine hours (h); these quantities depends on the length of the month.
> Without taking the month length into account one cannot really compare
> precipitation height during January with that of February (10 % difference
> in accumulation period) despite that the difference in the time coordinate
> is 1. If one on the other hand factor out the the length of the months by
> using precipitation flux (kg m-2 s-1), or precipitation rate (mm s-1, mm
> h-1, mm day-1, but not mm month-1), and fraction_of_sunshine_hours (1),
> things suddenly are OK.
> >>
> >> _at_David, I am not familiar with the software you are pointing at, but
> from your description (and my rather limited software engineering skills)
> it certainly sounds like a good way forward/solution. But what I am after
> is that the CF Conventions as such should be more specific and not back
> away from the issue by just referring to Udunits, especially as CF has
> otherwise done such an excellent work in sorting out the sometimes hairy
> issues pertaining to geophysical modelling data and observations. After
> all, the complications from having different calendars is specific to the
> community that CF sets out to support (and sort out...).
> >>
> >> _at_Chris (your last para in earlier post):
> >> "I know that CF refers to udunits for unit definitions, but we really
> need to either allow exceptions or periodically update udunits to meet the
> needs of CF."
> >> Yes and Yes to this suggestion !
> >>
> >>
> >> Lars
> >>
> >> ________________________________
> >> Fr?n: CF-metadata [cf-metadata-bounces at cgd.ucar.edu<mailto:
> cf-metadata-bounces at cgd.ucar.edu><mailto:cf-metadata-bounces at cgd.ucar.edu
> <mailto:cf-metadata-bounces at cgd.ucar.edu>>] f?r Jim Biard [
> jbiard at cicsnc.org<mailto:jbiard at cicsnc.org><mailto:jbiard at cicsnc.org
> <mailto:jbiard at cicsnc.org>>]
> >> Skickat: den 19 oktober 2018 22:32
> >> Kopia: cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu
> ><mailto:cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>>
> >> ?mne: Re: [CF-metadata] 'months since' and 'years since' time units
> >>
> >> Chris,
> >> I see what you are saying about category, yet it is metric. Within the
> domain of months and years (without days, leap days, etc), it is completely
> possible to do exact math, just as we can within the domain of seconds,
> minutes, hours, and days. The problem arises when you try to bridge between
> these two domains. The current UDUNITS tries to keep it all in one domain,
> with the result that values in months and years become problematic.
> >> Grace and peace,
> >> Jim
> >>
> >> On 10/19/18 3:53 PM, Chris Barker - NOAA Federal wrote:
> >> I remember this coming up on this list a while back.
> >>
> >> Yes, the concept of e.g. monthly averaged data is useful.
> >>
> >> But in that case, a "month" is really a category, not a continuous time
> variable.
> >>
> >> So we need a different way if describing it than a time access with
> units of month...
> >>
> >> -CHB
> >>
> >> Sent from my iPhone
> >>
> >> On Oct 19, 2018, at 8:45 AM, Jim Biard <jbiard at cicsnc.org<mailto:
> jbiard at cicsnc.org><mailto:jbiard at cicsnc.org<mailto:jbiard at cicsnc.org>>>
> wrote:
> >>
> >> I like how you have described the issue, Chris.
> >> Using month in anything except a 360-day calendar (assuming the month
> is defined correctly for that calendar) produces erroneous results if you
> try to do anything but math that remains in those units - such as convert a
> month count to a date. The same goes for physical years.
> >> And yet... If we are writing data collected on monthly, seasonal, or
> other large-scale time binnings (when doing climatologies, for example),
> month and year become regularized - that is, they become metric within the
> context of that data, and it would be so much more human-friendly to be
> able to work directly in months, seasons, and years, regardless of whether
> that is a real year, a 365 day year, or a 360 day year.
> >> It seems to me that time recorded in "month and larger" terms really
> represents a different thing, separate from 'time' as defined by CF. I see
> two different solutions that could be based around a new standard name of
> 'date', as Martin Juckes suggested.
> >>
> >> * Declare that the values for a standard name of 'date' are date
> strings following a YYYY-MM-DD format (and perhaps something for megayears,
> etc that would be useful for paleo people).
> >> * Declare that the values for a standard name of 'date' would use
> units of calendar_month, calendar_season, and calendar_year (or whatever
> names people like best). These units are entirely convertible between each
> other, and would need to be added to UDUNITS. UDUNITS would not convert
> between these units and time units. People would be free to write code that
> could do the conversion between time and date, but there wouldn't be any
> particular way that would be considered standard.
> >>
> >> Grace and peace,
> >>
> >> Jim
> >>
> >> On 10/19/18 11:13 AM, Chris Barker - NOAA Federal wrote:
> >>
> >> Calendars are a mess- both because the earth's rotation is not
> >> human-frendly, and because of human legacy.
> >>
> >> So we need to accept that, and not try to use calendars as though they
> >> are logical units for time.
> >>
> >> I like to think about it this way: there are time operations: "this
> >> much time has passed", and there are calendar operations: "this day
> >> next month".
> >>
> >> Calendar operations require a lot of machinations and a well defined
> >> calendar. Time operations are straightforward and behave "sensibly"
> >> with the usual math.
> >>
> >> Also - operations belong in libraries, not data files. CF should use
> >> only well defined units.
> >>
> >> Personally, I think udunits should never have defined "months" or
> >> "years" as a time unit. But in any case, CF can at least highly
> >> discourage, if not ban, their use.
> >>
> >> Possible exception: for "artificial" Calendars, a month can be well
> >> defined, but then the unit should be called something like
> >> "30_day_month", other well defined name.
> >>
> >> I know that CF refers to udunits for unit definitions, but we really
> >> need to either allow exceptions or periodically update udunits to meet
> >> the needs of CF.
> >>
> >> -CHB
> >>
> >>
> >>
> >>
> >> On Oct 19, 2018, at 5:13 AM, B?rring Lars <Lars.Barring at smhi.se<mailto:
> Lars.Barring at smhi.se>><mailto:Lars.Barring at smhi.se<mailto:
> Lars.Barring at smhi.se>> wrote:
> >>
> >> Dear all,
> >>
> >> I agree with Jonathan's wish for a more well-behaved Earth in the
> planetary system :-)
> >>
> >> However, awaiting this I think that we have two issues before us:
> >>
> >> 1. The fact that different datasets fundamentally are based on
> different length of a year, while Udunits defines a year to be the real
> world tropical year.
> >>
> >> 2. The common language notion of months of different length (and for
> February varying), while Udunits defines the length of a month to be one
> twelfth of the real world tropical year.
> >>
> >>
> >> I fully agree that for datasets from one source (having one calendar)
> the warning against using "month_since..." and "year_since..." is very
> well motivated. But, as Martin points out, when combining data based on
> different calendars the easiest /best/most relevant way is to use one of
> these. Having said that I will in turn go into some detail with the two
> issues.
> >>
> >>
> >> 1. Regarding the length of the year, which is what my previous posts
> concerned, I think that we have three options:
> >> A) Convince Udunits to change the behaviour of the year unit to depend
> on the calendar. To my mind this would actually solve the problem stated in
> the initial post of this thread. Whether this option s possible, have a
> chance to fly, or is something that the CF community actually want I do not
> now. But here is the idea for you to consider. If this solution is
> implemented, it would mean changing the text in Section 4.4.
> >> B) In the CF Conventions specify that the definition of the length of a
> year deviates from the one used in Udunits. This would mean changing the
> text in Section 4.4. Again, once this change has penetrated into software
> libraries, this would solve the initial poster's problem.
> >> C) Leave things as they are, preferably with some additional language
> in Section 4.4 to explain when unit year might be useful and when not to
> use it.
> >>
> >> Of course, there might be other options that I have not though of.
> Personally I advocate A) or B), because to my mind the impact of having
> defined different calendars is not fully implemented in CF because the
> different model calendar _do_ imply different length of the year. Moreover,
> at least for the 360_day calendar the Udunits definition of a month will
> then coincide with what is expected from that calendar.
> >>
> >> 2. The obvious root of the problem here is of course the different
> months' lengths in the real world calendar. Having said that, what adds to
> the confusion is that Udunits has chosen the name "month" for the unit
> year/12. I will put aside the problem of different month lengths and only
> consider the Udunits months, which I here will call "month12" for clarity,
> and focus on the implications of having data from different calendars being
> merged (think ensemble summary) and described using "month12_since" (where
> the time coordinate takes on integer values). For all calendars, but
> 360_day, this implies that a fractional day goes into the month12ly
> statistic. If this is a problem one may one may postulate that the
> fractional day belongs to the month12 where the larger part belongs. If
> this is complemented with the notion of calendar_month it may be
> reasonable to restrict at least the latter, or both, to only take on
> integer values, to avoid hairy questions like ones that Jonathan asked
> "What does 1 calendar_month since 31 January mean? What does 7.23
> calendar_months since 31 January mean?"
> >>
> >>
> >> Kind regards,
> >> Lars
> >>
> >> PS. For those of you without a Scandinavian keyboard; you might
> relieved to know that my first name is Lars (and B?rring is family name,
> despite what the email alias might suggest)
> >>
> >>
> >> ________________________________________
> >> Fr?n: CF-metadata [cf-metadata-bounces at cgd.ucar.edu<mailto:
> cf-metadata-bounces at cgd.ucar.edu>] för Taylor, Karl E. [
> taylor13 at llnl.gov<mailto:taylor13 at llnl.gov>]
> >> Skickat: den 19 oktober 2018 06:14
> >> Till: cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>
> >> ?mne: Re: [CF-metadata] 'months since' and 'years since' time units
> >>
> >> Dear all,
> >>
> >> I won't make any recommendations for udunits, but I will comment on the
> >> CF-conventions.
> >>
> >> In general, I think we should discourage usage of calendar month as a
> >> unit of measurement because for the real world, these are only defined
> >> for 12 special periods each year (the beginning to the end of each
> >> calendar month) and the "<mailto:Kindregards<mailto:Kindregards
> >,LarsPS.ForthoseofyouwithoutaScandinaviankeyboard;youmightrelievedtoknowthatmyfirstnameisLars%28andB%C3%A4rringisfamilyname,despitewhattheemailaliasmightsuggest%29________________________________________Fr%C3%A5n:CF-metadata[
> cf-metadata-bounces at cgd.ucar.edu<mailto:cf-metadata-bounces at cgd.ucar.edu
> >]förTaylor,KarlE.[taylor13 at llnl.gov<mailto:taylor13 at llnl.gov>]
> Skickat:den19oktober201806:14Till:cf-metadata at cgd.ucar.edu<mailto:
> Skickat%3Aden19oktober201806%3A14Till%3Acf-metadata at cgd.ucar.edu>%C3%84mne:Re:[CF-metadata]%27monthssince%27and%27yearssince%27timeunitsDearall,Iwon%27tmakeanyrecommendationsforudunits,butIwillcommentontheCF-conventions.Ingeneral,Ithinkweshoulddiscourageusageofcalendarmonthasaunitofmeasurementbecausefortherealworld,theseareonlydefinedfor12specialperiodseachyear%28thebeginningtotheendofeachcalendarmonth%29andthe>unit"
> is not constant throughout the year.
> >> Nevertheless there are some good arguments for considering adopting a
> >> special calendar month unit by the CF conventions, but only for limited
> >> and very specific purposes. (I'll refer to these new units as
> >> "calendar_months" in the following, with the understanding that the unit
> >> will depend on the calendar adopted and will in general vary for
> >> different months of the year and depend on whether or not the year is a
> >> leap year.) Here are reasons (already articulated by others) why we
> >> might want to adopt "calendar_months" as a unit:
> >>
> >> 1) there are existing data sets with monthly-mean data and a
> >> time-coordinate that is supposed to indicate how many calendar months
> >> have passed since some base month/year. Such data sets are not easily
> >> accommodated by CF.
> >> 2) for some judiciously selected reference times, coordinates expressed
> >> in "calendar_months since" can be easily generated without the help of
> >> calendar-aware time-translation algorithms. For example, given units of
> >> "calendar_months since 2001-01-01" we can trivially generate the
> >> coordinate values for the middle of each month: 0.5, 1.5, 2.5, etc. It
> >> would be much more difficult to generate the coordinate values in units
> >> of "days since ...".
> >> 3) monthly mean data sets with time coordinates based on different
> >> calendars can more easily be compared because if all the data sets
> >> adopted the same reference time, then comparable months would have the
> >> same coordinate values, independent of the actual month length defined
> >> by different calendars. [In contrast, if the time coordinate were given
> >> in units of "days since ... ", the coordinate values would depend on the
> >> calendar.]
> >>
> >> If we define a unit of "calendar_months since ..." we would need to
> >> address Jonathan's point that it is not immediately obvious how to
> >> handle fractions of months and reference times different from the
> >> beginning of a month. One approach we should consider is to restrict
> >> use of calendar_month units to datasets reporting monthly values only
> >> (not, daily, annual, hourly, etc.) Also for simplicity we could
> >> restrict the reference time to be the beginning of a calendar year
> >> (i.e., January 1 at 0:0:0 for some specified year). If we did this, it
> >> would be relatively easy to define what we mean by fractions of months
> >> and it would also be easy to generate the values needed to define the
> >> time-coordinates and the cell_bounds or the bounds needed to define
> >> climatologies. [It would be almost as easy if we allowed the reference
> >> time to be the beginning of *any* month, but let's consider the more
> >> restrictive "beginning of a year" option first.]
> >>
> >> I note that using calendar_month units to report data at intervals other
> >> than monthly intervals offers no advantages. For example different
> >> fractional month increments would have to be used to report data at an
> >> invariant daily interval. This would seem to introduce complexity to a
> >> simple time-coordinate and is why I would restrict use of calendar_month
> >> units to monthly data.
> >>
> >> Since months are not all equal in length, the interval of times
> >> represented by the same fraction may differ between months. For a
> >> Gregorian calendar, half-way through the month of January would be noon
> >> on the 16th of January (15.5 days from the beginning of the month),
> >> whereas half-way through the month of June would be the 16th of June at
> >> 00:00:00 (15.0 days from the beginning of the month. Thus 0.5 months
> >> since 2001-01-01 would be 2001-01-16 12:00:00 and 5.5 months since
> >> 2001-01-01 would be 2001-06-16 00:00:00. Also note that the date
> >> corresponding to the middle of February depends on whether the year is a
> >> leap year or not.
> >>
> >> Of course for a different calendar (e.g., 360_day), the dates
> >> corresponding to 0.5 months since 2001-01-01 and 5.5 months since
> >> 2001-01-01 would be different from those for the Gregorian calendar.
> >>
> >> The bottom line is that under the above restrictions, it is easy to
> >> convert fractions of months to dates (and also to alternative units such
> >> as "days since ...").
> >>
> >> Common types of "monthly" data sets are:
> >>
> >> 1) reports of monthly statistics computed from multiple samples within
> >> each month (e.g., means, standard deviation of daily values, maximum
> >> daily mean; requires cell_bounds)
> >>
> >> 2) reports of monthly accumulations (e.g. monthly precipitation amount;
> >> requires cell_bounds)
> >>
> >> 3) monthly climatologies (requiring climatology attribute pointing to
> >> climatology bounds)
> >>
> >> For all of the above the bounds coincide with the beginning and end of
> >> each month so with the reference time restricted to the beginning of a
> >> year, the bounds will all be integer values of "months since". The
> >> coordinate value for any of the above can be assigned any value in the
> >> interval defined by the bounds. For monthly statistics one might choose
> >> the middle of each month so coordinate values would be numbers like 0.5,
> >> 1.5, 2.5, etc. For monthly accumulations one might prefer to use the
> >> time at the end of each interval to represent the coordinate value
> >> (e.g., 1.0, 2.0, 3.0, etc.).
> >>
> >> I understand that defining a unit of calendar_months is not compatible
> >> with udunits, but I think we can rely on tools other than udunits to
> >> handle more generally this new unit and the various CF conventions
> >> calendars to convert coordinate values to other time units like "days
> >> since ...".
> >>
> >> Perhaps the biggest argument in favor of introducing a calendar_month
> >> unit is that it should make it much easier for data providers to
> >> generate correct time coordinates for data reported at monthly
> >> intervals. Regardless of calendar, I think it is easy to generate
> >> monthly time coordinates under the current CF standard that are simply
> >> wrong. In contrast, everyone should be able to trivially create a
> >> coordinate axis with values like (1, 2, 3, .... ) or (0.5, 1.5, 2.5,
> >> ...) without making a mistake.
> >>
> >> best regards,
> >> Karl
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> On 10/18/18 10:58 AM, Jonathan Gregory wrote:
> >> Dear Martin
> >>
> >> The definition of a calendar_month unit would also need rules about
> calendar
> >> arithmetic e.g. What does 1 calendar_month since 31 January mean? What
> does
> >> 7.23 calendar_months since 31 January mean?
> >>
> >> Best wishes
> >>
> >> Jonathan
> >>
> >>
> >> ----- Forwarded message from Martin Juckes - UKRI STFC <
> martin.juckes at stfc.ac.uk<mailto:martin.juckes at stfc.ac.uk>><mailto:
> martin.juckes at stfc.ac.uk<mailto:martin.juckes at stfc.ac.uk>> -----
> >>
> >>
> >>
> >> Date: Thu, 18 Oct 2018 16:33:28 +0000
> >> From: Martin Juckes - UKRI STFC <martin.juckes at stfc.ac.uk<mailto:
> martin.juckes at stfc.ac.uk>><mailto:martin.juckes at stfc.ac.uk<mailto:
> martin.juckes at stfc.ac.uk>>
> >> To: Jonathan Gregory <j.m.gregory at reading.ac.uk<mailto:
> j.m.gregory at reading.ac.uk>><mailto:j.m.gregory at reading.ac.uk<mailto:
> j.m.gregory at reading.ac.uk>>, "cf-metadata at cgd.ucar.edu<mailto:
> cf-metadata at cgd.ucar.edu>"<mailto:cf-metadata at cgd.ucar.edu<mailto:
> cf-metadata at cgd.ucar.edu>>
> >> <cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>><mailto:
> cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>>
> >> Subject: Re: [CF-metadata] 'months since' and 'years since' time units
> >>
> >> Dear Jonathan,
> >>
> >>
> >> I think you could go further and disallow the use of "month" or "year"
> as a time unit when the calendar is not standard.
> >>
> >>
> >> If the "ncdump -t" option produces what the user expects when he has
> units "months since 1900-01-01" and a 360 day calendar, then it is going
> to be inconsistent with the current convention.
> >>
> >>
> >> I still feel that there is an argument for enabling the storage of
> information in user months in the files. E.g. I wish to compare monthly
> mean data from 20 climate models which use a range of different calendars.
> The mean across the models is not on any specific calendar ... I could
> pretend it is and use units of "days since ...", but the mappings from
> input time coordinates to output time coordinates then become rather
> complex, when they should be trivial. Having a "date" standard name which
> allowed the input data to have a "calendar_month" coordinate would solve
> this (and I think Klaus's suggestion would also solve it),
> >>
> >>
> >> regards,
> >>
> >> Martin
> >>
> >> ________________________________
> >> From: CF-metadata <cf-metadata-bounces at cgd.ucar.edu<mailto:
> cf-metadata-bounces at cgd.ucar.edu>><mailto:cf-metadata-bounces at cgd.ucar.edu
> <mailto:cf-metadata-bounces at cgd.ucar.edu>> on behalf of Jonathan Gregory <
> j.m.gregory at reading.ac.uk<mailto:j.m.gregory at reading.ac.uk>><mailto:
> j.m.gregory at reading.ac.uk<mailto:j.m.gregory at reading.ac.uk>>
> >> Sent: 18 October 2018 17:10:46
> >> To: cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu><mailto:
> cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>>
> >> Subject: Re: [CF-metadata] 'months since' and 'years since' time units
> >>
> >> Dear all
> >>
> >> This is an interesting discussion, and I agree that's a tricky subject.
> If only
> >> we could have a well-behaved Earth which orbited the sun in an integral
> and
> >> easily factorisable number of days!
> >>
> >> So far I still think that we should not change the way we interpret the
> units
> >> string. It's in udunits format, and should be interpreted according to
> the
> >> calendar attribute. I would suggest that it's helpful to regard time
> coords as
> >> *encoded* and not necessarily easy for humans to read. It's certainly
> nice to
> >> see "time=1, 2, 3, ..." months since a reference date - that is easy to
> read -
> >> but when you get to 747 or 4689 months since a reference date, you
> don't know
> >> what it means any more (unless you're extremely good at mental
> arithmetic), and
> >> you might as well encode it as days.
> >>
> >> The antidote to inconvenient encoding is convenient software. For
> example,
> >> could cftime allow the user to construct a time coordinate variable
> with a
> >> spacing of calendar months, but encode it in the netCDF file in days?
> Then it's
> >> transparent. Similarly, time coordinate variables can be decoded into
> human-
> >> readable strings by calendar-aware software. It seems to me that this
> isn't
> >> different in principle from using ncdump to read a netCDF file, rather
> than
> >> insisting it should be intelligible when read in hexadecimal. In fact,
> ncdump
> >> itself has a -t option, which should help, according to the man page:
> >>
> >> "-t controls display of time data, if stored in a variable that uses a
> udunits
> >> compliant time representation such as `days since 1970-01-01' or
> `seconds since
> >> 2009-03-15 12:01:17' .... If this option is specified, time data values
> are
> >> displayed as human-readable date-time strings rather than numerical
> values,
> >> interpreted in terms of a `calendar' variable attribute, if specified.
> ...
> >> Calendar attribute values interpreted with this option include the CF
> >> Conventions values `gregorian' or `standard', `proleptic_gregorian',
> `noleap'
> >> or `365_day', `all_leap' or `366_day', `360_day', and `julian'."
> >>
> >> I agree with comments that if we introduced new units such as
> calendar_month
> >> or 30day_month, people might well not use them, and would still be
> disappointed
> >> that "month" wasn't what they expected.
> >>
> >> The CF conformance document has a recommendation that "year" and
> "month" should
> >> be used "with caution". I don't what the CF checker currently does with
> this.
> >> We could change it to a recommendation that they should *not* be used,
> in which
> >> case the checker would give a warning if they were.
> >>
> >> Best wishes
> >>
> >> Jonathan
> >>
> >> ----- Forwarded message from B?rring Lars <Lars.Barring at smhi.se<mailto:
> Lars.Barring at smhi.se>><mailto:Lars.Barring at smhi.se<mailto:
> Lars.Barring at smhi.se>> -----
> >>
> >>
> >>
> >> Date: Thu, 18 Oct 2018 13:31:10 +0000
> >> From: B?rring Lars <Lars.Barring at smhi.se<mailto:Lars.Barring at smhi.se
> >><mailto:Lars.Barring at smhi.se<mailto:Lars.Barring at smhi.se>>
> >> To: Martin Juckes - UKRI STFC <martin.juckes at stfc.ac.uk<mailto:
> martin.juckes at stfc.ac.uk>><mailto:martin.juckes at stfc.ac.uk<mailto:
> martin.juckes at stfc.ac.uk>>, David Blodgett
> >> <dblodgett at usgs.gov<mailto:dblodgett at usgs.gov>><mailto:
> dblodgett at usgs.gov<mailto:dblodgett at usgs.gov>>, Ryan Abernathey <
> ryan.abernathey at gmail.com<mailto:ryan.abernathey at gmail.com>><mailto:
> ryan.abernathey at gmail.com<mailto:ryan.abernathey at gmail.com>>
> >> Cc: "cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>"<mailto:
> cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>> <
> cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>><mailto:
> cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>>
> >> Subject: Re: [CF-metadata] 'months since' and 'years since' time units
> >>
> >> Dear Martin, David, all,
> >>
> >> As Klaus points out, the aim of my suggestion is to make software using
> CF aware of the fact that the unit "year" is different depending on which
> calendar the model is implementing. To give an example:
> >> If I want to know when the global average temperature has increased by
> 1.5C, or 4C, above pre-industrial time in the CMIP5 ensemble I will get
> answers as a timedelta in days. As this is not really helpful I might feel
> inclined to convert this to years, but now UDUNITS definition of year is
> not helpful for those models having a 360_day or 365_day calendar. However,
> with the calendar-aware definition of year such a calculation would be
> supported without having to deal with it manually.
> >>
> >> Now, on to the discussion about months. Before my previous post I
> quickly read through extensive exchange on this list back in 2011, so I
> really appreciate David's comment that it is a complex subject. And that is
> the reason for why I suggested is always month is always a year / 12. So,
> here is an attempt to summarise the suggestion in a different way:
> >>
> >> * standard and proleptic_gregorian calendars (status quo):
> >> o the number of days in a month is not an integer
> >> o same issue with respect to ordinary (western) world months.
> >>
> >> * 365_day calendar:
> >> + the number of seconds in a month would change from being
> "ill-defined (?)" as 84600 * 365.242198781 / 12 = 2574957.50141, to more
> properly 84600 * 365 / 12 = 2573250
> >> o same issue with respect to ordinary (western) world months.
> >>
> >> * 360_calendar:
> >> + the number of seconds in a month would change from being "very
> ill-defined (?)" as 84600 * 365.242198781 / 12 = 2574957.50141, to more
> properly 84600 * 360 / 12 = 2538000
> >> + the number of days in a month is an integer; 12 * 30 * 84600 =
> 2538000
> >> + the definition of a month is consistent with what is expected in the
> "360_day world"
> >> o same issue with respect to ordinary (western) world months.
> >>
> >> That is, even though the suggestion certainly do not solve everything
> (of course!), the only argument against it, that I can see, is the work to
> tease out the details and implement it in software packages. As was
> extensively discussed in the 2011 threads, the real problem is the varying
> length of the western world calendar months. But that is the topic for
> another thread.
> >>
> >>
> >> Kind regards,
> >> Lars
> >>
> >> ________________________________
> >> Fr?n: CF-metadata [cf-metadata-bounces at cgd.ucar.edu<mailto:
> cf-metadata-bounces at cgd.ucar.edu><mailto:cf-metadata-bounces at cgd.ucar.edu
> <mailto:cf-metadata-bounces at cgd.ucar.edu>>] f?r David Blodgett [
> dblodgett at usgs.gov<mailto:dblodgett at usgs.gov><mailto:dblodgett at usgs.gov
> <mailto:dblodgett at usgs.gov>>]
> >> Skickat: den 18 oktober 2018 13:58
> >> Till: Ryan Abernathey
> >> Kopia: cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu
> ><mailto:cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>>
> >> ?mne: Re: [CF-metadata] 'months since' and 'years since' time units
> >>
> >> Dear Ryan, All,
> >>
> >> I hesitate to chime in on this thread as I know just how fraught this
> topic can be, but then I think I know how fraught it can be so may have
> something to offer. My experience is at the intersection of climatological
> models and landscape models that are calibrated with "real" data. I've
> worked with daily and monthly time series model output and interpolated
> weather products that needs to match up to observations but uses a noleap
> or 360 calendar. It's an enormous pain and we as a community should do
> better. -- so the business case for taking this complexity head on is there!
> >>
> >> One resource I've found useful over the years is the [CDM
> implementation](
> https://www.unidata.ucar.edu/software/thredds/current/netcdf-java/CDM/CalendarDateTime.html
> )
> >> ut this does not
> >> There are two factors at play.
> >>
> >> 1) Adding "calendar" to a udunits string avoids conversion to a number
> of shorter time increments for long time increments (e.g. seconds per
> month). It keeps things in the declared time units so you hit the precise
> date boundaries you would expect.
> >> 2) The "calendar" attribute gets you to how to interpret the datum of
> the time axis.
> >>
> >> Especially relevant to this thread is:
> >>
> >> * uniform30day or 360_day = All years are 360 days divided into 30
> day months.
> >>
> >> With these two, I think the problems here are solved. However,
> inevitably, people will omit the addition attribute for calendar or fall
> back on normal "months since ..." when they actually mean "calendar months
> since ..." and tell us 'why would you interpret my data that way it makes
> no sense?!?' This is perfectly parallel to spatial coordinates where people
> don't declare a datum for their latitude/longidute coordinates. Without
> that information one can not use the information with a level of precision
> that some use cases require.
> >>
> >> What I'm getting at is that CF should probably:
> >> 1) adopt enough attribute precision to fully describe what we are
> trying to convey
> >> 2) make said attributes required or declare sensible defaults that
> reduce ambiguity when users come knocking.
> >>
> >> That said, I've had no success pushing the community to accept that
> there should be a default lat/lon datum for software developers to go on
> and I would not doubt that the same will be true here as ambiguity and
> uncertainty is better than dead wrong in many cases. My stance is that we
> should all be dead wrong for the same reason rather than each implementor
> making an arbitrary decision so we all get different answers (more
> ambiguity) from our software du-jour.
> >>
> >> All the best,
> >>
> >> Dave
> >>
> >>
> >> On Oct 18, 2018, at 6:08 AM, Martin Juckes - UKRI STFC <
> martin.juckes at stfc.ac.uk<mailto:martin.juckes at stfc.ac.uk><mailto:
> martin.juckes at stfc.ac.uk<mailto:martin.juckes at stfc.ac.uk>><mailto:
> martin.juckes at stfc.ac.uk<mailto:martin.juckes at stfc.ac.uk>><mailto:
> martin.juckes at stfc.ac.uk<mailto:martin.juckes at stfc.ac.uk>>> wrote:
> >>
> >> Hello All,
> >>
> >>
> >> I think the UNIDATA pull request referenced Jeff (
> https://github.com/Unidata/cftime/pull/69) is mis-quoting the CF
> Convention. As far as I can see, Unidata says that a month is exactly one
> 12th of a year, and CF inherits this -- with the statement "For similar
> reasons the unit month, which is defined in udunits.dat<
> http://www.unidata.ucar.edu/software/udunits/><
> http://www.unidata.ucar.edu/software/udunits/> to be exactly year/12,
> should also be used with caution."
> >>
> >>
> >> I can't see the difference between Lars's suggestion and the status
> quo. In UNIDATA a day is clearly defined as "period of time equal to 24
> hours", which gives 84600 seconds.
> >>
> >> regards,
> >> Martin
> >>
> >>
> >>
> >> ________________________________
> >> From: CF-metadata <cf-metadata-bounces at cgd.ucar.edu<mailto:
> cf-metadata-bounces at cgd.ucar.edu><mailto:cf-metadata-bounces at cgd.ucar.edu
> <mailto:cf-metadata-bounces at cgd.ucar.edu>><mailto:
> cf-metadata-bounces at cgd.ucar.edu<mailto:cf-metadata-bounces at cgd.ucar.edu
> >><mailto:cf-metadata-bounces at cgd.ucar.edu<mailto:
> cf-metadata-bounces at cgd.ucar.edu>>> on behalf of B?rring Lars <
> Lars.Barring at smhi.se<mailto:Lars.Barring at smhi.se><mailto:
> Lars.Barring at smhi.se<mailto:Lars.Barring at smhi.se>><mailto:
> Lars.Barring at smhi.se<mailto:Lars.Barring at smhi.se>><mailto:
> Lars.Barring at smhi.se<mailto:Lars.Barring at smhi.se>>>
> >> Sent: 18 October 2018 09:29:50
> >> To: Ryan Abernathey; whitaker.jeffrey at gmail.com<mailto:
> whitaker.jeffrey at gmail.com><mailto:whitaker.jeffrey at gmail.com<mailto:
> whitaker.jeffrey at gmail.com>><mailto:whitaker.jeffrey at gmail.com<mailto:
> whitaker.jeffrey at gmail.com>><mailto:whitaker.jeffrey at gmail.com<mailto:
> whitaker.jeffrey at gmail.com>>; cf-metadata at cgd.ucar.edu<mailto:
> cf-metadata at cgd.ucar.edu><mailto:cf-metadata at cgd.ucar.edu<mailto:
> cf-metadata at cgd.ucar.edu>><mailto:cf-metadata at cgd.ucar.edu<mailto:
> cf-metadata at cgd.ucar.edu>><mailto:cf-metadata at cgd.ucar.edu<mailto:
> cf-metadata at cgd.ucar.edu>>
> >> Subject: Re: [CF-metadata] 'months since' and 'years since' time units
> >>
> >> Hi,
> >>
> >> I have have come to think about this from a somewhat different
> perspective. For some analyses, as well as when calculating certain derived
> climatological statistics (aka climate indices), using datasets based on
> different calendars the problem becomes obvious.
> >>
> >> In the model world of a 365-day GCM one year _is_ 365 days, and in a
> 360_day GCM a year _is_ 360 days. In the case of a gregorian/standard
> calendar GCM I am not sure whether it is 365.25 or 365.242198781 but this
> is in practice less of a problem.
> >>
> >> For datasets based non-standard calendars imposing the current UDUNITS
> definition of a year leads to complications that require workarounds if one
> is interested in for example the time elapsed until something happens or
> the duration of some (long-lasting) events. One way to partly mitigate
> these issues would be to use the time unit of years_since_START or
> months_since_START, but this is warned against in the CF Conventions and
> may software tools do not support it .
> >>
> >> The fundamental issue is the inconsistency between the GCM year and the
> UDUNITS year. So I would like to call on the wisdom of this list to see
> whether the CF Convention could include a modification to the definition of
> a year and month:
> >>
> >> * standard calendar (no change)
> >> 1 day = 84600 seconds
> >> 1 year = 365.242198781 days
> >> 1 month = 365.242198781 / 12 days
> >>
> >> * 365_day calendar
> >> 1 day = 84600 seconds
> >> 1 year = 365 days
> >> 1 month = 365 / 12 days
> >>
> >> * 360_day calendar
> >> 1 day = 84600 seconds
> >> 1 year = 360 days
> >> 1 month = 360 / 12 days
> >>
> >> That is, the seconds per day ratio is not changed thus maintaining the
> consistency to other SI units. And, for the 360_day calendar month follows
> the suggestion by Ryan and Jeffrey.
> >>
> >>
> >> Kind regards,
> >> Lars
> >>
> >> --
> >> Lars B?rring
> >>
> >> FDr, Forskare
> >> PhD, Research Scientist
> >>
> >> SMHI / Swedish Meteorological and Hydrological Institute
> >> Rossby Centre
> >> SE - 601 76 NORRK?PING
> >> Tel / Phone: +46 (0)11 495 8604
> >> Fax: +46 (0)11 495 8001
> >> Bes?ksadress / Visiting address: Folkborgsv?gen 17
> >> ________________________________
> >> Fr?n: CF-metadata [cf-metadata-bounces at cgd.ucar.edu<mailto:
> cf-metadata-bounces at cgd.ucar.edu><mailto:cf-metadata-bounces at cgd.ucar.edu
> <mailto:cf-metadata-bounces at cgd.ucar.edu>><mailto:
> cf-metadata-bounces at cgd.ucar.edu<mailto:cf-metadata-bounces at cgd.ucar.edu
> >><mailto:cf-metadata-bounces at cgd.ucar.edu<mailto:
> cf-metadata-bounces at cgd.ucar.edu>>] f?r Ryan Abernathey [
> ryan.abernathey at gmail.com<mailto:ryan.abernathey at gmail.com><mailto:
> ryan.abernathey at gmail.com<mailto:ryan.abernathey at gmail.com>><mailto:
> ryan.abernathey at gmail.com<mailto:ryan.abernathey at gmail.com>><mailto:
> ryan.abernathey at gmail.com<mailto:ryan.abernathey at gmail.com>>]
> >> Skickat: den 17 oktober 2018 21:22
> >> Till: whitaker.jeffrey at gmail.com<mailto:whitaker.jeffrey at gmail.com
> ><mailto:whitaker.jeffrey at gmail.com<mailto:whitaker.jeffrey at gmail.com
> >><mailto:whitaker.jeffrey at gmail.com<mailto:whitaker.jeffrey at gmail.com
> >><mailto:whitaker.jeffrey at gmail.com<mailto:whitaker.jeffrey at gmail.com>>
> >> Kopia: cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu
> ><mailto:cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu
> >><mailto:cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu
> >><mailto:cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>>
> >> ?mne: Re: [CF-metadata] 'months since' and 'years since' time units
> >>
> >> Hi everyone,
> >>
> >> I am that user, and I'm new to this mailing list. Thank you all for
> your work on CF conventions. It's such a valuable effort!
> >>
> >> I want to note that this was inspired by the proliferation of datasets
> in the wild that use "month" as their units. For example, nearly all of the
> IRI Data Library does this, in conjunction with a 3"60_day" calendar
> (example:
> https://iridl.ldeo.columbia.edu/SOURCES/.NOAA/.NCEP-NCAR/.CDAS-1/.MONTHLY/.Diagnostic/.surface/.temp/
> ).
> >>
> >> My impression from talking to data providers is that no one is using
> "360_day" calendar and "months" as units, and then expecting "months" to be
> interpreted as 365.242198781/12 days. They all expect it to be interpreted
> as 30 days. While there are various workarounds that can be used at
> different levels of the software stack, the best solution, IMHO, is to
> explicitly allow in CF conventions what Jeff proposed: "months and years be
> interpreted as calendar months and years for those calendars where they
> have a fixed length". I don't think this will break existing applications.
> >>
> >> Thanks,
> >> Ryan
> >>
> >> On Wed, Oct 17, 2018 at 3:06 PM Jeffrey Whitaker <
> whitaker.jeffrey at gmail.com<mailto:whitaker.jeffrey at gmail.com><mailto:
> whitaker.jeffrey at gmail.com<mailto:whitaker.jeffrey at gmail.com>><mailto:
> whitaker.jeffrey at gmail.com<mailto:whitaker.jeffrey at gmail.com>><mailto:
> whitaker.jeffrey at gmail.com<mailto:whitaker.jeffrey at gmail.com>><mailto:
> whitaker.jeffrey at gmail.com<mailto:whitaker.jeffrey at gmail.com>><mailto:
> whitaker.jeffrey at gmail.com<mailto:whitaker.jeffrey at gmail.com>>> wrote:
> >> Hi: I'm a developer of the 'cftime' python package (
> https://github.com/Unidata/cftime). A user submitted a pull request (
> https://github.com/Unidata/cftime/pull/69) that implements support for a
> 30-day calendar month time unit for the '360_day' CF calendar. Although
> using a 'month' time unit is a tricky proposition in general, for this
> calendar it seems straightforward since every month has the same length.
> However, in the discussion of the pull request it was pointed out that CF
> expects that "the value of the units attribute is a string that can be
> recognized by UNIDATA's Udunits package", and that UDUNITS defines a month
> as 365.242198781/12 days. My question is this - is it reasonable for our
> python package to make an exception to this rule for the 360_day calendar?
> More generally, can months and years be interpreted as calendar months and
> years for those calendars where they have a fixed length, or will this
> deviate from the existing CF conventions and break existing applications?
> >>
> >> Regards, Jeff
> >>
> >> --
> >> Jeffrey S. Whitaker
> >> NOAA/OAR/PSD R/PSD1
> >> 325 Broadway, Boulder, CO, 80305-3328
> >> Phone: (303)497-6313
> >> FAX: (303)497-6449
> >>
> >> _______________________________________________
> >> CF-metadata mailing list
> >> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu><mailto:
> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>><mailto:
> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>><mailto:
> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>><mailto:
> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>><mailto:
> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>>
> >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> >> _______________________________________________
> >> CF-metadata mailing list
> >> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu><mailto:
> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>><mailto:
> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>><mailto:
> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>>
> >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> >>
> >> _______________________________________________
> >> CF-metadata mailing list
> >> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu><mailto:
> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>>
> >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> >>
> >>
> >> ----- End forwarded message -----
> >> _______________________________________________
> >> CF-metadata mailing list
> >> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu><mailto:
> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>>
> >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> >>
> >>
> >> ----- End forwarded message -----
> >> _______________________________________________
> >> CF-metadata mailing list
> >> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu><mailto:
> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>>
> >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> >>
> >>
> >> _______________________________________________
> >> CF-metadata mailing list
> >> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu><mailto:
> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>>
> >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> >> _______________________________________________
> >> CF-metadata mailing list
> >> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu><mailto:
> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>>
> >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> >>
> >>
> >> _______________________________________________
> >> CF-metadata mailing list
> >> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu><mailto:
> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>>
> >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> >>
> >>
> >> --
> >> [CICS-NC]<http://www.cicsnc.org/>Visit us on
> >> Facebook<http://www.facebook.com/cicsnc> Jim Biard
> >> Research Scholar
> >> Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/
> >
> >> North Carolina State University <http://ncsu.edu/>
> >> NOAA National Centers for Environmental Information <
> http://ncdc.noaa.gov/>
> >> formerly NOAA's National Climatic Data Center
> >> 151 Patton Ave, Asheville, NC 28801
> >> e: jbiard at cicsnc.org<mailto:jbiard at cicsnc.org><mailto:jbiard at cicsnc.org
> <mailto:jbiard at cicsnc.org>>
> >> o: +1 828 271 4900
> >>
> >> Connect with us on Facebook for climate<
> https://www.facebook.com/NOAANCEIclimate> and ocean and geophysics<
> https://www.facebook.com/NOAANCEIoceangeo> information, and follow us on
> Twitter at _at_NOAANCEIclimate<https://twitter.com/NOAANCEIclimate> and
> _at_NOAANCEIocngeo<https://twitter.com/NOAANCEIocngeo>.
> >>
> >> _______________________________________________
> >> CF-metadata mailing list
> >> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu><mailto:
> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>>
> >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> >>
> >> --
> >> [CICS-NC]<http://www.cicsnc.org/>Visit us on
> >> Facebook<http://www.facebook.com/cicsnc> Jim Biard
> >> Research Scholar
> >> Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/
> >
> >> North Carolina State University <http://ncsu.edu/>
> >> NOAA National Centers for Environmental Information <
> http://ncdc.noaa.gov/>
> >> formerly NOAA's National Climatic Data Center
> >> 151 Patton Ave, Asheville, NC 28801
> >> e: jbiard at cicsnc.org<mailto:jbiard at cicsnc.org><mailto:jbiard at cicsnc.org
> <mailto:jbiard at cicsnc.org>>
> >> o: +1 828 271 4900
> >>
> >> Connect with us on Facebook for climate<
> https://www.facebook.com/NOAANCEIclimate> and ocean and geophysics<
> https://www.facebook.com/NOAANCEIoceangeo> information, and follow us on
> Twitter at _at_NOAANCEIclimate<https://twitter.com/NOAANCEIclimate> and
> _at_NOAANCEIocngeo<https://twitter.com/NOAANCEIocngeo>.
> >>
> >> _______________________________________________
> >> CF-metadata mailing list
> >> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu><mailto:
> CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>>
> >> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> >>
> >
> > _______________________________________________
> > CF-metadata mailing list
> > CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>
> > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> > ________________________________
> > This message (and any attachments) is for the recipient only. NERC is
> subject to the Freedom of Information Act 2000 and the contents of this
> email and any reply you make may be disclosed by NERC unless it is exempt
> from release under the Act. Any material supplied to NERC may be stored in
> an electronic records management system.
> > ________________________________
> > _______________________________________________
> > CF-metadata mailing list
> > CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>
> > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> > _______________________________________________
> > CF-metadata mailing list
> > CF-metadata at cgd.ucar.edu
> > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> >
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20181030/52a2533b/attachment-0001.html>
Received on Tue Oct 30 2018 - 16:07:04 GMT
