Dear Jonathan and Dan,
Jonathan: The delineations of datasets are not yet clear at all, but I can very well imagine that one dataset might contain both types of indices. E.g. it is reasonable to assume that all "heat-related" indices are lumped together. For the other consideration, whether we would like to enable users to compare indices based on strict and non-strict inequalities, I am slightly more ambivalent. On the one hand, for many practical purposes the difference will be small (and should thus be allowed). On the other hand, with the possibility to set the threshold, my argument is that users should be encouraged to compare like with like. Users really keen on comparing the alternative types of indices will of course have the possibility to do so by means of a few extra processing steps.
Prompted by Dan's test case of Heathrow temperature I did a similar thing for Bromma airport in Stockholm (recorded to the nearest 0.1 degC): from 1951-01-01 to yesterday there are 1111 days with Tmax >= 25 degC, and 1073 days with Tmax > 25 degC. To echo Dan's view, I am not sure as to what difference the 38 occasions actual make in practice, but the distinction is more of a principal nature --- compare like with like. Moreover, the possibility for ET-SCI to change the definition of their TXgeNN indices to involve strict inequalities (which is what I personally thought was the cleaner way) have already been discussed (at some length) and ruled out. Thus, there are parties that do think that the distinction is important, and I think that it is relevant for CF to cater for this.
Kind regards,
Lars
-----Original Message-----
From: CF-metadata [mailto:cf-metadata-bounces at cgd.ucar.edu] On Behalf Of Hollis, Dan
Sent: den 29 mars 2017 11:23
To: Gregory, Jonathan; cf-metadata at cgd.ucar.edu
Subject: Re: [CF-metadata] Request for new standard names for climatological statistics based on thresholds
Hi Jonathan,
I find it hard to say whether the distinction is 'critical' - it just seems to me that, given CF goes to great lengths to be as precise as possible, we ought to be similarly precise when describing parameters derived from discretized data.
Just out of interest, I ran a query on our database to look at daily max temperatures at Heathrow Airport since 01/01/2000. There are 461 days when the maximum was >= 25.0 deg C. Of these, 12 days are exactly 25.0 deg C and would be excluded if a strict inequality was used.
Clearly there are uncertainties associated with observations of temperature, and the values themselves are only reported to the nearest 0.1 deg C. Whether a difference of 12 days is statistically significant I'm not sure. At the very least we know that "greater than" will always produce a lower count than "greater than or equal to" (except for small datasets that do not contain any instances of the threshold value) and it seems reasonable that the definition is clearly stated so that users can make an informed judgement when comparing datasets.
Regards,
Dan
-----Original Message-----
From: CF-metadata [mailto:cf-metadata-bounces at cgd.ucar.edu] On Behalf Of Jonathan Gregory
Sent: 28 March 2017 16:16
To: cf-metadata at cgd.ucar.edu
Subject: Re: [CF-metadata] Request for new standard names for climatological statistics based on thresholds
Dear Lars and Dan
I can't remember what I thought last time this was discussed, so I might be inconsistent! At present, it seems to me the most suitable solution is to have different standard names for strict inequalities and inequalities-allowing- equality, since as Lars says there that means only a dozen new standard names.
Even if more are needed in future, it doesn't seem likely that there will be hundreds - but if so, we can adopt another solution when the need arises.
But one other consideration is whether, in a given dataset, or when comparing datasets, the distinction is critical. Would you have both kinds in a single dataset, and would you decide that a <= quantity in one dataset should *not* be regarded as comparable to a < quantity from another dataset? If Yes in either case, the distinction is needed; but if No then it may not have to be made in the standard name. It could be recorded in a non-standardised way instead.
Best wishes
Jonathan
----- Forwarded message from B?rring Lars <Lars.Barring at smhi.se> -----
> Date: Tue, 28 Mar 2017 15:02:02 +0000
> From: B?rring Lars <Lars.Barring at smhi.se>
> To: "Hollis, Dan" <dan.hollis at metoffice.gov.uk>, "cf-metadata at cgd.ucar.edu"
> <cf-metadata at cgd.ucar.edu>
> Subject: Re: [CF-metadata] Request for new standard names for climatological
> statistics based on thresholds
>
> Hi Dan, all
>
> Thanks for these pointers. Obviously I somehow managed to miss this thread when searching the mail archive (which I actually did try despite all evidence to the contrary). I am sorry for that.
>
> Anyway, in an attempt to restart this discussion let me my summarising the previous thread:
>
> It started off from the need to distinguish between dry and wet days according to a threshold of some small precipitation amount (say 0.2 mm[/day]), that typically had been observed at limited precision (thus discretized). For floating point data from models this not a problem and the arguments were that it would be more appropriate to take actual limits induced by the discretization into account.
>
> There was a suggestion to use the existing standard name but adding a (non CF) logical flag attribute to handle non-strict inequality. But this met opposition as it would split the description of the data into two places (the standard name and the logical flag attribute).
>
> It was noted that a strict and a non-strict inequality are indeed two different things and should not be mixed up in the context of standard names.
> The last comment was to suggest that the way forward would be to rename (redefine) all the relevant standard names to include non-strict inequalities. But, as was noted, this seems cumbersome.
>
> I hope this is a reasonable summary of that thread. With this background, let me add some real use-cases that may shed some new light on the need to make the distinction.
>
> I have in previous posts referred to the two WMO/WCRP/etc. expert teams ETCCDI and ET-SCI defining and producing core sets of climate indices (aka climate indicators). I am member of neither and have no vested interests, other than I do see the need to make this kind of data more easily available in a unified way.
>
> ETCCDI defines Summer Days ("SU") as the number of days when the daily maximum temperature is strictly above +25 degC. This perfectly handled by the standard name number_of_days_with_air_temperature_above_threshold.
>
> ET-SCI defines Hot Days ("TXge30" previously known as "SU30") as the number of days when the daily maximum temperature is equal to or above +30 degC. In structure it similar to SU with the crucial exception of a non-strict inequality. And, this small but crucial difference in definition cannot be changed for any number reasons.
>
> Now, in various user-oriented tools and web services the requirement is to allow the users to change the threshold value to enable them to explore the data and carry out simple sensitivity studies on their own. Thus, there is a concrete need to handle the situation of having the ETCCDI index SU modified to use the threshold value +30 degC (in which case it might perhaps be be called "SU30"). This was in fact the very reason for renaming the ET-SCI index "SU30" to "TXge30" to avoid misunderstandings. Likewise, ET-SCI index TXge30 can be modified to use the threshold value +25 degC (which then would be called "TXge25").
>
> To facilitate the dissemination of these kinds of datasets to a wider community it would indeed be helpful to have standard names that reflect the perhaps small but still crucial difference between strict and non-strict inequalities.
>
> I therefore like to again raise the suggestion to introduce the two constructs "._at_or_above_threshold" and "._at_or_below_threshold" for the relevant standad names.
>
> As I have outlined, we have real use-cases that go beyond what was the
> driver for the discussion in the previous thread. Moreover, with nine
> new standard names requested in my initial post the additional burden
> on the standard name space is modest. However, if this is a concern
> the use cases I have outlined currently cover the following standard
> names number_of_days_with_air_temperature_above_threshold
> number_of_days_with_air_temperature_below_threshold
> number_of_days_with_lwe_thickness_of_precipitation_amount_above_thresh
> old The remaining six I included without having an exhaustive overview
> of what climate indices are used out there, and as they are identical in their structure they could be changed at the same in one go without having to revisit this issue again every time the need pops up.
>
>
> Regards,
> Lars
>
>
> --
> Lars B?rring
>
> FDr, Forskare
> PhD, Research Scientist
>
> SMHI / Swedish Meteorological and Hydrological Institute Rossby
> Centre SE - 601 76 NORRK?PING http://www.smhi.se
>
> E-post / Email: lars.barring at smhi.se
> Tel / Phone: +46 (0)11 495 8604
> Fax: +46 (0)11 495 8001
> Bes?ksadress / Visiting address: Folkborgsv?gen 17
>
>
>
>
>
> -----Original Message-----
> From: Hollis, Dan [mailto:dan.hollis at metoffice.gov.uk]
> Sent: den 27 mars 2017 13:06
> To: B?rring Lars; cf-metadata at cgd.ucar.edu
> Subject: RE: Request for new standard names for climatological
> statistics based on thresholds
>
> Hi Lars,
>
> I raised a very similar question a couple of years ago:
> http://mailman.cgd.ucar.edu/pipermail/cf-metadata/2014/057605.html
>
> The outcome was inconclusive. One suggestion was to add a Boolean attribute that indicated whether the threshold value was included or not. Others thought that adding more standard names was the way to go, while others thought that a single name was sufficient.
>
> I encourage you to read through the rest of the thread and see if it helps with your current request:
> http://mailman.cgd.ucar.edu/pipermail/cf-metadata/2014/thread.html#576
> 05
>
> For info, we continue to use the existing standard names even though they do not strictly match the definitions of our 'days of rain' statistics.
>
> Regards,
>
> Dan
>
>
> Dan Hollis?? Climatologist
> Met Office?? Hadley Centre FitzRoy Road?? Exeter?? Devon?? EX1 3PB?? United Kingdom
> Tel: +44 (0)1392 884535?? Mob: +44 (0)7342058682 Fax: +44 (0)1392 885681
> E-mail: dan.hollis at metoffice.gov.uk?? Website:
> http://www.metoffice.gov.uk For UK climate and past weather
> information, visit http://www.metoffice.gov.uk/climate
>
>
> -----Original Message-----
> From: CF-metadata [mailto:cf-metadata-bounces at cgd.ucar.edu] On Behalf
> Of B?rring Lars
> Sent: 24 March 2017 08:10
> To: cf-metadata at cgd.ucar.edu
> Subject: [CF-metadata] Request for new standard names for
> climatological statistics based on thresholds
>
> Dear all,
>
> Several standard names oriented towards climate indices for various impacts are based on thresholds, and the standard name includes the construct "..._above_threshold" or "..._below_threshold". However, several well-established climate indices use non-strict inequalities in their definition.
>
> For model output using floating point precision the difference between using a strict and a non-strict inequality is small or even negligible, but for observational data discretized to some limited precision (typically one or no decimal digit) this makes a difference.
>
> At a workshop last week people involved in WMO/CCl Expert Team on Sector-specific Climate Indices (ET-SCI) and the joint CCl/WCRP/JCOMM Expert Team on Climate Change Detection and Indices (ETCCDI), as well as the European ECA&D programme and several research projects discussed this.
>
> The outcome of these discussions is to suggest new standard names similar to the existing ones but using the contructs "..._at_or_above_threshold" and "..._at_or_below_threshold". In all other respects these new standard names should be patterned after the following existing ones:
>
> number_of_days_with_air_temperature_above_threshold
> number_of_days_with_air_temperature_below_threshold
> number_of_days_with_lwe_thickness_of_precipitation_amount_above_thresh
> old number_of_days_with_surface_temperature_below_threshold
> number_of_days_with_wind_speed_above_threshold
> spell_length_of_days_with_air_temperature_above_threshold
> spell_length_of_days_with_air_temperature_below_threshold
> spell_length_of_days_with_lwe_thickness_of_precipitation_amount_above_
> threshold
> spell_length_of_days_with_lwe_thickness_of_precipitation_amount_below_
> threshold
>
> The specific use cases for these extension are several ET-SCI defined indices that involves non-strict inequalities.
>
> The alternative of changing the ET-SCI definitions to use a strict inequality is not an option because they have been painstakingly defined in collaboration with user communities and/or are directly related to well-established operational usage.
>
> Likewise, to just adjust the threshold in order to turn the non-strict inequality to a strict equality (say from 30 C to 29.9 C or 29.99 C or ...) is not attractive and prone to cause confusion.
>
> Kind regards,
> Lars
>
> --
> Lars B?rring
>
> FDr, Forskare
> PhD, Research Scientist
>
> SMHI / Swedish Meteorological and Hydrological Institute Rossby
> Centre SE - 601 76 NORRK?PING http://www.smhi.se
>
> E-post / Email: lars.barring at smhi.se
> Tel / Phone: +46 (0)11 495 8604
> Fax: +46 (0)11 495 8001
> Bes?ksadress / Visiting address: Folkborgsv?gen 17
>
>
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
----- End forwarded message -----
_______________________________________________
CF-metadata mailing list
CF-metadata at cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
_______________________________________________
CF-metadata mailing list
CF-metadata at cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Received on Wed Mar 29 2017 - 04:10:11 BST
