I can appreciate Ken's point of view, particularly where the convenience is like the example Jonathan posted of having one scalar coordinate, there's not much added by encoding this as a scalar.
However I support Jonathan's statement about omitting size one dimensions, particularly where there are a number. It is often helpful not to encode information about the ordering of dimensions of size one.
In addition to this, I am keen to make sure that data creators are able to encode metadata where the dependency relationship between multiple scalars is not encoded. In numerous cases, the relationship is not uniquely defined and should not be encoded.
For example a data creator may have a set of descriptors and identifiers which define how a particular data set is defined with respect to a larger study, such as a multi-model analysis meta-experiment. Each data set is produced by a model with a collection of scalar coordinates from that model run, e.g. 'ensemble member number', 'experiment id', 'perturbation scheme', 'forcing parameter a', forcing parameter b', ... etc.
Given different collections of such data sets, different relationships may emerge from the collection, enabling different types of analysis. It is a really useful approach to encode all these quantities as scalars, and interpret these scalars as potential degrees of freedom with potential inter-relations; the degrees of freedom and inter-relationships are then emergent properties of the collection, not defined in any individual member data set.
This seems to me to be the logical conclusion of the use of the term 'scalar coordinate' (as contrasted to 'vector coordinate', i assume) and it is really useful.
I think it would be a regressive step for CF to limit or complicate this facet of data comprehension by constraining the meaning of 'scalar coordinate' in the way that has been suggested. I don't think such a constraint has been clear up to now in the conventions and people are making real use of the perceived flexibility.
mark
________________________________________
From: CF-metadata [cf-metadata-bounces at cgd.ucar.edu] on behalf of Jonathan Gregory [j.m.gregory at reading.ac.uk]
Sent: 21 May 2013 18:42
To: cf-metadata at cgd.ucar.edu
Subject: Re: [CF-metadata] scalar coordinates
Dear Ken
Your argument is one that we should not have scalar coordinates at all. You
could propose that they should be removed from the next version of CF (and
that would certainly simplify this discussion, if agreed :-). They have been
in there for several years now, and I guess they are quite widely used,
because it is convenient for data-writers. I agree, it does require a bit
more work for data-readers. However, CF-compliant software should expect to
inspect the coordinates attribute in any case, and if it does that it will
automatically come across the scalar coordinate variables.
I think that an attractive feature of omitting the size-one dimensions is
not having to decide on the order of them in the data variable, which is
really arbitrary for storage in netCDF files, since it makes no difference
to the order of the data elements.
Best wishes
Jonathan
----- Forwarded message from "Kenneth S. Casey - NOAA Federal" <kenneth.casey at noaa.gov> -----
> From: "Kenneth S. Casey - NOAA Federal" <kenneth.casey at noaa.gov>
> Date: Tue, 21 May 2013 07:22:34 -0400
> To: "Hedley, Mark" <mark.hedley at metoffice.gov.uk>
> X-Mailer: Apple Mail (2.1503)
> CC: "cf-metadata at cgd.ucar.edu" <cf-metadata at cgd.ucar.edu>
> Subject: Re: [CF-metadata] scalar coordinates
>
> Hi Everyone,
>
> After spending 15 minutes reading this and Jonathan's previous post, and trying hard to be sure I really understand them, I am left wondering if a "convenience feature" is really a convenience at all. I know programmers don't like clutter in their code and have certain aesthetic to uphold (I used to do much more serious programming in my younger days, but am now just a lazy Matlaber), but is the added complexity of having these two options, copied from Jonathan's post:
>
> float height; // scalar coordinate variable
> height: standard_name="height";
> float temp(lat,lon);
> temp: standard_name="air_temperature";
> temp: coordinates="height";
>
> float height(height); // size-one coordinate var, with dimension height=1
> height: standard_name="height";
> float temp(height,lat,lon);
> temp: standard_name="air_temperature";
>
> really worth it? The manual must be longer to describe the "convenience", and the application programmers and downstream users of the data now have to build complexity into THEIR code to handle both cases. This list has to take time explaining and debating the options, and I have to take time explaining it to both data producers and data consumers who have never used netCDF or CF before. Maybe that is not such a big deal on a case by case basis, but in the broader picture when folks like me are trying to get everyone and their brothers and sisters and mothers and fathers using CF-netCDF, it is a real pain. My vote is to simplify. Use the second example, since it handles the case where height=1 as easily as the case where height=1000. I say be explicit. Does the first example gain us any functionality? If not, then don't use it and stay away from it. Encourage others to do the same.
>
> Ken
>
>
> On May 21, 2013, at 5:48 AM, "Hedley, Mark" <mark.hedley at metoffice.gov.uk> wrote:
>
> > The term 'convenience feature' is mentioned in the conventions document:
> >
> > 'The new scalar coordinate variable is a convenience feature which avoids adding size one dimensions to variables.'
> >
> > Data creators have seen the benefits in not encoding size one dimensions and made use of this feature, it has proved very convenient. The conventions go on to say:
> >
> > 'Scalar coordinate variables have the same information content and can be used in the same contexts as a size one coordinate variable.'
> >
> > But this statement is not quite true: the ordering of dimensions is not encoded, and the ability to link many coordinates to the same dimension is lost. The assumption in this statement is an aspiration which I think cannot be delivered without particularly strict limitations on the use of scalars during encoding.
> >
> > Nowhere in the conventions does it state that if more than one single-valued coordinate is related to the same degree of freedom, a dimension must be declared for these and this relationship explicitly encoded.
> >
> > Later, the case of character strings is addressed:
> >
> > 'If a character variable has only one dimension (the maximum length of the string), it is regarded as a string-valued scalar coordinate variable, analogous to a numeric scalar coordinate variable (see Section 5.7, ?Scalar Coordinate Variables?) '
> >
> > which is a required feature, but the NUG only allows numerical valued data arrays as Coordinate Variables, so a further section is added, in the Terminology:
> >
> > 'scalar coordinate variable
> > A scalar variable that contains coordinate data. Functionally equivalent to either a size one coordinate variable or a size one auxiliary coordinate variable. '
> >
> > These statements together provide information on how to write files, but they are limited in their assistance to file reading and interpretation.
> >
> > The conventions are not clear how to, or whether to make a distinction for a particular scalar coordinate: it does not say that a scalar coordinate is a Coordinate Variable or an Auxiliary Coordinate Variable, it says it is functionally equivalent to either one or the other.
> >
> > I have read these sections to mean that by encoding a scalar coordinate the data creator is not providing information about how the coordinate is related to the dimensions in the file, other than to say it applies to all of the cells currently in the file.
> >
> > As such, I disagree with the statement that that
> >
> > 'Scalar coordinate variables have the same information content and can be used in the same contexts as a size one coordinate variable.'
> >
> > In many cases this will turn out to be a valid interpretation but it is not the only one, and this nuance is a really useful feature, which many data creators have benefited from.
> >
> > From one point of view, a third type of Coordinate exists in CF, the Scalar Coordinate, which is neither a Coordinate Variable, nor an Auxiliary Coordinate. From another point of view a Scalar Coordinate is an Auxiliary Coordinates which has the potential to be an emergent Coordinate Variable, if required and consistent for the data consumer. (I am sure there are other useful perspectives we can consider)
> >
> > We have come across many data sets from other data creators where a considered reading of the data suggests that they have taken an interpretation such as this as well. No distinction has been made between scalars which represent a degree of freedom and scalars which do not.
> >
> > The scalar coordinate is a convenient feature allowing metadata to be simply encoded in a clear manner and I feel that the conventions document should adapt to reflect the usage some sections of the community have adopted. It is not ambiguous, it provides sufficient information to work with the file and the data and metadata are well specified.
> >
> > Indeed when converting from other formats (such as GRIB and BUFR) to CF it is the logical way to encode the available metadata.
> >
> > I am concerned about the implications for these data sets if the interpretation of scalar coordinates is tightened in a future version of the conventions document to explicitly disallow this useful and well used point of view. I would like to stress again Jonathan's point, that all of this data is CF compliant, the question is how consumers interpret the semantics of the data set.
> >
> > I think the utility of the scalar coordinate variable is significantly diminished if Option B or some derivative of it is pursued for the next version of the conventions. Option A preserves all of the interpretations of Option B intact, but with caution needed on loading and interpretation not to read too much information into any scalar coordinates present.
> >
> > _______________________________________________
> > CF-metadata mailing list
> > CF-metadata at cgd.ucar.edu
> > http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>
> Kenneth S. Casey, Ph.D.
> Technical Director
> NOAA National Oceanographic Data Center
> 1315 East-West Highway
> Silver Spring MD 20910
> 301-713-3272 x133
> http://www.nodc.noaa.gov
>
>
>
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
----- End forwarded message -----
_______________________________________________
CF-metadata mailing list
CF-metadata at cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Received on Thu May 23 2013 - 03:04:44 BST
