[CF-metadata] [cf-satellite] Sharing quality flags amongmultiple variables

From: Mike Grant <mggr>
Date: Wed, 02 Nov 2011 20:20:03 +0000

Now I *think* I mostly understand the problem - thanks for the summary
explanation, Jonathan, and apologies for the earlier missing-the-point :)

However, I probably still need a bit more straightening out or have
missed the point again.. I'd be grateful to understand more about why
one needs a different (official?) standard_name for each different usage
of a single flag variable by multiple data variables?

If it's for machine usage, the machine already knows that it's a flag
variable (because it has flag_meanings) and the machine already knows
the standard_name of the "parent" variable (because it's followed an
ancillary_variable in the parent). It could then make the appropriate
inference of naming/meaning for the parent variable it's considering.
In this case, not having a standard_name seems appropriate (or just
having a generic one for humans and ignored by machines).

If it's just for human (display) purposes, because a lot of software
uses standard_name, one could name the flags variable something fairly
generic (and non-standard) - e.g. "sea_ice_flags" would reasonably
reflect the usage of a bunch of flags that generally apply to multiple
sea ice variables. Wanting to give specific standard_names to each
usage for display/human purposes is fine but both proposed solutions
probably break existing software. Given that the meaning can already be
inferred, that seems quite a high cost if there are no other reasons.

(use case 2) Upendra has a fair point on overheads of flag_meanings,
which seems to be a separate but related issue. As a satellite data
person, the overhead is small for me and I'd thus weight the value of
breaking software to fix this relatively low, but I see that's different
for insitu or other small data.

(use case 3) To conflate this further with another related issue,
there's also the nice possibility of using the same flags data storage
variable but with different flags_meanings for it in different usages.
For example, one might use bits 0, 1, 2, 3 for sea_ice_x_displacement
and bits 0, 1, 4, 5 for sea_ice_y_displacement. That seems a small but
genuine advance in machine understandable use of the flags - the machine
usage described above would presume that all flags have some useful
meaning for the parent variable while this would allow one to be
specific about which bits mattered. This isn't possible with either
proposal, but #2 could be extended to be fully generic and cover it.
I'll put that in another email as this is already way too long and I
suspect it's unworkable ;)

On 02/11/11 14:45, Jonathan Gregory wrote:
> * Have only one status_flag variable, with flag_values and flag_meanings
> attributes. This one variable will be pointed to by the ancillary_variables
> attribute of several data variables. This means we need a new convention for
> the standard_name attribute so that it can be associated with data variables
> that may have various standard_names.

This may break existing software that relies on standard_name holding a
single value (presumably all software?), but primarily it will only
break the correct naming of the variables with multiple standard_names.
 However, it seems only to cater for naming convenience rather than
adding any real semantic meaning, as most of the meaning can be inferred.

> * Have a separate status_flag variable for each data variable. In that case
> the standard_name can be specific to the data variable. To avoid repeating
> the definitions of flags, introduce a new convention to allow the flag_values
> and flag_meanings attributes to be attached to a separate container variable
> that can be pointed to by all the data variables.

This will only break existing software that deals with flags (this
definitely exists for satellite data). The old software will be
incapable of using the new-style flag variables as it won't be able to
find the necessary elements. Other software should be ok.

Cheers,

Mike.

P.S. I'm assuming it's not proposed to add a lot of flags names to the
standard table, because there'd be a lot of names and the usage/meaning
isn't standard. One could have a general rule of a derivative "_flags"
being allowed for any standard_name, but this doesn't really advance
machine understanding (see above) and is presumably just for display /
validation-passing.

--------------------------------------------------------------------------------
Plymouth Marine Laboratory
 
Registered Office:
Prospect Place
The Hoe
Plymouth PL1 3DH
 
Website: www.pml.ac.uk
Click here for PML Annual Review
Registered Charity No. 1091222
PML is a company limited by guarantee
registered in England & Wales
company number 4178503

Please think before you print

--------------------------------------------------------------------------------
This e-mail, its content and any file attachments are confidential.

If you have received this e-mail in error please do not copy, disclose it to any third party or use the contents or attachments in any way. Please notify the sender by replying to this e-mail or e-mail forinfo at pml.ac.uk and then delete the email without making any copies or using it in any other way.

The content of this message may contain personal views which are not the views of Plymouth Marine Laboratory unless specifically stated.

You are reminded that e-mail communications are not secure and may contain viruses. Plymouth Marine Laboratory accepts no liability for any loss or damage which may be caused by viruses.
--------------------------------------------------------------------------------
Received on Wed Nov 02 2011 - 14:20:03 GMT