Hi Martin, Ken,
Is there anything wrong with including multiple "status_flag"
variables to capture all separate state you wish? The CF document
unfortunately only includes an example of how to encode the status of
a sensor, but the actual meanings of the flag values are entirely up
to you, and this will not change with this proposal. Perhaps the CF
document would benefit from additional examples (e.g. one that only
shows data quality flags).
-Barna
On Mon, Jul 22, 2019 at 9:04 AM Kehoe, Kenneth E. <kkehoe at ou.edu> wrote:
>
> Hi Martin,
>
> I see status encompassing multiple metadata pieces of information. For
> example it could be a state of the instrument as it cycles through a
> pre-programed routine (Look at calibration target, look at sky, look at
> ground, look at second calibration target, repeat...). Or the sources of
> the inputs for a model where the availability or some other reason could
> require making a decision on what source(s) to use. For provenance this
> source information is important to report on a time step basis. Or the
> status could be a data providers method to provide uncertainty
> information (I see this as incorrect but some people do see it this
> way). Each of these are important metadata but the method of use is
> different than a strictly quality variable. A quality variable provides
> information indicating if the data should be used or possibly could be
> used in a weighted mean method to favor high quality data over low
> quality data. The way the metadata is used is different depending on the
> metadata type. A state of the instrument would be used for sub-setting
> calibration vs. data. There is no ambiguity in this as data from a
> calibration target is not used in a weather research analysis. But
> quality is more subjective and is decided by the data user. If the
> quality variable has 20 different quality tests the user would need to
> decided if all 20 test results should be used or only a subset. Also,
> the code for applying the quality is different than the state of the
> instrument view (in my example above).
>
> It is possible to have a quality test result from the state of the
> instrument, but not the other way around (typically). So I need a way to
> distinguish the two for automated or semi-automated tools. Hence my
> point of quality_flag essentially being a subset of status_flag
>
> Ken
>
>
>
> On 2019-7-22 02:57, Martin Juckes - UKRI STFC wrote:
> > Dear Ken,
> >
> >
> > Can you expand on the distinction between "quality" and "status"? I understand that they are different in principle, but, in order to support this new standard name I think we need a clear objective statement of how we would want to distinguish between them in CF.
> >
> > The conventions section on flags (3.5) mixes the two up (http://cfconventions.org/cf-conventions/cf-conventions.html#flags ), so some re-wording of the document would also be needed,
> >
> > regards,
> > Martin
> >
> > ________________________________
> > From: CF-metadata <cf-metadata-bounces at cgd.ucar.edu> on behalf of Kehoe, Kenneth E. <kkehoe at ou.edu>
> > Sent: 19 July 2019 06:42
> > To: cf-metadata at cgd.ucar.edu
> > Subject: [CF-metadata] New standard_name of quality_flag for corresponding quality control variables
> >
> > Dear CF,
> >
> > I am proposing a new standard name of "quality_flag" to indicate a variable is purely a quality control variable. A quality control variable would use flag_values or flag_masks along with flag_meanings to allow declaring levels of quality or results from quality indicating tests of the data variable. This variable be a subset of the more general "status_flag" standard name. Currently the definition of "status_flag" is:
> >
> > - A variable with the standard name of status_flag contains an indication of quality or other status of another data variable. The linkage between the data variable and the variable with the standard_name of status_flag is achieved using the ancillary_variables attribute.
> >
> > This definition includes a variable used to define the state or other status information of a variable and can not be distinguished by standard name alone from a state of the instrument, processing decision, source information, needed metadata about the data variable or other ancillary variable type. Since there is no other way to define a purely quality control variable, the use of "status_flag" is too general for strictly quality control variables. By having a method to define a variable as strictly quality control the results of quality control tests can be applied to the data with a software tool based on requests by the user. This would not affect current datasets that do use "status_flag" nor require a change to the definition outside of the indication that "quality_flag" standard name is available and a better use for pure quality control variables.
> >
> > Proposed addition:
> >
> > quality_flag = A variable with the standard name of quality_flag contains an indication of quality information of another data variable. The linkage between the data variable and the variable or variables with the standard_name of quality_flag is achieved using the ancillary_variables attribute.
> >
> > Proposed change:
> >
> > status_flag = A variable with the standard name of status_flag contains an indication of status of another data variable. The linkage between the data variable and the variable with the standard_name of status_flag is achieved using the ancillary_variables attribute. For data quality information use quality_flag.
> >
> > Thanks,
> >
> > Ken
> >
> >
> >
> > --
> > Kenneth E. Kehoe
> > Research Associate - University of Oklahoma
> > Cooperative Institute for Mesoscale Meteorological Studies
> > ARM Climate Research Facility - Data Quality Office
> > e-mail: kkehoe at ou.edu<mailto:kkehoe at ou.edu> | Office: 303-497-4754
>
> --
> Kenneth E. Kehoe
> Research Associate - University of Oklahoma
> Cooperative Institute for Mesoscale Meteorological Studies
> ARM Climate Research Facility - Data Quality Office
> e-mail: kkehoe at ou.edu | Office: 303-497-4754
>
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Received on Mon Jul 22 2019 - 10:35:01 BST
