Barna,
Yes an update to the CF document should follow after the new
standard_name is implemented. I think multiple examples are needed since
status_flag covers many different types of state variables.
Ken
On 2019-7-22 10:35, Andrew Barna wrote:
> Hi Martin, Ken,
>
> Is there anything wrong with including multiple "status_flag"
> variables to capture all separate state you wish? The CF document
> unfortunately only includes an example of how to encode the status of
> a sensor, but the actual meanings of the flag values are entirely up
> to you, and this will not change with this proposal. Perhaps the CF
> document would benefit from additional examples (e.g. one that only
> shows data quality flags).
>
> -Barna
>
>
> On Mon, Jul 22, 2019 at 9:04 AM Kehoe, Kenneth E. <kkehoe at ou.edu> wrote:
>> Hi Martin,
>>
>> I see status encompassing multiple metadata pieces of information. For
>> example it could be a state of the instrument as it cycles through a
>> pre-programed routine (Look at calibration target, look at sky, look at
>> ground, look at second calibration target, repeat...). Or the sources of
>> the inputs for a model where the availability or some other reason could
>> require making a decision on what source(s) to use. For provenance this
>> source information is important to report on a time step basis. Or the
>> status could be a data providers method to provide uncertainty
>> information (I see this as incorrect but some people do see it this
>> way). Each of these are important metadata but the method of use is
>> different than a strictly quality variable. A quality variable provides
>> information indicating if the data should be used or possibly could be
>> used in a weighted mean method to favor high quality data over low
>> quality data. The way the metadata is used is different depending on the
>> metadata type. A state of the instrument would be used for sub-setting
>> calibration vs. data. There is no ambiguity in this as data from a
>> calibration target is not used in a weather research analysis. But
>> quality is more subjective and is decided by the data user. If the
>> quality variable has 20 different quality tests the user would need to
>> decided if all 20 test results should be used or only a subset. Also,
>> the code for applying the quality is different than the state of the
>> instrument view (in my example above).
>>
>> It is possible to have a quality test result from the state of the
>> instrument, but not the other way around (typically). So I need a way to
>> distinguish the two for automated or semi-automated tools. Hence my
>> point of quality_flag essentially being a subset of status_flag
>>
>> Ken
>>
>>
>>
>> On 2019-7-22 02:57, Martin Juckes - UKRI STFC wrote:
>>> Dear Ken,
>>>
>>>
>>> Can you expand on the distinction between "quality" and "status"? I understand that they are different in principle, but, in order to support this new standard name I think we need a clear objective statement of how we would want to distinguish between them in CF.
>>>
>>> The conventions section on flags (3.5) mixes the two up (http://cfconventions.org/cf-conventions/cf-conventions.html#flags ), so some re-wording of the document would also be needed,
>>>
>>> regards,
>>> Martin
>>>
>>> ________________________________
>>> From: CF-metadata <cf-metadata-bounces at cgd.ucar.edu> on behalf of Kehoe, Kenneth E. <kkehoe at ou.edu>
>>> Sent: 19 July 2019 06:42
>>> To: cf-metadata at cgd.ucar.edu
>>> Subject: [CF-metadata] New standard_name of quality_flag for corresponding quality control variables
>>>
>>> Dear CF,
>>>
>>> I am proposing a new standard name of "quality_flag" to indicate a variable is purely a quality control variable. A quality control variable would use flag_values or flag_masks along with flag_meanings to allow declaring levels of quality or results from quality indicating tests of the data variable. This variable be a subset of the more general "status_flag" standard name. Currently the definition of "status_flag" is:
>>>
>>> - A variable with the standard name of status_flag contains an indication of quality or other status of another data variable. The linkage between the data variable and the variable with the standard_name of status_flag is achieved using the ancillary_variables attribute.
>>>
>>> This definition includes a variable used to define the state or other status information of a variable and can not be distinguished by standard name alone from a state of the instrument, processing decision, source information, needed metadata about the data variable or other ancillary variable type. Since there is no other way to define a purely quality control variable, the use of "status_flag" is too general for strictly quality control variables. By having a method to define a variable as strictly quality control the results of quality control tests can be applied to the data with a software tool based on requests by the user. This would not affect current datasets that do use "status_flag" nor require a change to the definition outside of the indication that "quality_flag" standard name is available and a better use for pure quality control variables.
>>>
>>> Proposed addition:
>>>
>>> quality_flag = A variable with the standard name of quality_flag contains an indication of quality information of another data variable. The linkage between the data variable and the variable or variables with the standard_name of quality_flag is achieved using the ancillary_variables attribute.
>>>
>>> Proposed change:
>>>
>>> status_flag = A variable with the standard name of status_flag contains an indication of status of another data variable. The linkage between the data variable and the variable with the standard_name of status_flag is achieved using the ancillary_variables attribute. For data quality information use quality_flag.
>>>
>>> Thanks,
>>>
>>> Ken
>>>
>>>
>>>
>>> --
>>> Kenneth E. Kehoe
>>> Research Associate - University of Oklahoma
>>> Cooperative Institute for Mesoscale Meteorological Studies
>>> ARM Climate Research Facility - Data Quality Office
>>> e-mail: kkehoe at ou.edu<mailto:kkehoe at ou.edu> | Office: 303-497-4754
>> --
>> Kenneth E. Kehoe
>> Research Associate - University of Oklahoma
>> Cooperative Institute for Mesoscale Meteorological Studies
>> ARM Climate Research Facility - Data Quality Office
>> e-mail: kkehoe at ou.edu | Office: 303-497-4754
>>
>> _______________________________________________
>> CF-metadata mailing list
>> CF-metadata at cgd.ucar.edu
>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
--
Kenneth E. Kehoe
Research Associate - University of Oklahoma
Cooperative Institute for Mesoscale Meteorological Studies
ARM Climate Research Facility - Data Quality Office
e-mail: kkehoe at ou.edu | Office: 303-497-4754
Received on Mon Jul 22 2019 - 14:22:03 BST
