⇐ ⇒

[CF-metadata] Ancillary Data

From: Hedley, Mark <mark.hedley>
Date: Thu, 14 Mar 2013 11:53:19 +0000

Thank you for the responses on this topic.

So far I have not found an example of ancillary variable use where

  the ancillary variable references file dimensions which are not referenced by the data variable with the referencing ancillary_variables attribute.

I do not think this should be valid, as it is not practical to relate the ancillary variable data to the data variable. However, I do not think the Conventions are clear enough in banning this.

I have raised a trac ticket:
https://cf-pcmdi.llnl.gov/trac/ticket/98,
which requires a moderator; please may someone volunteer to moderate this ticket?

thank you
mark

________________________________________
From: CF-metadata [cf-metadata-bounces at cgd.ucar.edu] on behalf of Hedley, Mark [mark.hedley at metoffice.gov.uk]
Sent: 18 February 2013 10:45
To: andrew walsh
Cc: cf-metadata at cgd.ucar.edu
Subject: Re: [CF-metadata] Ancillary Data

Hello Andrew

thank you for the response, my reading of your cases is that they are all 'a' or 'b' in the terms I stated.

There are no ancillary variables which reference a dimension of the netCDF file not referenced by the data variable with the ancillary_variables attribute.

I think these are types of case I am particularly concerned to find examples of, or, preferably, discount explicitly.

much obliged
mark

> On 2/13/13 12:15 PM, Hedley, Mark wrote:
>> a. reference the same file dimensions as the data variable with the
>> ancillary_variables attribute references
>>
>> b. reference a subset of the file dimensions referenced by the data
>> variable with the ancillary_variables attribute
>>
>> c. reference file dimensions which are not referenced by the data variable
>> with the ancillary_variables attribute

-----Original Message-----
From: andrew walsh [mailto:awalsh at metoc.gov.au]
Sent: Fri 2/15/2013 3:20 AM
To: Hedley, Mark
Cc: ngalbraith at whoi.edu; cf-metadata at cgd.ucar.edu
Subject: Re: [CF-metadata] Ancillary Data

Hi Mark,

We are using the ancilliary_variables attribute in a real world case for CTD
profile data
(1 netCDF per CTD profile). Not sure if our use case fits with with your
examples a,b,c but here is a abbreviated CDL version:-

 dimensions:

 pressure = UNLIMITED ; // (9 currently)

variables:
 double time ;
  time:standard_name = "time" ;
 ....
 byte time_qc_flag;
  time_qc_flag:long_name = "quality control flag for time (Level 1 flag)" ;
 ....

 double latitude ;
  latitude:standard_name = "latitude" ;
...

 double longitude ;
  longitude:standard_name = "longitude" ;
 ...

 byte position_qc_flag;
  position_qc_flag:long_name = "quality control flag for position (Level 1
flag)" ;
 ....

 double pressure(pressure) ;
  pressure:standard_name = "sea_water_pressure" ;
...

 double temperature(pressure) ;
  temperature:_FillValue = -99.99 ;
  temperature:standard_name = "sea_water_temperature" ;
  temperature:units = "degrees_C" ;
  temperature:valid_min = -2 ;
  temperature:valid_max = 40 ;
  temperature:ancillary_variables = "temperature_whole_profile_flag
temperature_qc_flag temperature_sd_test" ;
  temperature:coordinates = "time latitude longitude pressure" ;

 byte temperature_whole_profile_flag ;
  temperature_whole_profile_flag:long_name = "qc flag for whole temperature
profile (primary L1 flag)" ;
  temperature_whole_profile_flag:quality_control_convention = "Proposed IODE qc
scheme March 2012" ;
  temperature_whole_profile_flag:valid_min = 1 ;
  temperature_whole_profile_flag:valid_max = 9 ;
  temperature_whole_profile_flag:flag_values = 1b, 2b, 3b, 4b, 9b ;
  temperature_whole_profile_flag:flag_meanings = "good not_evaluated_or_unknown
suspect bad missing" ;

 byte temperature_qc_flag(pressure) ;
  temperature_qc_flag:long_name = "quality control flag for temperature (primary
Level 1 flag)" ;
  temperature_qc_flag:standard_name = "sea_water_temperature status_flag" ;
  temperature_qc_flag:quality_control_convention = "Proposed IODE qc scheme
March 2012" ;
  temperature_qc_flag:valid_min = 1 ;
  temperature_qc_flag:valid_max = 9 ;
  temperature_qc_flag:flag_values = 1b, 2b, 3b, 4b, 9b ;
  temperature_qc_flag:flag_meanings = "good not_evaluated_or_unknown suspect bad
missing" ;
  temperature_qc_flag:coordinates = "time latitude longitude pressure" ;

 byte temperature_sd_test(pressure) ;
  temperature_sd_test:long_name = "qc flag for monthly temperature standard
deviation test (secondary L2 flag)"
  temperature_sd_test:quality_control_convention = "Proposed IODE qc scheme
March 2012" ;
  temperature_sd_test:valid_min = 0 ;
  temperature_sd_test:valid_max = 2 ;
  temperature_sd_test:flag_values = 0b, 1b, 2b ;
  temperature_sd_test:flag_meanings = "passed failed unknown" ;
  temperature_sd_test:coordinates = "time latitude longitude pressure" ;

 double salinity(pressure) ;
  salinity:_FillValue = -99.99 ;
  salinity:standard_name = "sea_water_practical_salinity" ;
  salinity:units = "psu" ;
  salinity:valid_min = 0 ;
  salinity:valid_max = 45 ;
  salinity:ancillary_variables = "salinity_whole_profile_flag salinity_qc_flag
salinity_sd_test"
  salinity:coordinates = "time latitude longitude pressure" ;

 byte salinity_whole_profile_flag ;
  salinity_whole_profile_flag:long_name = "qc flag for whole salinity profile
(primary L1 flag)" ;
  salinity_whole_profile_flag:quality_control_convention = "Proposed IODE qc
scheme March 2012" ;
  salinity_whole_profile_flag:valid_min = 1 ;
  salinity_whole_profile_flag:valid_max = 9 ;
  salinity_whole_profile_flag:flag_values = 1b, 2b, 3b, 4b, 9b ;
  salinity_whole_profile_flag:flag_meanings = "good not_evaluated_or_unknown
suspect bad missing" ;

 byte salinity_qc_flag(pressure) ;
  salinity_qc_flag:long_name = "quality control flag for salinity (primary Level
1 flag)" ;
  salinity_qc_flag:standard_name = "sea_water_practical_salinity status_flag" ;
  salinity_qc_flag:quality_control_convention = "Proposed IODE qc scheme March
2012" ;
  salinity_qc_flag:valid_min = 1 ;
  salinity_qc_flag:valid_max = 9 ;
  salinity_qc_flag:flag_values = 1b, 2b, 3b, 4b, 9b ;
  salinity_qc_flag:flag_meanings = "good not_evaluated_or_unknown suspect bad
missing" ;
  salinity_qc_flag:coordinates = "time latitude longitude pressure" ;

 byte salinity_sd_test(pressure) ;
  salinity_sd_test:long_name = "qc flag for monthly salinity standard deviation
test (secondary L2 flag)"
  salinity_sd_test:quality_control_convention = "Proposed IODE qc scheme March
2012" ;
  salinity_sd_test:valid_min = 0 ;
  salinity_sd_test:valid_max = 2 ;
  salinity_sd_test:flag_values = 0b, 1b, 2b ;
  salinity_sd_test:flag_meanings = "passed failed unknown" ;
  salinity_sd_test:coordinates = "time latitude longitude pressure" ;

 int profile ; //Unique integer to identify each profile
  profile:long_name = "profile identifier"
....

Andrew Walsh

----- Original Message -----
From: "Hedley, Mark" <mark.hedley at metoffice.gov.uk>
To: <ngalbraith at whoi.edu>; <cf-metadata at cgd.ucar.edu>
Sent: Thursday, February 14, 2013 10:40 PM
Subject: Re: [CF-metadata] Ancillary Data


> Hi Nan
>
> I think I understand you approach, it seems logical and helpful to me.
>
> It feels like all your examples are in my category b:
>
>> b. reference a subset of the file dimensions referenced by the data variable
>> with the ancillary_variables attribute
>
> and thus relate to the data variable in the same way auxiliary_coordinates do,
> just with different inferred semantics.
>
> many thanks for the feedback
>
> mark
>
> -----Original Message-----
> From: CF-metadata on behalf of Nan Galbraith
> Sent: Wed 13/02/2013 19:55
> To: cf-metadata at cgd.ucar.edu
> Subject: Re: [CF-metadata] Ancillary Data
>
> I use a singleton variable to record the magnetic correction that's been
> applied to my wind and/or current variables; it's dim(1) and is listed as
> an ancillary to any variables that have been affected by the rotation.
>
> It could be an attribute of each of those variables, but making it a
> free-standing
> variable lets me give it units and attributes. That's important because I
> record the model name and version date, the URL of the NGDC calculator
> site,
> the inputs to the model (date and location for which the calculation was
> done)
> and the estimated rate of change.
>
> Maybe this is just a technicality - but I also use an empty 'container'
> variable for
> instruments, which has ancillary variables with a depth dimension for
> manufacturer,
> model, serial number, and reference URL. The instrument variable is
> tied to 'obs data'
> variables via NODC's 'instrument' attribute, but has no dimensions of
> its own; the
> individual component variables (like serial number) have a dimension
> that matches
> the depth dimension of the obs data variables.
>
> - Nan
>
> On 2/13/13 12:15 PM, Hedley, Mark wrote:
>> Hello CF community
>>
>> I have been perusing the CF conventions again, particularly the section on
>> Ancillary Data
>> http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.6/ch03s04.html
>>
>> The conventions make the statement:
>>
>> The nature of the relationship between variables associated via
>> ancillary_variables must be determined by other attributes.
>>
>> The example given provides data variables which reference the same dimensions
>> in the file: thus they data arrays are the same size. However, the
>> conventions do not seem to mandate this.
>>
>> I am interested in the use of the
>> ancillary_variables
>> attribute in real world datasets.
>>
>> Do people have examples they can share of ancillary datasets which:
>>
>> a. reference the same file dimensions as the data variable with the
>> ancillary_variables attribute references
>>
>> b. reference a subset of the file dimensions referenced by the data
>> variable with the ancillary_variables attribute
>>
>> c. reference file dimensions which are not referenced by the data variable
>> with the ancillary_variables attribute
>>
>> I am particularly interested in examples of case c, as I feel this is
>> markedly different from cases a and b and requires a different kind of
>> support if it is in use.
>>
>> many thanks
>> mark
>> _______________________________________________
>> CF-metadata mailing list
>> CF-metadata at cgd.ucar.edu
>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>>
>
>
> --
> *******************************************************
> * Nan Galbraith Information Systems Specialist *
> * Upper Ocean Processes Group Mail Stop 29 *
> * Woods Hole Oceanographic Institution *
> * Woods Hole, MA 02543 (508) 289-2444 *
> *******************************************************
>
>
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>



_______________________________________________
CF-metadata mailing list
CF-metadata at cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Received on Thu Mar 14 2013 - 05:53:19 GMT

This archive was generated by hypermail 2.3.0 : Tue Sep 13 2022 - 23:02:41 BST

⇐ ⇒