Hello Jim
you present an interesting case, thank you
I can see how I can relate
<energy deposited>(x,y,z,t)
to an ancillary variable of
<model atmosphere density profile>(x,y,z)
In this case, for any value in my <energy deposited> data array I can retrieve an appropriate value for my ancillary variable, <model atmosphere density profile>.
I am assuming that I can aggregate (integrate?) my <energy deposited> data over z,to result in the total energy deposited as a function of x, y and t.
The atmosphere density profile was an input to this calculation, I believe.
If i try to store this as an ancillary variable for my result I am no longer able to reference an appropriate value in the <model atmosphere density profile> data array for a given value in the <energy deposited>.
The conventions state
'The nature of the relationship between variables associated via ancillary_variables must be determined by other attributes.'
(
http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.6/ch03s04.html)
I am worried that in this case I can no longer understand this association. The ancillary variable is defined with respect to a different domain from the data variable.
I might happily choose to ship two data variables in the same file, but do I really mean that the <model atmosphere density profile> is an intrinsic part of my <energy deposited> dataset?
what do you think?
mark
________________________________
From: CF-metadata [cf-metadata-bounces at cgd.ucar.edu] on behalf of Jim Biard [jim.biard at noaa.gov]
Sent: 14 March 2013 13:51
To: cf-metadata at cgd.ucar.edu
Subject: Re: [CF-metadata] Ancillary Data
Mark,
I've been following this thread silently, but your statement in your last posting has prompted me to speak up. I can easily imagine a situation in which I might desire to have one variable reference another variable in which not all dimensions are shared. As an example, let's say that I have a variable a[X,Y,T], which contains the total energy deposited per day into the atmosphere by charged particles (from the solar wind & Van Allen belts) on a lon/lat grid as a function of time. (Something I used to work on many years ago.) Let's then say that I have another variable, b[X,Y,Z], which represents the model atmosphere density profile on the same lon/lat grid as a function of altitude. This density profile was used in the calculation of the energy deposition values stored in variable a. Variable b is clearly valid as an ancillary variable for a, isn't it?
Grace and peace,
Jim
Jim Biard
Research Scholar
Cooperative Institute for Climate and Satellites
Remote Sensing and Applications Division
National Climatic Data Center
151 Patton Ave, Asheville, NC 28801-5001
jim.biard at noaa.gov<mailto:jim.biard at noaa.gov>
828-271-4900
On Mar 14, 2013, at 7:53 AM, "Hedley, Mark" <mark.hedley at metoffice.gov.uk<mailto:mark.hedley at metoffice.gov.uk>> wrote:
Thank you for the responses on this topic.
So far I have not found an example of ancillary variable use where
the ancillary variable references file dimensions which are not referenced by the data variable with the referencing ancillary_variables attribute.
I do not think this should be valid, as it is not practical to relate the ancillary variable data to the data variable. However, I do not think the Conventions are clear enough in banning this.
I have raised a trac ticket:
https://cf-pcmdi.llnl.gov/trac/ticket/98,
which requires a moderator; please may someone volunteer to moderate this ticket?
thank you
mark
________________________________________
From: CF-metadata [cf-metadata-bounces at cgd.ucar.edu<mailto:cf-metadata-bounces at cgd.ucar.edu>] on behalf of Hedley, Mark [mark.hedley at metoffice.gov.uk<mailto:mark.hedley at metoffice.gov.uk>]
Sent: 18 February 2013 10:45
To: andrew walsh
Cc: cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>
Subject: Re: [CF-metadata] Ancillary Data
Hello Andrew
thank you for the response, my reading of your cases is that they are all 'a' or 'b' in the terms I stated.
There are no ancillary variables which reference a dimension of the netCDF file not referenced by the data variable with the ancillary_variables attribute.
I think these are types of case I am particularly concerned to find examples of, or, preferably, discount explicitly.
much obliged
mark
On 2/13/13 12:15 PM, Hedley, Mark wrote:
a. reference the same file dimensions as the data variable with the
ancillary_variables attribute references
b. reference a subset of the file dimensions referenced by the data
variable with the ancillary_variables attribute
c. reference file dimensions which are not referenced by the data variable
with the ancillary_variables attribute
-----Original Message-----
From: andrew walsh [mailto:awalsh at metoc.gov.au<
http://metoc.gov.au>]
Sent: Fri 2/15/2013 3:20 AM
To: Hedley, Mark
Cc: ngalbraith at whoi.edu<mailto:ngalbraith at whoi.edu>; cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>
Subject: Re: [CF-metadata] Ancillary Data
Hi Mark,
We are using the ancilliary_variables attribute in a real world case for CTD
profile data
(1 netCDF per CTD profile). Not sure if our use case fits with with your
examples a,b,c but here is a abbreviated CDL version:-
dimensions:
pressure = UNLIMITED ; // (9 currently)
variables:
double time ;
time:standard_name = "time" ;
....
byte time_qc_flag;
time_qc_flag:long_name = "quality control flag for time (Level 1 flag)" ;
....
double latitude ;
latitude:standard_name = "latitude" ;
...
double longitude ;
longitude:standard_name = "longitude" ;
...
byte position_qc_flag;
position_qc_flag:long_name = "quality control flag for position (Level 1
flag)" ;
....
double pressure(pressure) ;
pressure:standard_name = "sea_water_pressure" ;
...
double temperature(pressure) ;
temperature:_FillValue = -99.99 ;
temperature:standard_name = "sea_water_temperature" ;
temperature:units = "degrees_C" ;
temperature:valid_min = -2 ;
temperature:valid_max = 40 ;
temperature:ancillary_variables = "temperature_whole_profile_flag
temperature_qc_flag temperature_sd_test" ;
temperature:coordinates = "time latitude longitude pressure" ;
byte temperature_whole_profile_flag ;
temperature_whole_profile_flag:long_name = "qc flag for whole temperature
profile (primary L1 flag)" ;
temperature_whole_profile_flag:quality_control_convention = "Proposed IODE qc
scheme March 2012" ;
temperature_whole_profile_flag:valid_min = 1 ;
temperature_whole_profile_flag:valid_max = 9 ;
temperature_whole_profile_flag:flag_values = 1b, 2b, 3b, 4b, 9b ;
temperature_whole_profile_flag:flag_meanings = "good not_evaluated_or_unknown
suspect bad missing" ;
byte temperature_qc_flag(pressure) ;
temperature_qc_flag:long_name = "quality control flag for temperature (primary
Level 1 flag)" ;
temperature_qc_flag:standard_name = "sea_water_temperature status_flag" ;
temperature_qc_flag:quality_control_convention = "Proposed IODE qc scheme
March 2012" ;
temperature_qc_flag:valid_min = 1 ;
temperature_qc_flag:valid_max = 9 ;
temperature_qc_flag:flag_values = 1b, 2b, 3b, 4b, 9b ;
temperature_qc_flag:flag_meanings = "good not_evaluated_or_unknown suspect bad
missing" ;
temperature_qc_flag:coordinates = "time latitude longitude pressure" ;
byte temperature_sd_test(pressure) ;
temperature_sd_test:long_name = "qc flag for monthly temperature standard
deviation test (secondary L2 flag)"
temperature_sd_test:quality_control_convention = "Proposed IODE qc scheme
March 2012" ;
temperature_sd_test:valid_min = 0 ;
temperature_sd_test:valid_max = 2 ;
temperature_sd_test:flag_values = 0b, 1b, 2b ;
temperature_sd_test:flag_meanings = "passed failed unknown" ;
temperature_sd_test:coordinates = "time latitude longitude pressure" ;
double salinity(pressure) ;
salinity:_FillValue = -99.99 ;
salinity:standard_name = "sea_water_practical_salinity" ;
salinity:units = "psu" ;
salinity:valid_min = 0 ;
salinity:valid_max = 45 ;
salinity:ancillary_variables = "salinity_whole_profile_flag salinity_qc_flag
salinity_sd_test"
salinity:coordinates = "time latitude longitude pressure" ;
byte salinity_whole_profile_flag ;
salinity_whole_profile_flag:long_name = "qc flag for whole salinity profile
(primary L1 flag)" ;
salinity_whole_profile_flag:quality_control_convention = "Proposed IODE qc
scheme March 2012" ;
salinity_whole_profile_flag:valid_min = 1 ;
salinity_whole_profile_flag:valid_max = 9 ;
salinity_whole_profile_flag:flag_values = 1b, 2b, 3b, 4b, 9b ;
salinity_whole_profile_flag:flag_meanings = "good not_evaluated_or_unknown
suspect bad missing" ;
byte salinity_qc_flag(pressure) ;
salinity_qc_flag:long_name = "quality control flag for salinity (primary Level
1 flag)" ;
salinity_qc_flag:standard_name = "sea_water_practical_salinity status_flag" ;
salinity_qc_flag:quality_control_convention = "Proposed IODE qc scheme March
2012" ;
salinity_qc_flag:valid_min = 1 ;
salinity_qc_flag:valid_max = 9 ;
salinity_qc_flag:flag_values = 1b, 2b, 3b, 4b, 9b ;
salinity_qc_flag:flag_meanings = "good not_evaluated_or_unknown suspect bad
missing" ;
salinity_qc_flag:coordinates = "time latitude longitude pressure" ;
byte salinity_sd_test(pressure) ;
salinity_sd_test:long_name = "qc flag for monthly salinity standard deviation
test (secondary L2 flag)"
salinity_sd_test:quality_control_convention = "Proposed IODE qc scheme March
2012" ;
salinity_sd_test:valid_min = 0 ;
salinity_sd_test:valid_max = 2 ;
salinity_sd_test:flag_values = 0b, 1b, 2b ;
salinity_sd_test:flag_meanings = "passed failed unknown" ;
salinity_sd_test:coordinates = "time latitude longitude pressure" ;
int profile ; //Unique integer to identify each profile
profile:long_name = "profile identifier"
....
Andrew Walsh
----- Original Message -----
From: "Hedley, Mark" <mark.hedley at metoffice.gov.uk<mailto:mark.hedley at metoffice.gov.uk>>
To: <ngalbraith at whoi.edu<mailto:ngalbraith at whoi.edu>>; <cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>>
Sent: Thursday, February 14, 2013 10:40 PM
Subject: Re: [CF-metadata] Ancillary Data
Hi Nan
I think I understand you approach, it seems logical and helpful to me.
It feels like all your examples are in my category b:
b. reference a subset of the file dimensions referenced by the data variable
with the ancillary_variables attribute
and thus relate to the data variable in the same way auxiliary_coordinates do,
just with different inferred semantics.
many thanks for the feedback
mark
-----Original Message-----
From: CF-metadata on behalf of Nan Galbraith
Sent: Wed 13/02/2013 19:55
To: cf-metadata at cgd.ucar.edu<mailto:cf-metadata at cgd.ucar.edu>
Subject: Re: [CF-metadata] Ancillary Data
I use a singleton variable to record the magnetic correction that's been
applied to my wind and/or current variables; it's dim(1) and is listed as
an ancillary to any variables that have been affected by the rotation.
It could be an attribute of each of those variables, but making it a
free-standing
variable lets me give it units and attributes. That's important because I
record the model name and version date, the URL of the NGDC calculator
site,
the inputs to the model (date and location for which the calculation was
done)
and the estimated rate of change.
Maybe this is just a technicality - but I also use an empty 'container'
variable for
instruments, which has ancillary variables with a depth dimension for
manufacturer,
model, serial number, and reference URL. The instrument variable is
tied to 'obs data'
variables via NODC's 'instrument' attribute, but has no dimensions of
its own; the
individual component variables (like serial number) have a dimension
that matches
the depth dimension of the obs data variables.
- Nan
On 2/13/13 12:15 PM, Hedley, Mark wrote:
Hello CF community
I have been perusing the CF conventions again, particularly the section on
Ancillary Data
http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.6/ch03s04.html
The conventions make the statement:
The nature of the relationship between variables associated via
ancillary_variables must be determined by other attributes.
The example given provides data variables which reference the same dimensions
in the file: thus they data arrays are the same size. However, the
conventions do not seem to mandate this.
I am interested in the use of the
ancillary_variables
attribute in real world datasets.
Do people have examples they can share of ancillary datasets which:
a. reference the same file dimensions as the data variable with the
ancillary_variables attribute references
b. reference a subset of the file dimensions referenced by the data
variable with the ancillary_variables attribute
c. reference file dimensions which are not referenced by the data variable
with the ancillary_variables attribute
I am particularly interested in examples of case c, as I feel this is
markedly different from cases a and b and requires a different kind of
support if it is in use.
many thanks
mark
_______________________________________________
CF-metadata mailing list
CF-metadata at cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
--
*******************************************************
* Nan Galbraith Information Systems Specialist *
* Upper Ocean Processes Group Mail Stop 29 *
* Woods Hole Oceanographic Institution *
* Woods Hole, MA 02543 (508) 289-2444 *
*******************************************************
_______________________________________________
CF-metadata mailing list
CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
_______________________________________________
CF-metadata mailing list
CF-metadata at cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
_______________________________________________
CF-metadata mailing list
CF-metadata at cgd.ucar.edu<mailto:CF-metadata at cgd.ucar.edu>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
_______________________________________________
CF-metadata mailing list
CF-metadata at cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20130315/93d8cc77/attachment-0001.html>
Received on Fri Mar 15 2013 - 05:37:08 GMT
