Didn't realize that this message is from cf-satellite group. The example
that I gave is for in-situ data. But as can be seen, similar issue
arises for in-situ data as well. CCing this to CF-METADATA group.
Upendra
On 11/1/2011 10:47 AM, Upendra Dadi wrote:
> The same issue occurs with World Ocean Database which consists of
> mainly profile data. Each profile typically consists of several
> variables measured along the depth. The quality flags used for all the
> variable are same. I have attached a snippet below of how a CF
> compliant netCDF would look. The over-head due to repeating the
> quality flags is not much when the data is stored is a multi-profile
> format, but when each profile is stored in an individual file, the
> overhead becomes significant, since these flags have to be present in
> each file.
>
> Upendra
>
>
> float temperature(z) ;
> temperature:standard_name = "sea_water_temperature" ;
> temperature:long_name = "sea water temperature" ;
> temperature:units = "degree_C" ;
> temperature:coordinates = "time lon lat z" ;
> temperature:WODprofile_flag = 0 ;
> temperature:_FillValue = -1.e+10f ;
> temperature:grid_mapping = "crs" ;
> temperature:cell_methods = "time:point longitude:point
> latitude:
> point z:point" ;
> temperature:ancillary_variables = "temperatu
> re_WODflag" ;
> int temperature_WODflag(z) ;
> temperature_WODflag:standard_name =
> "sea_water_temperature statu
> s_flag" ;
> temperature_WODflag:flag_values = "0, 1, 2, 3, 4, 5,
> 6, 7, 8, 9"
> ;
> temperature_WODflag:flag_meanings = "accepted_value
> range_outlie
> r failed_inversion_check failed_gradient_check
> observed_level_bullseye_flag_and_
> zero_gradient_check combined_gradient_and_inversion_checks
> failed_range_and_inve
> rsion_checks failed_range_and_gradient_checks
> failed_range_and_questionable_data
> _checks failed_range_and_combined_gradient_and_inversion_checks" ;
> temperature_WODflag:reference = "" ;
>
>
> float salinity(z) ;
> salinity:standard_name = "sea_water_salinity" ;
> salinity:long_name = "sea water salinity" ;
> salinity:units = "" ;
> salinity:coordinates = "time lon lat z" ;
> salinity:WODprofile_flag = 0 ;
> salinity:_FillValue = -1.e+10f ;
> salinity:grid_mapping = "crs" ;
> salinity:cell_methods = "time:point longitude:point
> latitude:poi
> nt z:point" ;
> salinity:ancillary_variables = "salinity_WODfla
> g" ;
> int salinity_WODflag(z) ;
> salinity_WODflag:standard_name = "sea_water_salinity
> status_flag
> " ;
> salinity_WODflag:flag_values = "0, 1, 2, 3, 4, 5, 6,
> 7, 8, 9" ;
> salinity_WODflag:flag_meanings = "accepted_value
> range_outlier f
> ailed_inversion_check failed_gradient_check
> observed_level_bullseye_flag_and_zer
> o_gradient_check combined_gradient_and_inversion_checks
> failed_range_and_inversi
> on_checks failed_range_and_gradient_checks
> failed_range_and_questionable_data_ch
> ecks failed_range_and_combined_gradient_and_inversion_checks" ;
> salinity_WODflag:comment = "" ;
>
>
> float oxygen(z) ;
> oxygen:standard_name =
> "volume_fraction_of_oxygen_in_sea_water"
> ;
> oxygen:long_name = "Oxygen" ;
> oxygen:units = "ml/l" ;
> oxygen:coordinates = "time lon lat z" ;
> oxygen:WODprofile_flag = 0 ;
> oxygen:_FillValue = -1.e+10f ;
> oxygen:grid_mapping = "crs" ;
> oxygen:cell_methods = "time:point longitude:point
> latitude:point
> z:point" ;
> oxygen:ancillary_variables = "oxygen_WODflag" ;
> int oxygen_WODflag(z) ;
> oxygen_WODflag:standard_name =
> "volume_fraction_of_oxygen_in_sea
> _water status_flag" ;
> oxygen_WODflag:flag_values = "0, 1, 2, 3, 4, 5, 6, 7,
> 8, 9" ;
> oxygen_WODflag:flag_meanings = "accepted_value
> range_outlier fai
> led_inversion_check failed_gradient_check
> observed_level_bullseye_flag_and_zero_
> gradient_check combined_gradient_and_inversion_checks
> failed_range_and_inversion
> _checks failed_range_and_gradient_checks
> failed_range_and_questionable_data_chec
> ks failed_range_and_combined_gradient_and_inversion_checks" ;
> oxygen_WODflag:comment = "" ;
>
>
> float phosphate(z) ;
> phosphate:standard_name =
> "mole_concentration_of_phosphate_in_se
> a_water " ;
> phosphate:long_name = "Phosphate" ;
> phosphate:units = "umol/l" ;
> phosphate:coordinates = "time lon lat z" ;
> phosphate:WODprofile_flag = 0 ;
> phosphate:_FillValue = -1.e+10f ;
> phosphate:grid_mapping = "crs" ;
> phosphate:cell_methods = "time:point longitude:point
> latitude:po
> int z:point" ;
> phosphate:ancillary_variables = "phosphate_WOD
> flag" ;
> int phosphate_WODflag(z) ;
> phosphate_WODflag:standard_name =
> "mole_concentration_of_phospha
> te_in_sea_water status_flag" ;
> phosphate_WODflag:flag_values = "0, 1, 2, 3, 4, 5, 6,
> 7, 8, 9" ;
> phosphate_WODflag:flag_meanings = "accepted_value
> range_outlier
> failed_inversion_check failed_gradient_check
> observed_level_bullseye_flag_and_ze
> ro_gradient_check combined_gradient_and_inversion_checks
> failed_range_and_invers
> ion_checks failed_range_and_gradient_checks
> failed_range_and_questionable_data_c
> hecks failed_range_and_combined_gradient_and_inversion_checks" ;
> phosphate_WODflag:comment = "" ;
>
>
> float silicate(z) ;
> silicate:standard_name =
> "mole_concentration_of_silicate_in_sea_
> water" ;
> silicate:long_name = "Silicate" ;
> silicate:units = "umol/l" ;
> silicate:coordinates = "time lon lat z" ;
> silicate:WODprofile_flag = 0 ;
> silicate:_FillValue = -1.e+10f ;
> silicate:grid_mapping = "crs" ;
> silicate:cell_methods = "time:point longitude:point
> latitude:point z:point" ;
> silicate:ancillary_variables = "silicate_sigfigs
> silicate_WODflag" ;
> int silicate_WODflag(z) ;
> silicate_WODflag:standard_name =
> "mole_concentration_of_silicate_in_sea_water status_flag" ;
> silicate_WODflag:flag_values = "0, 1, 2, 3, 4, 5, 6,
> 7, 8, 9" ;
> silicate_WODflag:flag_meanings = "accepted_value
> range_outlier failed_inversion_check failed_gradient_check
> observed_level_bullseye_flag_and_zero_gradient_check combined_gradient_an
> d_inversion_checks failed_range_and_inversion_checks
> failed_range_and_gradient_checks
> failed_range_and_questionable_data_checks
> failed_range_and_combined_gradient_and_inversion_checks" ;
> silicate_WODflag:comment = "" ;
>
>
> float nitrate(z) ;
> nitrate:standard_name =
> "mole_concentration_of_nitrate_in_sea_water" ;
> nitrate:long_name = "Nitrate" ;
> nitrate:units = "umol/l" ;
> nitrate:coordinates = "time lon lat z" ;
> nitrate:WODprofile_flag = 0 ;
> nitrate:_FillValue = -1.e+10f ;
> nitrate:grid_mapping = "crs" ;
> nitrate:cell_methods = "time:point longitude:point
> latitude:point z:point" ;
> nitrate:ancillary_variables = "nitrate_WODflag" ;
> int nitrate_WODflag(z) ;
> nitrate_WODflag:standard_name =
> "mole_concentration_of_nitrate_in_sea_water status_flag" ;
> nitrate_WODflag:flag_values = "0, 1, 2, 3, 4, 5, 6, 7,
> 8, 9" ;
> nitrate_WODflag:flag_meanings = "accepted_value
> range_outlier failed_inversion_check failed_gradient_check
> observed_level_bullseye_flag_and_zero_gradient_check
> combined_gradient_and
> _inversion_checks failed_range_and_inversion_checks
> failed_range_and_gradient_checks
> failed_range_and_questionable_data_checks
> failed_range_and_combined_gradient_and_inversion_checks" ;
> nitrate_WODflag:comment = "" ;
>
>
> float nitrite(z) ;
> nitrite:long_name = "Nitrite" ;
> nitrite:units = "umol/l" ;
> nitrite:coordinates = "time lon lat z" ;
> nitrite:WODprofile_flag = 0 ;
> nitrite:_FillValue = -1.e+10f ;
> nitrite:grid_mapping = "crs" ;
> nitrite:cell_methods = "time:point longitude:point
> latitude:point z:point" ;
> nitrite:ancillary_variables = "nitrite_WODflag" ;
> int nitrite_WODflag(z) ;
> nitrite_WODflag:flag_values = "0, 1, 2, 3, 4, 5, 6, 7,
> 8, 9" ;
> nitrite_WODflag:flag_meanings = "accepted_value
> range_outlier failed_inversion_check failed_gradient_check
> observed_level_bullseye_flag_and_zero_gradient_check
> combined_gradient_and
> _inversion_checks failed_range_and_inversion_checks
> failed_range_and_gradient_checks
> failed_range_and_questionable_data_checks
> failed_range_and_combined_gradient_and_inversion_checks" ;
> nitrite_WODflag:comment = "" ;
>
> On 10/31/2011 12:12 PM, Randy Horne wrote:
>> In the application I am working on, there are specific types of
>> products where where the same quality flags apply to multiple
>> variables. These products will be contained in a single NetCDF file.
>>
>> The current CF conventions dictate that quality flags are attached to
>> specific variables. The implication is that comforming with CF
>> conventions would require the same quality flags to be stored
>> multiple times in our NetCDF product files. This is potentially
>> problematic due to the resulting increase in product file size. Our
>> variables have on the order of 10s of millions of data points where
>> each has a distinct quality flag.
>>
>> This issue has come up before, and any design to optimize flags has
>> been dismissed due to the overriding objective of having variables
>> that are self-describing.
>>
>> Product file compression is a potential answer, but this presents its
>> own set of performance issues.
>>
>> Maybe flags could be handled in a similar manner as coordinates to
>> allow sharing among variables ?
>>
>>
>>
>> ..............End of Message ...............................-->
>>
>>
>>
>>
>>
>> _______________________________________________
>> cf-satellite mailing list
>> cf-satellite at unidata.ucar.edu
>> For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>
> _______________________________________________
> cf-satellite mailing list
> cf-satellite at unidata.ucar.edu
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
Received on Tue Nov 01 2011 - 08:59:21 GMT
