I agree that it is best to follow standard netcdf recommendations when
possible and avoid extra metadata which will not be supported outside
of CF (where possible and practical).
Etienne
On Wed, Sep 26, 2012 at 12:08 AM, Russ Rew <russ at unidata.ucar.edu> wrote:
> Hi Phil,
>
>> The final para of section 2.5.1 of the CF conventions document describes
>> the use of the _FillValue (or missing_value) attribute in the case of
>> data packed using the scale-and-offset method. What is not clear - at
>> least to me - is what the preferred application behaviour should be in
>> the case where the data is unpacked and then written out to a new netCDF
>> file. In particular, what fill value should be used for the unpacked
>> data variable?
>>
>> I presume that one wouldn't normally want to use the original fill value
>> since that value (typically an 8- or 16-bit integer) is quite likely to
>> fall within the normal range of the unpacked data (typically a 32- or
>> 64-bit float).
>>
>> In the absence of explicitly setting a fill value attribute on the
>> unpacked data variable I assume that the netCDF default fill value will
>> be used for the data type in question. Which may not always be desirable
>> (certainly not for 32-bit floats, where the default fill value can give
>> rise to subtle precision-related problems).
>>
>> With this in mind, I was wondering if there is any merit in defining a
>> new attribute called, say, _UnpackedFillValue (or
>> unpacked_missing_value)? If client software detected this attribute then
>> the associated value (same data type as the scale_factor and add_offset
>> attributes) would be used as the fill value for the unpacked data
>> variable.
>>
>> Alternatively, the names _FillValueUnpacked (missing_value_unpacked)
>> might be preferable since they would then appear together pair-wise in
>> CDL-type listings, e.g.
>>
>> short pkd_var(z, y, x) :
>> ...
>> pkd_var:_FillValue =3D -32768 ;
>> pkd_var:_FillValueUnpacked =3D -1.0e30 ;
>> pkd_var:add_offset =3D 42.0 ;
>> pkd_var:scale_factor =3D 1234.0 ;
>> ...
>>
>>
>> Any merit/mileage in this idea?
>
> A more detailed recommendation for treating special values such as
> _FillValue or missing_value for packed data is described, with
> associated formulas, in the "Packed Data Values" section of a
> best-practices document that we wrote a few years ago:
>
> http://www.unidata.ucar.edu/netcdf/docs/BestPractices.html#Packed%20Data%20Values
>
> It provides a recommendation for ensuring the unpacked special value is
> not in the range of other unpacked data values. If that recommendation
> is followed, I think there is no need for an additional
> _FillValueUnpacked (or missing_value_unpacked) attribute.
>
> If you agree that is an acceptable approach, perhaps we should add it to
> CF ...
>
> --Russ
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
Received on Wed Sep 26 2012 - 05:47:01 BST
