Hi Phil,
> The final para of section 2.5.1 of the CF conventions document describes
> the use of the _FillValue (or missing_value) attribute in the case of
> data packed using the scale-and-offset method. What is not clear - at
> least to me - is what the preferred application behaviour should be in
> the case where the data is unpacked and then written out to a new netCDF
> file. In particular, what fill value should be used for the unpacked
> data variable?
>
> I presume that one wouldn't normally want to use the original fill value
> since that value (typically an 8- or 16-bit integer) is quite likely to
> fall within the normal range of the unpacked data (typically a 32- or
> 64-bit float).
>
> In the absence of explicitly setting a fill value attribute on the
> unpacked data variable I assume that the netCDF default fill value will
> be used for the data type in question. Which may not always be desirable
> (certainly not for 32-bit floats, where the default fill value can give
> rise to subtle precision-related problems).
>
> With this in mind, I was wondering if there is any merit in defining a
> new attribute called, say, _UnpackedFillValue (or
> unpacked_missing_value)? If client software detected this attribute then
> the associated value (same data type as the scale_factor and add_offset
> attributes) would be used as the fill value for the unpacked data
> variable.
>
> Alternatively, the names _FillValueUnpacked (missing_value_unpacked)
> might be preferable since they would then appear together pair-wise in
> CDL-type listings, e.g.
>
> short pkd_var(z, y, x) :
> ...
> pkd_var:_FillValue =3D -32768 ;
> pkd_var:_FillValueUnpacked =3D -1.0e30 ;
> pkd_var:add_offset =3D 42.0 ;
> pkd_var:scale_factor =3D 1234.0 ;
> ...
>
>
> Any merit/mileage in this idea?
A more detailed recommendation for treating special values such as
_FillValue or missing_value for packed data is described, with
associated formulas, in the "Packed Data Values" section of a
best-practices document that we wrote a few years ago:
http://www.unidata.ucar.edu/netcdf/docs/BestPractices.html#Packed%20Data%20Values
It provides a recommendation for ensuring the unpacked special value is
not in the range of other unpacked data values. If that recommendation
is followed, I think there is no need for an additional
_FillValueUnpacked (or missing_value_unpacked) attribute.
If you agree that is an acceptable approach, perhaps we should add it to
CF ...
--Russ
Received on Tue Sep 25 2012 - 21:08:26 BST
