Begin forwarded message:
> From: Richard Ullman <Richard.Ullman at noaa.gov>
> Date: November 16, 2006 4:36:27 PM EST
> To: Ed Hartnett <ed at unidata.ucar.edu>
> Cc: Jonathan Gregory <j.m.gregory at reading.ac.uk>, cf-
> metadata at cgd.ucar.edu
> Subject: Re: [CF-metadata] packed flags
>
> Ed,
>
> Yes, the idea is to save space and also to be consistent with
> heritage data products that use bit-masked flags.
> Yes, compression would probably save the space too (probably even
> more!)
> HDF does allow "c" like structures, would this help in mapping to
> CF? I had assumed that a single HDF dataset was a single CF variable.
>
> I'm afraid I'm unfamiliar with the details of netCDF API, I was
> just looking at the CF metadata attributes and trying to map them
> to the
> NPOESS metadata attributes.
>
> -Rich
>
> On Nov 14, 2006, at 10:10 AM, Ed Hartnett wrote:
>
>> Jonathan Gregory <j.m.gregory at reading.ac.uk> writes:
>>
>>> Dear Richard
>>>
>>> I am not sure I quite understand what you are saying. Is the
>>> problem that
>>> you want to use less than a byte for your flags and pack them
>>> into words,
>>> but that CF does not provide a convention for packing that
>>> handles less
>>> than a byte? Does HDF allow you to define datatypes of n bits? If
>>> so, perhaps
>>> netCDF is going to support this at some point.
>>
>> Sorry, no. HDF5 does not support the creation of types other than 8,
>> 16, 32, and 64 bits. (There is a way to "fake" it, but behind the
>> scenes, if you define an 11-bit data type, HDF5 will round up the
>> size
>> and store each value in 16 bits anyway.
>>
>> HDF5 (and netCDF-4) has a way of defining data types of n bytes (the
>> opaque type), but not n bits.
>>
>>>
>>> This strikes me as a problem it would be awkward to deal with in
>>> CF. It's not
>>> a metadata limitation, but one which refers to file format. CF is
>>> based on
>>> netCDF variables, and in this case you want to pack several
>>> variables into one,
>>> which "breaks" the logical model.
>>>
>>
>> I didn't quite follow the original message either. Is the
>> intention to
>> do this to save disk space?
>>
>> I have to wonder whether the use of compression might be almost as
>> useful for saving space, but considerably easier for everyone to
>> program with!
>>
>> One could use gzip on netCDF-3 files, and one can turn on compression
>> in netCDF-4 on a per variable basis (that is, turn it on for some and
>> not for others.)
>>
>> I also wonder if the netCDF-4 compound type might be useful here. It
>> allows a user-defined data type like a C struct...
>>
>> Thanks,
>>
>> Ed
>> --
>> Ed Hartnett -- ed at unidata.ucar.edu
>>
>
> - - - - - - - - - - -
> Richard E. Ullman
> NASA
> Goddard Space Flight Center
> richard.e.ullman at nasa.gov
> +1 301 614-5228
> NESDIS/IPO
> System Engineering, Data/Information Architecture
> richard.ullman at noaa.gov
> +1 301 713 4866
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20061116/68164e3e/attachment-0002.html>
Received on Thu Nov 16 2006 - 14:37:56 GMT
