Ed,
Yes, the idea is to save space and also to be consistent with
heritage data products that use bit-masked flags.
Yes, compression would probably save the space too (probably even more!)
HDF does allow "c" like structures, would this help in mapping to
CF? I had assumed that a single HDF dataset was a single CF variable.
I'm afraid I'm unfamiliar with the details of netCDF API, I was just
looking at the CF metadata attributes and trying to map them to the
NPOESS metadata attributes.
-Rich
On Nov 14, 2006, at 10:10 AM, Ed Hartnett wrote:
> Jonathan Gregory <j.m.gregory at reading.ac.uk> writes:
>
>> Dear Richard
>>
>> I am not sure I quite understand what you are saying. Is the
>> problem that
>> you want to use less than a byte for your flags and pack them into
>> words,
>> but that CF does not provide a convention for packing that handles
>> less
>> than a byte? Does HDF allow you to define datatypes of n bits? If
>> so, perhaps
>> netCDF is going to support this at some point.
>
> Sorry, no. HDF5 does not support the creation of types other than 8,
> 16, 32, and 64 bits. (There is a way to "fake" it, but behind the
> scenes, if you define an 11-bit data type, HDF5 will round up the size
> and store each value in 16 bits anyway.
>
> HDF5 (and netCDF-4) has a way of defining data types of n bytes (the
> opaque type), but not n bits.
>
>>
>> This strikes me as a problem it would be awkward to deal with in
>> CF. It's not
>> a metadata limitation, but one which refers to file format. CF is
>> based on
>> netCDF variables, and in this case you want to pack several
>> variables into one,
>> which "breaks" the logical model.
>>
>
> I didn't quite follow the original message either. Is the intention to
> do this to save disk space?
>
> I have to wonder whether the use of compression might be almost as
> useful for saving space, but considerably easier for everyone to
> program with!
>
> One could use gzip on netCDF-3 files, and one can turn on compression
> in netCDF-4 on a per variable basis (that is, turn it on for some and
> not for others.)
>
> I also wonder if the netCDF-4 compound type might be useful here. It
> allows a user-defined data type like a C struct...
>
> Thanks,
>
> Ed
> --
> Ed Hartnett -- ed at unidata.ucar.edu
>
- - - - - - - - - - -
Richard E. Ullman
NASA
Goddard Space Flight Center
richard.e.ullman at nasa.gov
+1 301 614-5228
NESDIS/IPO
System Engineering, Data/Information Architecture
richard.ullman at noaa.gov
+1 301 713 4866
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20061116/eeaed4c8/attachment-0002.html>
Received on Thu Nov 16 2006 - 14:36:27 GMT
