Hi Martin,
Let me enumerate a third disaggregation method to your list:
> There should probably be two ways for converting group files to flat
> files:
> a) flatten everything into one file with "." separated name spaces
> b) flatten groups into individual files (tools like NCO could then
> use the name space as part of the file names to be generated)
The c) I would add is flattening groups into a single file and
discarding group names completely. When variable name conflicts arise,
then fail. This is useful for flattening hierarcichal files that
contain groups of data with no namespace conflicts.
NCO's ncks does b) and c) now. I do not know yet of any tool that
does method a), although ncl_convert2nc may do something like this
when it flattens HDF files into netCDF files.
Regards,
cz
Le 17/09/2013 10:12, Schultz, Martin a ?crit :
> Hi again,
>
> I fully support Jim's view! Let's not get hung up on whether groups/hierarchies are good or bad. Instead we should consider them a reality and an option rather than a must. They are a bit like a suitcase which you can pack and unpack to carry things around or store them in your basement until you need them again. In your closet (i.e. on your harddrive) you may prefer to have flat access to your shirts and socks, while on travel (distributing datasets to students, for example) you may prefer some sort of packaging. And don't tell me there would be a unique way how to pack your suitcase!
>
>> treat groups as files
> There should probably be two ways for converting group files to flat files:
> a) flatten everything into one file with "." separated name spaces
> b) flatten groups into individual files (tools like NCO could then use the name space as part of the file names to be generated)
> Conversely, one may want to define rules for aggregating flat files into a hierarchy - allowing for different "models" for how data should be structured
> [this is probably beyond CF but related as far as attributes are concerned]
>
>> allow for the concept of inheritance of dimensions (which is native to netCDF-4)
> I may be na?ve, but this seems relatively straightforward to me as long as we stay clear of multiple or circular hierarchies. We should be aware of the consequences, though: not all HDF5 files may fit into the new CF framework then! It would actually be good to get a view from the HDF5 experts as to how widespread the multiple ancestor or circular "features" of HDF5 are exploited in real datasets (what would we miss if we adhere to the netcdf4 concept?).
>
>> ... and attributes (which would be a CF convention)
--
Charlie Zender, Earth System Sci. & Computer Sci.
University of California, Irvine 949-891-2429 )'(
Received on Tue Sep 17 2013 - 15:11:02 BST
