Hi Steve,
Your stern but steady caution about the tension between flexibility
and interoperability is well-taken here.
> What have been the down sides to the
> use of groups and hierarchies?
Those who respond to this please be careful to distinguish between
HDF5 _allowed_ hierarchies and netCDF4-API _exposed_ hierarchies.
In the context of CF we strongly suggest allowing only the latter
(netCDF4) amount of flexibility. Lack of coordinates and cyclic
groups are downsides to interoperability of some HDF5 datasets with
themselves and with netCDF. Use of these features is diminishing
as a result. Evolution in action.
> How could those downsides have been
> minimized through more restrictive conventions?
To me, the CDM/netCDF4 API exposes enough flexibility and power.
NASA is moving towards HDF datasets that can be completely represented
by the netCDF4 API. Not there yet, and may never get there totally,
but a good chunk of Level 3 data is there now.
It would IMHO be better if group metadata inheritance were enshrined
by convention, rather than being implicit or ignored.
Best,
cz
> Hi Corey,
>
> Your question hits on the underlying dilemma. CF is more _powerful_
> when it offers the greatest possible flexibility for creators of files;
> like a programming language it enables you to go wherever your
> imagination can lead you. But CF is more _interoperable_ when it
> restricts the ways you may organize your file in enough to ensure that
> both the people and the machines receiving it will know (without
> exploration) how to pull semantically meaningful data from it. I think
> most everone would agree that the reason we create conventions is in
> order to restrict behavior. The battle lines get drawn over how
> severely we restrict it. In these email dialogs I have several time
> used the quotation '/To create quality software [standards], the ability
> to say ?no? is usually far more important than the ability to say
> ?yes.?/' (The Rise and Fall of CORBA (*)
> <http://queue.acm.org/detail.cfm?id=1142044>). It's a bummer to be a
> wet blanket, but it's a bummer to watch a standard go south, too. And
> plenty of them do ....
>
> We have not yet touched on the impacts that embedding groups and
> hierarchies into files may have on the need to aggregate files along
> their time axes; or on how to make sure that the way groups and
> hierarchies are used doesn't stand in the way of generating quality
> metadata that describes the contents of a CF file. NASA and other HDF5
> projects no doubt have tons of experiences in these issues that would be
> very interesting to hear about. What have been the down sides to the
> use of groups and hierarchies? How could those downsides have been
> minimized through more restrictive conventions?
>
> - Steve
>
> (*) thanks to Russ Rew for contributing this citation into the CF
> discussions long ago
>
--
Charlie Zender, Earth System Sci. & Computer Sci.
University of California, Irvine 949-891-2429 )'(
Received on Thu Sep 19 2013 - 15:11:18 BST
