[CF-metadata] CF provisional standards

From: John Caron <caron>
Date: Mon, 20 Nov 2006 13:06:04 -0700

Egil St?ren wrote:
> John Caron wrote:
>
>> Let me just appreciate that any individual interested in CF is already
>> doing much more than is the "normal practice" of generating files with
>> no published documentation at all. So even if some small group of data
>> publishers creates some special thing, at least they documented it,
>> and if its even on CF web page where I might know to look for it, they
>> deserve thanks. So all the variant positions being argued here, IMO,
>> are a huge improvement over usual practice.
>>
>> We have the "who pays vs who benefits" social dilemna here. Data
>> writers must pay the price of doing the conventions right for the
>> benefit of data readers. Data readers must pay the price of suboptimal
>> conventions for the benefit of writers who need fast solutions, and
>> dont want to rewrite their data. Depending on which role you play in
>> your day job, you will have some visceral feelings on these issues. We
>> need both POVs and the solutions will be tradeoffs.
>
>
> Maybe it is more clearifying to consider three groups with different
> interests:
>
> 1. Data writers
> 2. Writers of general software
> 3. Users of both data and software
>
> What you and others mean by "data readers" seem to correspond to group 2
> above.

Yes, good point, perhaps "Software for reading general files" is another description.

Naturally, this group has a noble goal of providing software that
> can easily be used directly against files produced by the data writers.
> This is a goal that is very difficult to obtain, and in general, users
> have to close the gap between the actual files they must work with, and
> the software tools available to them. This is done by scripting
> languages and tools like NCO etc.

In general, one has to do this. But I think one goal of the CF Convention is to try to eliminate this intervention step as much as possible.

I think most users are happy with this
> situation, since it gives them a sense of freedom.

I guess some are happy, but there are many who are unable to close this gap.

>
> Accordingly, most uses of CF compliant files will involve a stage where
> the user investigates the actual kind of files she/he will work with.
> Although these files are CF-compliant, they may also obey some local
> standards the user have to consider. Typically, the user will set up a
> runtime environment for a) converting source files to some type of
> destination files, and b) using general software tools for producing
> results from the destination files. Considering this, the most important
> thing about CF is that it is understandable from a human point of view.
> A CF that is easy to implement in general software tools is a second
> priority. We already have general software tools that work on the netCDF
> level that will satisfy the needs of the users.
>
> As a conclusion, I tend to look at CF primarily as a tool for data
> writers to create files rich in metadata that are well understood by
> humans.

Yes, understanding by humans is the first and most important thing to do.

Also useful is to see how far we can make software "understand" the data. Being able to georeference, display, and subset the data, I think are doable goals for many, perhaps most, important datasets.

Regards,
John
Received on Mon Nov 20 2006 - 13:06:04 GMT