⇐ ⇒

[CF-metadata] CF Section 5.3 "Reduced Horizontal Grid" and Section 8.2 "compression by gathering" somewhat inconsistent

From: Steve Hankin <Steven.C.Hankin>
Date: Tue, 21 Oct 2008 16:36:03 -0700

Jonathan Gregory wrote:
> Dear John and Steve
>
> I may have misunderstood, but I think Ex 5.3 is correct. It is showing how
> the data variable PS and the auxiliary coord variables lon and lat, which are
> all 2D in principle, may all be compressed by gathering. (As John says, lon and
> lat are not 1D in this example.) The uncompressed dimensions londim and latdim
> are needed in order to decode the values of rgrid, for scattering back to the
> original 2D arrays.
Hi Jonathan,

The difficulty with Section 5.3 is not that it is strictly speaking
"incorrect". It is that with the advent of compressed chunking in
netCDF-4-classic, the gains from this feature no longer justify the cost
in complexity. This seems like the right time to announce an intention
to deprecate this method of compressing data.

Virtually all software developers who have attempted to develop general
clients for CF agree that CF suffers from too much complexity. I wonder
how many applications even implement the Reduced Horizontal Grid feature
of CF. Envision the complexity that is imposed on an application that
it is asked to extract a time series at an XY location from a Reduced
Horizontal Grid. Imho the feature is a clear candidate for removal
from CF.
> The compress attribute indicates that the rgrid variable
> is to be treated in this way, just like Sect 8. So I don't think there is a
> problem, but perhaps it could be clarified what is going on.
>
Example 8.2 uses the "Reduced Horizontal (i.e. *XY*)" encoding to
compress *in Z*, too. This is just a semantics problem in the name of
the feature, since the technique is, in principle, applicable to any
number of dimensions.
> I agree that this scheme doesn't indicate anything about the structure of the
> grid, not that it is a Gaussian grid, for instance. I think it was put in to
> show *how disk space could be saved*, that's all.
>
If you'll forgive me for getting onto a soap box: Disk space is not
saved if there is no client that can read the data. Instead that disk
space is 100% lost. In making judgments about CF I believe we need to
do more to balance the client-writer's perspective against the
data-writer's perspective.

    - Steve
> Best wishes
>
> Jonathan
> _______________________________________________
> CF-metadata mailing list
> CF-metadata at cgd.ucar.edu
> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>

-- 
Steve Hankin, NOAA/PMEL -- Steven.C.Hankin at noaa.gov
7600 Sand Point Way NE, Seattle, WA 98115-0070
ph. (206) 526-6080, FAX (206) 526-6744
"The only thing necessary for the triumph of evil is for good men
to do nothing." -- Edmund Burke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20081021/873176cf/attachment-0002.html>
Received on Tue Oct 21 2008 - 17:36:03 BST

This archive was generated by hypermail 2.3.0 : Tue Sep 13 2022 - 23:02:40 BST

⇐ ⇒