⇐ ⇒

[CF-metadata] CF-1.6 Conformance Requirements/Recommendations

From: Bert Jagers <bert.jagers>
Date: Tue, 27 Mar 2012 10:28:16 +0200

Hi all,

Yesterday, I briefly mentioned privately another use case for missing
data in auxiliary coordinates to Jonathan.
I have now uploaded some text and images for that case on our
wiki-server such that I can distribute it to the wider community.
See:
http://publicwiki.deltares.nl/display/NETCDF/Structured+grids#Structuredgrids-Missingcoordinates
In our case it concerns missing data in latitude/longitude (or
projected) coordinate for a curvilinear grid.
Our models run most of the time on structured curvilinear grids in which
coordinates are only associated with part of the full grid matrix.

Best regards,

Bert
----
Nan Galbraith wrote:
> Hi  all -
>
>>  From a CDM developer perspective, an auxiliary coordinate is "just as
>>  good" as a regular coordinate variable. The extra requirements on
>>  coordinate variables are helpful in knowing when to optimize, eg
>>  monotonicity allows one to efficiently find the index given the
>>  coordinate value.
>
> Are we reaching agreement that fill values are allowed in auxiliary
> coordinate variables, aside from
>
>>  In my understanding, we have always prohibiting missing data in aux
>>  coord vars, and in section 9 we explicitly allowed for the first
>>  time. Evidently we should be clear, one way or the other (which was
>>  one of the intentions of the defect ticket I opened).
>
> Should we be discussing this on the trac ticket? That would provide a
> better trail - Mailman isn't very good if you're searching for a thread
> like this - the subject doesn't mention coordinates.
>
> Thanks -
> Nan
>
>
>>
>> On 3/26/2012 10:05 AM, Jim Biard wrote:
>>> I am working with satellite data, and I, for example, have 
>>> timestamps that arrive in the data stream along with sensor 
>>> measurements.  I can have independent missing values in both my time 
>>> variable and my measurement variables.  I want to preserve all the 
>>> incoming data, and the restriction on "true" coordinate variables 
>>> having no missing values prevents me from designating my time 
>>> variable as a coordinate variable.  If I want to represent the 
>>> relationship between the time variable and the measurement 
>>> variables, the only recourse I have is to designate the time 
>>> variable as an auxiliary coordinate variable in my measurement 
>>> variables.
>>>
>>> In the observational world, you cannot always be assured of meeting 
>>> all the restrictions imposed on "true" coordinate variables.  In 
>>> fact, other restrictions imposed on coordinate variables might not 
>>> be met (monotonicity, for example), even though the contents of the 
>>> variable do function properly as coordinate information for another 
>>> variable.  The only mechanism that I am aware of within CF to 
>>> document the relationship is the auxiliary coordinate attribute.
>>>
>>> On Mon, Mar 26, 2012 at 11:20 AM, Nan Galbraith <ngalbraith at whoi.edu 
>>> <mailto:ngalbraith at whoi.edu>> wrote:
>>>
>>>     Hi Jonathan -
>>>
>>>     For underway CTD profiles (gliders and floats, too, I'd think) if
>>>     the pressure
>>>     sensor fails intermittently, you can approximate Z by
>>>     interpolating over
>>>     time, assuming there are good values along the track.  In "final"
>>>     data, we might
>>>     do that, but we might like to present raw data in CF files, too.
>>>     So, yes, this data
>>>     would be useful, with fill values here and there in the pressure
>>>     record.
>>>
>>>     Are we getting into a grey area that might be outside the scope
>>>     of the CF
>>>     standard -  judgements made about the usefulness of data?      
>>> Having all your
>>>     coordinates seems like an excellent  NetCDF "best practice" and
>>>     could certainly
>>>     be a "rule" for many tools, but it could preempt the use of the
>>>     CF standard for
>>>     some kinds of observational data.
>>>
>>>     Cheers -
>>>     Nan
>>>
>>>
>>>     On 3/26/12 10:48 AM, Jonathan Gregory wrote:
>>>
>>>         Dear Nan and John
>>>
>>>         It's a good thing we're having this discussion! In my
>>>         understanding, we have
>>>         always prohibiting missing data in aux coord vars, and in
>>>         section 9 we
>>>         explicitly allowed for the first time. Evidently we should be
>>>         clear, one way
>>>         or the other (which was one of the intentions of the defect
>>>         ticket I opened).
>>>
>>>         The restriction on coord vars not having missing data is, I
>>>         think, hard to
>>>         avoid because they are required to be distinct and monotonic.
>>>
>>>         In Nan's case:
>>>
>>>             For something like towed CTD data, you might have a
>>>             period of time where
>>>             data from the pressure sensor is missing. If neither the
>>>             coordinate or aux
>>>             coordinate can contain null values, does this mean the
>>>             only options are
>>>             interpolating Z or removing that section of data?
>>>
>>>         When the sensor is missing, does that mean the data can't be
>>>         geolocated? As
>>>         you know, geolocation is one thing CF tries hard to do. Is
>>>         the data useful if
>>>         you don't know where it comes from? Perhaps you know in some
>>>         other way?
>>>
>>>         In John's case
>>>
>>>             Consider a geosynch satellite with lat/lon aux
>>>             coordinates. the
>>>             nature of the image is that there are many points around
>>>             the outside
>>>             of the image that are not located on the earth. i dont
>>>             see any good
>>>             choices other than to put missing values there for 
>>> lat/lon..
>>>
>>>         Bert has coincidentally mentioned a similar case (not on the
>>>         list). I tend to
>>>         agree that if index space contains locations that simply do
>>>         not exist, you
>>>         could put missing data in aux coord vars, but I think this
>>>         needs a change to
>>>         the CF convention.
>>>
>>>             To add insult to injury, it seems possible that there are
>>>             valid data
>>>             values at these locations. Not sure about that however.
>>>             Can anyone
>>>             with geosynch data expertise comment?
>>>
>>>         As in Nan's case, I am curious about how the data can be
>>>         useful if it doesn't
>>>         have a location.
>>>
>>>         Best wishes
>>>
>>>         Jonathan
>>>         _______________________________________________
>>>         CF-metadata mailing list
>>>         CF-metadata at cgd.ucar.edu <mailto:CF-metadata at cgd.ucar.edu>
>>>         http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>>>
>>>
>>>
>>>     --     *******************************************************
>>>     * Nan Galbraith        Information Systems Specailist *
>>>     * Upper Ocean Processes Group            Mail Stop 29 *
>>>     * Woods Hole Oceanographic Institution                *
>>>     * Woods Hole, MA 02543 (508) 289-2444 <tel:%28508%29%20289-2444> *
>>>     *******************************************************
>>>
>>>
>>>
>>>     _______________________________________________
>>>     CF-metadata mailing list
>>>     CF-metadata at cgd.ucar.edu <mailto:CF-metadata at cgd.ucar.edu>
>>>     http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>>>
>>>
>>>
>>>
>>> -- 
>>> Jim Biard
>>> Research Scholar
>>> Cooperative Institute for Climate and Satellites
>>> Remote Sensing and Applications Division
>>> National Climatic Data Center
>>> 151 Patton Ave, Asheville, NC 28801-5001
>>>
>>> jim.biard at noaa.gov <mailto:jim.biard at noaa.gov>
>>> 828-271-4900
>>>
>>>
>>>
>>> _______________________________________________
>>> CF-metadata mailing list
>>> CF-metadata at cgd.ucar.edu
>>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>>
>>
>> _______________________________________________
>> CF-metadata mailing list
>> CF-metadata at cgd.ucar.edu
>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>
>
 
DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited.
The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.
Received on Tue Mar 27 2012 - 02:28:16 BST

This archive was generated by hypermail 2.3.0 : Tue Sep 13 2022 - 23:02:41 BST

⇐ ⇒