All,
Jeez, my mind must still be on vacation - I realized after I hit
"send" that I got that inner/outer dimension argument exactly
backwards! The way we write ADCP data currents now (with time varying
most slowly) should lead to slower time series extraction at a certain
depth. But that doesn't negate points (1), (2) and (3).
-Rich
On Thu, Dec 30, 2010 at 7:39 AM, Rich Signell <rsignell at usgs.gov> wrote:
> Roy, Nan & CF-ers,
>
> I imagine Nan would be okay with her ADCP currents to be type
> "timeSeriesProfile" as long as:
> 1) ?both "timeSeriesProfile" ?and "timeSeries" featureTypes can exist
> in the same dataset. ?(as Roy points out)
> 2) ?she had a function to extract a featureType "timeSeries" from a
> featureType "timeSeriesProfile".
> 3) ?the extraction process (2) does not cause a noticable slowdown in
> her workflow
>
> In the past, we have stored both ADCP currents (time,depth) and
> temperature (time) as degenerate "grid" featureTypes
> V(time,depth,lat,lon) and T(time,lat,lon) with time being the slowest
> varying dimension. ? ?Thus extraction of time series at different ADCP
> bins (depths), the most common operation by far, is efficient and
> fast.
>
> If storing as ?"timeSeriesProfile" means that the outer (slowest
> varying) dimension becomes depth instead of time, and that slows down
> the extraction of time series data a specified depths, that could be
> hard sell indeed (as Nan points out).
>
> -Rich
>
> On Wed, Dec 29, 2010 at 5:13 PM, Lowry, Roy K. <rkl at bodc.ac.uk> wrote:
>> Hi Nan,
>>
>> Let's consider how you're moored instrument data are mapped into NetCDF. ?If a given parameter is stored as a 2-D array with time as one dimension and instrument depth as the other then I would describe the data in that array as a 'profile series'. ? If they are stored as a set of 1-D vectors against a common time channel, then I would call them a 'time series'. ?It is perfectly possible to have a mixture of 'profile series' and 'time series' in a single file - such as 2-D current arrays and 1-D water temperature from an ADCP.
>>
>> So far I have considered the feature type as a property of individual variables, not the file as a whole. ?Is this the CF intention (it's been a long time since I read the proposal)? ?If so, I can't see Nan's problem. However, I think she is talking about file-level feature types, which we also need in our system to drive visualisation software. What I've done is to define our 'profile series' equivalent as a file containing one or more 2D arrays (plotted as contoured parameters) with zero or more 1D vectors (plotted as time series plots stacked with the contour plots on a common time x-axis). ?Our 'time series' is defined as one or more 1D vectors that are plotted as a stacked time series plot. In our working practice, these come from a single instrument, but providing all instruments have a common time channel this doesn't have to be so. Consequently, 'profile series' and 'time series' work for me as all I want to do is plot the data as discrete variables.
>>
>> However, Nan, I'm guessing that you have other use cases. ?It would be helpful for my understanding of what you need if you could give examples of the variables that would be in your files and the information that you expect to obtain from the file feature type. This should help clarify ?any extensions required to the feature type vocabulary.
>>
>> Cheers, Roy.
>> ________________________________________
>> From: cf-metadata-bounces at cgd.ucar.edu [cf-metadata-bounces at cgd.ucar.edu] On Behalf Of Nan Galbraith [ngalbraith at whoi.edu]
>> Sent: 29 December 2010 18:41
>> To: cf-metadata at cgd.ucar.edu
>> Subject: Re: [CF-metadata] CF feature types and definitions
>>
>> Bob's message reminds me that once these terms are agreed upon
>> they may be used for many purposes, across many different types
>> of systems, while a lot of the discussion of this convention has been
>> focused on just describing the shape of a chunk of data to be returned
>> by software queries.
>>
>> For that reason, I still feel strongly that 'timeSeries' data should be
>> defined in a way that allows data from moored instruments at multiple
>> depths in a single file. ?In the current version, the term 'location' in
>> the
>> definition of the timeSeries feature type is misleading; it can be taken
>> as x-y location, but I think you mean x-y-z. The definition needs to
>> be more specific, and I hope it will allow multiple depths.
>>
>> A time-consuming search of the email archive didn't turn up a specific
>> message, but I recall that in the past I've been told that moorings with
>> data at different depths should be classified as a collection of profiles.
>> While that might be acceptable in terms of defining the shape of the
>> data returned in a query, it will not be useful when these terms are used
>> in other ways.
>>
>> There's a big difference between a series of profiles at a single x-y
>> location
>> and a time series of data taken at the same x-y location by different
>> instruments at different depths. Not only are different variables measured
>> at different depths, the characteristics of the measurements can be
>> different -
>> instruments have different response times, resolution/accuracy, ranges,
>> etc.
>>
>> As I've said before, it would be a pretty hard sell to describe our station
>> time series data sets as collections of profiles; a terrific 2d time series
>> is not likely to be seen as a good collection of profiles.
>>
>> I hope we can find a better way to include data from moored buoys in
>> this vocabulary, without having to distort the data into something else.
>>
>> Thanks -
>> Nan
>>
>>
>>
>> On 12/23/10 11:06 AM, John Caron wrote:
>>> Attached is a message from Bob Simon at ERD/NOAA pointing out the
>>> inconsistencies in "data type" and "feature type" names in various
>>> Unidata related efforts. The almost-ready CF discrete sampling
>>> proposal has made a start at standardizing some of these names, and
>>> there is an interest, I think, between Steve, Jon and I to extend that
>>> to other types like grid. Essentially its a controlled vocabulary for
>>> classifying data.
>>>
>>> If this group is interested, I would propose a new ticket that would
>>> add probably an Appendix that would specify this vocabulary and their
>>> definitions. I anticipate it will be added to and clarified over time.
>>>
>>> John
>>>
>>> -------- Original Message --------
>>> Subject: ? ? ?Re: [netcdf-java] CDM names
>>> Date: ? ? ? ? Thu, 23 Dec 2010 08:48:41 -0700
>>> From: ? ? ? ? John Caron <caron at unidata.ucar.edu>
>>> To: ? netcdf-java at unidata.ucar.edu
>>>
>>>
>>>
>>> Hi Bob:
>>>
>>> Yes, you are right, there are too many forms of the "data type" and
>>> "feature type" names, with different lineages.
>>>
>>> 1) The CF discrete sampling proposal will be the recommended one for
>>> point data when thats finalized. Unfortunately, it will be somewhat
>>> different from whats gone before. The CF: prefix is dropped until the
>>> namespace proposal can be completed. So those feature types are now
>>> proposed to be:
>>>
>>> ? ? * *point*: one or more parameters measured at a set of points in
>>> ? ? ? time and space
>>> ? ? * *timeSeries*: a time-series of data points at the same location,
>>> ? ? ? with varying time
>>> ? ? * *trajectory*: a connected set of data points along a 1D curve in
>>> ? ? ? time and space
>>> ? ? * *profile*: a set of data points along a vertical line
>>> ? ? * *timeSeriesProfile*: a time-series of profiles at a named location
>>> ? ? * *trajectoryProfile*: a collection of profiles which originate
>>> ? ? ? along a trajectory
>>>
>>> The CDM will be backwards compatible, including:
>>>
>>> ? ? * ? accepting the CF: prefix
>>> ? ? * ? being case insensitive
>>> ? ? * ? "station" and "stationTimeSeries"as aliases for "timeSeries"
>>> ? ? * ? "stationProfile" as alias for "timeSeriesProfile"
>>> ? ? * ? "section" as alias for "trajectoryProfile"
>>>
>>> I know that CF wants to standardize on other feature types also. Its
>>> hard to anticipate what they will come with, but its likely:
>>>
>>> ? ? * grid
>>> ? ? * swath
>>>
>>> maybe:
>>>
>>> ? ? * image
>>> ? ? * radial
>>> ? ? * unstructuredGrid
>>>
>>> 2) The DataDiscoveryAttConvention is due for another round of work,
>>> esp in light of the ISO work that Ted and Dave have done. That might
>>> be a good opportunity to try to reconcile.
>>>
>>> 3) I will work on the CDM library to standardize.
>>>
>>> Thanks for bringing this up.
>>>
>>> On 12/22/2010 4:36 PM, Bob Simons wrote:
>>>> It is unfortunate that the CDM names listed at the sites below are
>>>> all slightly different (different sets of names, different names for
>>>> the same feature, different case).
>>>> And it is unfortunate that there are two global attributes to
>>>> identify the CDM feature/data type (#2 and #3 below).
>>>>
>>>> Is it possible that these could be standardized?
>>>>
>>>> 1)
>>>> http://www.unidata.ucar.edu/software/netcdf-java/v4.0/javadoc/index.html
>>>> ?CF.FeatureType ?(e.g., "stationTimeSeries")
>>>>
>>>> 2) The cdm_data_type global attribute:
>>>> http://www.unidata.ucar.edu/software/netcdf-java/formats/DataDiscoveryAttConvention.html
>>>> (e.g., "Station")
>>>> The UAF GeoIDE project is using this
>>>> https://nosc.ngdc.noaa.gov/dmc/swg/wiki/index.php?title=NetCDF_Attribute_Convention_for_Dataset_Discovery
>>>>
>>>>
>>>> 3) The proposed CF:featureType global attribute:
>>>> https://cf-pcmdi.llnl.gov/trac/wiki/PointObservationConventions
>>>> section 9.8.3 (e.g., "timeSeries")
>>>>
>>>> 4)The THREDDS dataType types
>>>> http://www.unidata.ucar.edu/projects/THREDDS/tech/catalog/InvCatalogSpec.html#dataType
>>>> (e.g., "Station")
>>>>
>>>> Thank you for looking into this.
>>>>
>>>> Sincerely,
>>>>
>>>> Bob Simons
>>
>> --
>> *******************************************************
>> * Nan Galbraith ? ? ? ? ? ? ? ? ? ? ? ?(508) 289-2444 *
>> * Upper Ocean Processes Group ? ? ? ? ? ?Mail Stop 29 *
>> * Woods Hole Oceanographic Institution ? ? ? ? ? ? ? ?*
>> * Woods Hole, MA 02543 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?*
>> *******************************************************
>>
>>
>>
>> _______________________________________________
>> CF-metadata mailing list
>> CF-metadata at cgd.ucar.edu
>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata--
>> This message (and any attachments) is for the recipient only NERC
>> is subject to the Freedom of Information Act 2000 and the contents
>> of this email and any reply you make may be disclosed by NERC unless
>> it is exempt from release under the Act. Any material supplied to
>> NERC may be stored in an electronic records management system.
>> _______________________________________________
>> CF-metadata mailing list
>> CF-metadata at cgd.ucar.edu
>> http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
>>
>
>
>
> --
> Dr. Richard P. Signell?? (508) 457-2229
> USGS, 384 Woods Hole Rd.
> Woods Hole, MA 02543-1598
>
--
Dr. Richard P. Signell?? (508) 457-2229
USGS, 384 Woods Hole Rd.
Woods Hole, MA 02543-1598
Received on Thu Dec 30 2010 - 05:45:46 GMT
