Hi Upendra and Jim,
Thanks for answering my questions about capabilities of the viewing sofware
e.g ncBrowse and also the ideas around aggregation.
A NCML enhancement to allow aggregation of different sized depth/pressure arrays
could
be very useful for ocean profile data from e.g. CTD, ARGO, Glider and XBT
instruments.
Regards,
Andrew
----- Original Message -----
From: Upendra Dadi
To: cf-metadata at cgd.ucar.edu
Sent: Wednesday, April 04, 2012 4:48 AM
Subject: Re: [CF-metadata] Ocean CTD data following CF Conventions v1.6
Hi,
I think Jim is talking about NCML (virtual) aggregation. THREDDS can
aggregate even when the variable is a scalar using "joinNew". But I think
THREDDS can aggregate only over regular arrays. That is, all the dimensions
other than over which a variable is aggregated should be of same size across all
the files. This is possible, for example, only if all the profiles have same
depth( or pressure) levels. Which in general is not true. NCML aggregation I
guess is of limited use when dealing with jagged arrays. But I agree with Jim in
that ability to aggregate should be an important consideration when coming up
with a representation. Physical aggregation is still possible. But I prefer
virtual aggregation, since letting data to be stored in individual files in
often operationally convenient, but users benefit from aggregated views. I
wonder if NCML could be extended to be able to aggregate jagged arrays into
incomplete multidimensional array representations. I heard that ERDDAP has
similar capability though I don't think it has anything like NCML where users
can create views over remote data, not sure though.
Upendra
On 4/3/2012 1:27 PM, Jim Biard wrote:
Hi.
A colleague of mine has told me that if you use size-1 array for a variable,
then data servers such as THREDDS can aggregate the variable over multiple files
and deliver a single file in which the variables will be arrays of size > 1. He
said that if a variable is a scalar, THREDDS won't do this. (I don't mess with
THREDDS, so I am just parroting what he said.)
If this is correct, then you might want to consider this point when deciding
which way to represent coordinates.
Grace and peace,
Jim
On Tue, Apr 3, 2012 at 1:02 PM, Upendra Dadi <upendra.dadi at noaa.gov> wrote:
Andrew,
However I have another small concern about all of this. Given there is more than
1 way to treat dimensions does the netCDF plotting and visualising sofware out
there understand both approaches. That is can we expect something like
ncBrowse, IDV or ODV? give me a plot of say salinity vs. pressure
and work OK with BOTH approaches to coding the netCDF?
Software like ncBrowse do not understand CF conventions. They can read netCDF
files, but wouldn't know how to interpret the attributes like "coordinates". So
if you want to plot the profile location on a map, for example, it wouldn't be
able to do it by itself. It would need to be told how to interpret the
coordinates. A CF based software wouldn't have to be supplied with the
additional semantics since it can read from the file by itself. But if you want
to plot salinity vs pressure, software like ncBrowse can already do it since
lot of these software make it easy to make plots based on a shared dimension.
And here salinity and pressure share the same dimension. So I guess, the
correct answer to your question is - it depends on what kind of plot or task you
want to do and if the software can understand CF conventions.
Upendra
Thanks and Regards All,
Andrew Walsh
----- Original Message -----
From: Lowry, Roy K.
To: Upendra Dadi ; andrew walsh
Cc: Luke Callcut ; cf-metadata at cgd.ucar.edu ; sdn2-tech at seadatanet.org
Sent: Tuesday, April 03, 2012 9:31 AM
Subject: RE: [CF-metadata] Ocean CTD data following CF Conventions v1.6
Hi Upendra,
I like the idea of a station dimension. It goes a long way to resolving the
issue raised in my response to Jim which was based on the tunnel vision of
having pressure/depth as a dimension. I have yet to look at the recently
published NODC NetCDF templates. Is this CTD encoding included in them? If so,
I'll bump up looking at them on my 'todo' list. I'd also recommend that Andrew
and my colleagues in SeaDataNet take a look.
Cheers, Roy.
From: cf-metadata-bounces at cgd.ucar.edu [cf-metadata-bounces at cgd.ucar.edu] On
Behalf Of Upendra Dadi [upendra.dadi at noaa.gov]
Sent: 02 April 2012 17:21
To: andrew walsh
Cc: Luke Callcut; cf-metadata at cgd.ucar.edu
Subject: Re: [CF-metadata] Ocean CTD data following CF Conventions v1.6
Hi Andrew,
Either way it should be okay as far as CF compliance is concerned. But the
dimensions - latitude, longitude and time are not really required. If it is
required to indicate that there is only one station(profile) in the file, there
could be a dimension for number of stations instead, with a value of 1. Also,
using a station dimension is the way to go if storing a collection of profiles
in a single file. Here at NODC, we took the approach that we would use the same
consistent representation whether there is a single instance or a collection in
a file.
Upendra
On Mon, Apr 2, 2012 at 2:51 AM, andrew walsh <awalsh at metoc.gov.au> wrote:
Hi CF lis
We are working on coding up some 1000's netCDF files off CTD
instruments and want to make usre we are following the
latest netCDF conventions (v1.6) OK. As background the CTD records
a profile pressure, temperature and salinity.
Here is a summarised CDL version (not all attributes+variables+qc flags there,
just majors for now)
of what we propose:
dimensions:
TIME=1
PRESSURE=729
LATITUDE=1
LONGITUDE=1
variables:
double TIME(TIME) ;
TIME:standard_name = "time" ;
TIME:units = "days since 1950-01-01 00:00:00Z" ;
TIME:axis = "T" ;
TIME:valid_min = 0. ;
TIME:valid_max = 999999. ;
double LATITUDE(LATITUDE) ;
LATITUDE:standard_name = "latitude" ;
LATITUDE:units = "degrees_north" ;
LATITUDE:axis = "Y" ;
LATITUDE:valid_min = -90. ;
LATITUDE:valid_max = 90. ;
double LONGITUDE(LONGITUDE) ;
LONGITUDE:standard_name = "longitude" ;
LONGITUDE:units = "degrees_east" ;
LONGITUDE:axis = "X" ;
LONGITUDE:valid_min = -180. ;
LONGITUDE:valid_max = 180. ;
double PRESSURE(PRESSURE) ;
PRESSURE:standard_name = "sea_water_pressure" ;
PRESSURE:units = "decibars" ;
PRESSURE:axis = "Z" ;
PRESSURE:valid_min = 0. ;
PRESSURE:valid_max = 12000. ;
PRESSURE:positive = "down" ;
double TEMPERATURE(PRESSURE) ;
TEMPERATURE:standard_name = "sea_water_temperature" ;
TEMPERATURE:units = "degrees_C" ;
TEMPERATURE:_FillValue = -99.99 ;
TEMPERATURE:valid_min = -2. ;
TEMPERATURE:valid_max = 40. ;
TEMPERATURE:coordinates="TIME LATITUDE LONGITUDE PRESSURE"
double SALINITY(PRESSURE) ;
SALINITY:standard_name = "sea_water_salinity" ;
SALINITY:units = "psu" ;
SALINITY:_FillValue = -99.99 ;
SALINITY:valid_min = 0. ;
SALINITY:valid_max = 40. ;
SALINITY:coordinates="TIME LATITUDE LONGITUDE PRESSURE"
// global attributes:
:conventions = "CF-1.6" ;
:featureType = "profile"
:cdm_data_type = "profile"
+ several other attributes later for ISO19115 metadata generation
I am not sure if I should have TEMPERATURE and SALINITY arrays with 4
dimensions
like TEMPERATURE(TIME,LATITUDE,LONGITUDE,PRESSURE) or just 1 dimension
like I have above i.e. TEMPERATURE(PRESSURE). ?
Any feedback on the above is greatly appreciated.
Andrew Walsh
_______________________________________________
Received on Tue Apr 03 2012 - 20:44:48 BST
