Hello Roy and All,
I am involved in Argo data-management, OceanSITES and the MyOcean EU
project.
These 3 projects are intensive users of NetCDF CTD data files.
Within these communities :
? The natural vertical reference for a CTD is pressure. This is
also true for Argo floats, gliders or sea-mammals.
? The natural vertical reference for XBTs, moorings and buoys is
depth.
So the NetCDF files vertical reference can be either pressure or depth, but
not both.
If another community wants an homogeneous vertical reference, a profile with
additional data (pressure or depth) ?could? be provided, as a product.
BUT, this adds complexity; I feel more comfortable with data reported as
naturally as possible :
? If you add a vertical reference, you have to decide which one is
the Z axis (PRES:axis = "Z" or DEPH:axis = "Z" attribute).
? All the QC flags assigned to pressure should be transferred to
depth; but this is not straightforward.
If the salinity is bad or spiky, from a good pressure you will have a bad
depth.
Here are links to different NetCDF user?s manual managing vertical profiles
with pressure or depth (but not both) vertical axes:
? Argo documentation <
http://www.argodatamgt.org/Documentation> ,
the user?s manual
<
http://www.argodatamgt.org/content/download/12096/80327/file/argo-dm-user-m
anual.pdf> : we are heading toward 1 million profiles
? OceanSITES <
http://www.oceansites.org/data/index.html> , the
user?s manual
<
http://www.oceansites.org/docs/oceansites_user_manual_version1.2.pdf> :
1400 long time-series from mooring sites
The EU MyOcean project <
http://www.myocean.eu.org/> adopted the OceanSITES
implementation of NetCDF CF for oceanographic in-situ data.
The last 3 years of in-situ observations are available in OceanSITES NetCDF
CF files (see attached map of 3.3 million vertical profiles).
This format is also used for EGO gliders
<
http://ego.dt.insu.cnrs.fr/dokuwiki/doku.php?id=start> , where pressure is
the vertical axis.
Example : Pheidippides glider data
<
http://www.ifremer.fr/co/ego/ego/pheidippides/> (NetCDF OceanSITES
vertical profiles) operating now around Cyprus island.
The above user?s manuals for NetCDF-CF CTDs (and others) files may be
enhanced with this ongoing v1.6 proposal.
Best regards,
Thierry
-----Message d'origine-----
De : cf-metadata-bounces at cgd.ucar.edu
[mailto:cf-metadata-bounces at cgd.ucar.edu] De la part de Lowry, Roy K.
Envoy? : mardi 3 avril 2012 12:56
? : Lowry, Roy K.; andrew walsh
Cc : Luke Callcut; cf-metadata at cgd.ucar.edu; Upendra Dadi;
sdn2-tech at seadatanet.org; greg at metoc.gov.au
Objet : Re: [CF-metadata] Ocean CTD data following CF Conventions v1.6
Hello again,
It has been pointed out to me that in the current SeaDataNet format
specification (written and updated by me!) that the initial position of
insisting upon depth for the z co-ordinate was relaxed to allow either
pressure or depth. So, I guess we need to be relaxed and leave z co-ordinate
harmonisation to the the application tools.
Yet another 'senior moment'......
Cheers, Roy.
________________________________________
From: cf-metadata-bounces at cgd.ucar.edu [cf-metadata-bounces at cgd.ucar.edu] On
Behalf Of Lowry, Roy K. [rkl at bodc.ac.uk]
Sent: 03 April 2012 06:46
To: andrew walsh
Cc: cf-metadata at cgd.ucar.edu; sdn2-tech at seadatanet.org; greg at metoc.gov.au;
Luke Callcut; Upendra Dadi
Subject: Re: [CF-metadata] Ocean CTD data following CF Conventions v1.6
Hi again,
The reason that I prefer depth to pressure as the dimension for CTD is that
the information required for pressure to depth conversion is virtually
always present in the CTD data. However, in other oceanographic profiles -
say nutrient bottle data - it is quite possible to have depth present
without temperature and salinity, making the conversion to pressure
impossible. Calculating depth from pressure at the time of standardised
formatting of CTD data is as you say is trivial. So, why not make the CTD
data more compatible with bottle data for very little cost?
SeaDataNet already mandate inclusion of depth in their other data formats
for CTD data delivery and so I can't see them switching to pressure for the
dimension in NetCDF.
Cheers, Roy.
________________________________________
From: andrew walsh [awalsh at metoc.gov.au]
Sent: 03 April 2012 06:15
To: Lowry, Roy K.
Cc: Jim Biard; Upendra Dadi; cf-metadata at cgd.ucar.edu; Luke Callcut;
greg at metoc.gov.au; sdn2-tech at seadatanet.org
Subject: Re: [CF-metadata] Ocean CTD data following CF Conventions v1.6
Hi Roy/All,
Agree totally it would be good to get this CTD netCDF done in an
interoperable way.
Regarding having pressure vs. depth. We liked to use pressure because:
1) It is the thing that is measured, let us store just that and calculate
depth if needed.
2) Depth can later be easily be calculated on the fly using the 'Pressure to
Depth' algorithm in "UNESCO (1983) Techinical Papers in Marine Science 44,
Algorithms for Computation of fundamental properties of Seawater. One can
use the python seawater library (see
http://packages.python.org/seawater/ )
and seawater.csiro.depth(p, lat) to get depth from pressure and
seawater.csiro.pres(depth, lat) to get pressure from depth.
3) I noted that the ARGO project (10000's CTD like profiles) and others like
CSIRO Oceanography in Aust. make data available with just pressure.
4) It makes our processing and QC a whole lot simpler. We don't have to
worry about calculating and managing the extra 'depth' variable.
Is there any problem with having the "pressure" as a co-ordinate which
isn't really a "dimensional"
quantity like depth (z) in the 4-D sense i.e x,y,z,t ?
However I note that pressure (decibar) is allowed as a vertical axis e.g see
section 4.3. Vertical (Height or Depth) Coordinate of CF v1.6 conventions
document which
says:
"A vertical coordinate will be identifiable by:
. units of pressure; or
. the presence of the positive attribute with a value of up or down (case
insensitive).
"
AND
section 4.3.1. "Dimensional Vertical Coordinate" which says:
"The units attribute for dimensional coordinates will be a string formatted
as per the udunits.dat3 file. The acceptable units for vertical (depth or
height) coordinate variables are:
. units of pressure as listed in the file udunits.dat. For vertical axes the
most commonly used of these include include bar, millibar, decibar,
atmosphere (atm), pascal (Pa), and hPa.
...
..."?
Regards,
Andrew
----- Original Message -----
From: Lowry, Roy K.
To: andrew walsh
Cc: Jim Biard ; Upendra Dadi ; cf-metadata at cgd.ucar.edu ; Luke Callcut ;
greg at metoc.gov.au ; sdn2-tech at seadatanet.org
Sent: Tuesday, April 03, 2012 2:16 PM
Subject: RE: [CF-metadata] Ocean CTD data following CF Conventions v1.6
Hi Andrew,
SeaDataNet will very soon be considering how it is going to encode data,
including single CTD casts, in CF-compliant NetCDF and so I think the time
is ripe for agreeing how the significant numbers of us who indulge in this
practice for whatever reason do it. That way we'll end up with
interoperable data.
I think there are a number of people on this list who have already encoded
single CTDs into NetCDF and so it would be useful to start by asking for
descriptions (like Andrew's examples) of how this has been done and what
tools are dependent upon that encoding.
The z co-ordinate parameter (pressure/depth) is also an issue worth
resolving.
Whilst interconversions are relatively straightforward, agreement would make
life much easier. My preference leans dowards having depth as the dimension
with pressure as an optional variable. That way we interoperate with other
kinds of oceanographic profile data such as bottle data.
If we can get that far, we can then look at how to aggregate multiple CTDs
into a file according to the CF point data conventions.
Cheers, Roy.
From: andrew walsh [awalsh at metoc.gov.au]
Sent: 03 April 2012 04:39
To: Lowry, Roy K.
Cc: Jim Biard; Upendra Dadi; cf-metadata at cgd.ucar.edu; Luke Callcut;
greg at metoc.gov.au
Subject: Re: [CF-metadata] Ocean CTD data following CF Conventions v1.6
Hello Roy, Upendra, Jim and CF list,
Thanks for all your feedbacks.
My proposal relates to single CTD profile data (a vertical profile of
pressure, Temp. Salinity) not a trajectory in x,y,z,t and I have put
:featureType = "profile" as recommended in the global attributes section. As
Roy has mentioned to Jim CTD data is usually processed and QC'ed as a single
profile per netCDF file so that's why I am doing it this way.
Aggregations using multiple CTD profiles per netCDF file may be constructed
later at say national/international data centres and these aggregrations
would follow the CF conventions v1.6, Chapter 9 - Discrete Sampling
geometries and also the new netCDF templates provided by NODC, thanks NODC
-:)
Roy,
The recent NODC netCDF templates don't have an aggregation example for CTD
however the "Profile/World Ocean Database Observed Level" example comes
close (see <
http://www.nodc.noaa.gov/data/formats/netcdf/>
http://www.nodc.noaa.gov/data/formats/netcdf/) and click on Profile/World
Ocean Database Observed Level link. This example appears to be for ocean
station/bottle samples with vertical dimension of depth (z) (m) rather than
case of CTD which would use pressure (dbar) as the vertical (z) dimension.
It would be useful I think to have a NODC netCDF template for an aggregation
of CTD casts.
Upendra,
Based on your responses and what I have seen at NODC and other places it
seems there are 2 methods to do this:
1) You can have dimensions for lat, long time i.e an array with a single
value. That is:
dimensions:
TIME=1
PRESSURE=729
LATITUDE=1
LONGITUDE=1
variables:
double TIME(TIME) ;
+ std. attrs.
double LATITUDE(LATITUDE) ;
+ std. attrs.
double LONGITUDE(LONGITUDE) ;
+ std. attrs.
double PRESSURE(PRESSURE) ;
+ std. attrs.
double TEMPERATURE(PRESSURE) ;
+ std. attrs.
TEMPERATURE:coordinates="TIME LATITUDE LONGITUDE PRESSURE"
double SALINITY(PRESSURE) ;
+ std. attrs.
SALINITY:coordinates="TIME LATITUDE LONGITUDE PRESSURE"
(2) Other way is to have Lat, long and time are single valued scalars:
dimensions:
PRESSURE=729
variables:
double LATITUDE ;
+ std attrs
double LONGITUDE ;
+ std attrs
double TIME ;
+std attrs
double PRESSURE(PRESSURE) ;
+ std. attrs.
double TEMPERATURE(PRESSURE) ;
+ std. attrs.
TEMPERATURE:coordinates="TIME LATITUDE LONGITUDE PRESSURE"
double SALINITY(PRESSURE) ;
+ std. attrs.
SALINITY:coordinates="TIME LATITUDE LONGITUDE PRESSURE"
I can see Method 2 above is much simpler and I would prefer to use this.
However I have another small concern about all of this. Given there is more
than
1 way to treat dimensions does the netCDF plotting and visualising sofware
out there understand both approaches. That is can we expect something like
ncBrowse, IDV or ODV? give me a plot of say salinity vs. pressure and work
OK with BOTH approaches to coding the netCDF?
Thanks and Regards All,
Andrew Walsh
----- Original Message -----
From: Lowry, Roy K.
To: Upendra Dadi ; andrew walsh
Cc: Luke Callcut ; cf-metadata at cgd.ucar.edu ; sdn2-tech at seadatanet.org
Sent: Tuesday, April 03, 2012 9:31 AM
Subject: RE: [CF-metadata] Ocean CTD data following CF Conventions v1.6
Hi Upendra,
I like the idea of a station dimension. It goes a long way to resolving the
issue raised in my response to Jim which was based on the tunnel vision of
having pressure/depth as a dimension. I have yet to look at the recently
published NODC NetCDF templates. Is this CTD encoding included in them? If
so, I'll bump up looking at them on my 'todo' list. I'd also recommend that
Andrew and my colleagues in SeaDataNet take a look.
Cheers, Roy.
From: cf-metadata-bounces at cgd.ucar.edu [cf-metadata-bounces at cgd.ucar.edu] On
Behalf Of Upendra Dadi [upendra.dadi at noaa.gov]
Sent: 02 April 2012 17:21
To: andrew walsh
Cc: Luke Callcut; cf-metadata at cgd.ucar.edu
Subject: Re: [CF-metadata] Ocean CTD data following CF Conventions v1.6
Hi Andrew,
Either way it should be okay as far as CF compliance is concerned. But the
dimensions - latitude, longitude and time are not really required. If it is
required to indicate that there is only one station(profile) in the file,
there could be a dimension for number of stations instead, with a value of
1. Also, using a station dimension is the way to go if storing a collection
of profiles in a single file. Here at NODC, we took the approach that we
would use the same consistent representation whether there is a single
instance or a collection in a file.
Upendra
On Mon, Apr 2, 2012 at 2:51 AM, andrew walsh < <mailto:awalsh at metoc.gov.au>
awalsh at metoc.gov.au> wrote:
Hi CF lis
We are working on coding up some 1000's netCDF files off CTD instruments and
want to make usre we are following the latest netCDF conventions (v1.6) OK.
As background the CTD records a profile pressure, temperature and salinity.
Here is a summarised CDL version (not all attributes+variables+qc flags
there, just majors for now) of what we propose:
dimensions:
TIME=1
PRESSURE=729
LATITUDE=1
LONGITUDE=1
variables:
double TIME(TIME) ;
TIME:standard_name = "time" ;
TIME:units = "days since 1950-01-01 00:00:00Z" ; TIME:axis = "T" ;
TIME:valid_min = 0. ; TIME:valid_max = 999999. ;
double LATITUDE(LATITUDE) ;
LATITUDE:standard_name = "latitude" ;
LATITUDE:units = "degrees_north" ;
LATITUDE:axis = "Y" ;
LATITUDE:valid_min = -90. ;
LATITUDE:valid_max = 90. ;
double LONGITUDE(LONGITUDE) ;
LONGITUDE:standard_name = "longitude" ; LONGITUDE:units = "degrees_east" ;
LONGITUDE:axis = "X" ; LONGITUDE:valid_min = -180. ; LONGITUDE:valid_max =
180. ;
double PRESSURE(PRESSURE) ;
PRESSURE:standard_name = "sea_water_pressure" ; PRESSURE:units = "decibars"
; PRESSURE:axis = "Z" ; PRESSURE:valid_min = 0. ; PRESSURE:valid_max =
12000. ; PRESSURE:positive = "down" ;
double TEMPERATURE(PRESSURE) ;
TEMPERATURE:standard_name = "sea_water_temperature" ; TEMPERATURE:units =
"degrees_C" ; TEMPERATURE:_FillValue = -99.99 ; TEMPERATURE:valid_min =
-2. ; TEMPERATURE:valid_max = 40. ; TEMPERATURE:coordinates="TIME LATITUDE
LONGITUDE PRESSURE"
double SALINITY(PRESSURE) ;
SALINITY:standard_name = "sea_water_salinity" ; SALINITY:units = "psu" ;
SALINITY:_FillValue = -99.99 ; SALINITY:valid_min = 0. ;
SALINITY:valid_max = 40. ; SALINITY:coordinates="TIME LATITUDE LONGITUDE
PRESSURE"
// global attributes:
:conventions = "CF-1.6" ;
:featureType = "profile"
:cdm_data_type = "profile"
+ several other attributes later for ISO19115 metadata generation
I am not sure if I should have TEMPERATURE and SALINITY arrays with 4
dimensions like TEMPERATURE(TIME,LATITUDE,LONGITUDE,PRESSURE) or just 1
dimension like I have above i.e. TEMPERATURE(PRESSURE). ?
Any feedback on the above is greatly appreciated.
Andrew Walsh--
This message (and any attachments) is for the recipient only. NERC is
subject to the Freedom of Information Act 2000 and the contents of this
email and any reply you make may be disclosed by NERC unless it is exempt
from release under the Act. Any material supplied to NERC may be stored in
an electronic records management system.
_______________________________________________
CF-metadata mailing list
<mailto:CF-metadata at cgd.ucar.edu> CF-metadata at cgd.ucar.edu
<
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
_______________________________________________
CF-metadata mailing list
<mailto:CF-metadata at cgd.ucar.edu> CF-metadata at cgd.ucar.edu
<
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata>
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
-------------- next part --------------
A non-text attachment was scrubbed...
Name: myo-global-200904-station.png
Type: image/png
Size: 735401 bytes
Desc: not available
URL: <
http://mailman.cgd.ucar.edu/pipermail/cf-metadata/attachments/20120403/e24a6480/attachment-0001.png>
Received on Tue Apr 03 2012 - 12:31:02 BST
