Dear all,
I have to reckon that I was a bit confused by the huge amount of
information received in the last few days. After reading them carefully,
below there is some comments in as a structured way as I found possible:
- I agree that the whole issue of dealing with ensembles, multi-models,
multi-forecast systems, operational systems, etc. is far from obvious. I
guess that is the reason why the job was not done before.
- Please, bear in mind that if I used the project ENSEMBLES as an
example, or Jamie the IPCC, it's only as an illustration. My intention
has been since the beginning to try to propose a solution to make NetCDF
more useful and attractive to **operational forecasting centres**. The
word operational is key here. It's not by chance that institutions such
as NCEP, UK Met Office, JMA or ECMWF do not disseminate their daily
operational ensemble products in NetCDF format.
- My original intention when started this discussion was to find a clear
way of writing files with multi-forecast system ensemble integrations.
In response to John's message, this means being able to write all the
members of an ensemble (coming from either several forecast systems or
from a single one) in a single file and being able to identify them. Of
course, the type of file may change with the user: in the context of a
multi-model, the contributors might prefer to prepare files with the
ensemble forecasts produced with their own individual models if they run
them in separate institutions, while in a perturbed-physics context a
single institution will prefer to write the forecasts from the different
model versions in a single file.
- Bryan wonders about the adequacy of defining specific variable
metadata in multi-forecast system files to distinguish ensemble member,
model, system and so on, and compares this problem with the interest of
having additional informative metadata in a file containing station
data. In my opinion, the essential difference is that a file with
multi-forecast system ensemble simulations is not a simple gathering of
predictions from different sources or with different initial conditions,
but a complete forecast in itself. The additional metadata is not a
wish, but a need to describe an entity. Without the appropriate metadata
the file is not self-descriptive and won't be operationally
disseminated, in the same way that operational centres don't disseminate
deterministic forecasts in any format without clearly specifying the
forecast system.
- I understood at the beginning that the main interest of NetCDF is that
one doesn't need to rely on external tables to identify the data. While
the use of external vocabularies might offer a simple way of avoiding
the creation of additional metadata, it uses a strategy that I presumed
made the difference between NetCDF and GRIB: the use of external tables
and extensions. In my opinion, Bryan is right: the core of the
discussion is issue D. However, from my experience with operational
ensembles, forecasters may make the move from GRIB1 to GRIB2 without
paying much attention to the use of NetCDF due the lack of rules to
encode the operational forecasts in NetCDF. Maybe WMO/IPCC should take
the lead on this.
- Of course, an immediate issue, as John and Bryan have identified, is
how to disseminate these files using a THREDDS Data Server. At present,
the largest file I can imagine would have as dimensions
(realization,forecast_reference_time,forecast_period,lev,lat,lon). The
most likely integrations, in my opinion, would take place on the
dimensions realization and forecast_reference_time. In ENSEMBLES, we
plan to work with NetCDF, although we try to learn from what is done
with GRIB2 in TIGGE.
- In case the option of adding standard names (instead of the use of
external vocabularies) is accepted, I'd like to propose the standard
name "ensemble_member" in lieu of the suggested "initial_condition". It
could be used as an integer to tag the members of the ensemble
identified with particular values of "experiment_id", "source",
"institution", regardless of the way the initial conditions of the
ensemble have been generated.
Apologies for the long message.
Best regards,
Paco
--
________________________________________
Francisco J. Doblas-Reyes
European Centre for Medium-Range
Weather Forecasting (ECMWF)
Shinfield Park, RG2 9AX
Reading, UK
Tel: +44 (0)118 9499 655
Fax: +44 (0)118 9869 450
f.doblas-reyes at ecmwf.int
_______________________________________
Received on Tue Oct 31 2006 - 00:42:23 GMT
